+mobile-l

mobile-l recipients, if replying, if you would please reply-all in case any people on the CC: line aren't on mobile-l, it would be appreciated.

Nik,

Thanks for the update. Glad to hear there's even faster performance coming and also that there's no need to structure too much fallback stuff depending on whether the reflex time is okay. With any luck, it would be just fast enough. I don't think there'd be too much hammering on the suggest term; only if the resultset is insufficient does it seem like it would make sense to orchestrate the client side (or server side, for that matter) call. The apps do have a key tap timer thing on them to help avoid spurious searching, so that should help. I think I understand the ellipse related stuff - parsing the snippet text is no problem, but if there's an even simpler way to get text condensed to the point where there's no work to avoid wrapping on most form factors, cool!...and if I misunderstood, well, we'll get to the bottom of that on Friday.

-Adam


On Tue, Apr 1, 2014 at 3:53 PM, Dan Garry <dgarry@wikimedia.org> wrote:
I don't want to bloat the meeting into something massive, but I did invite Kenan and Howie since we're going to be talking about product consistency and that's something that should involve them.

Thanks for setting this up, Jared!

Dan


On 1 April 2014 15:05, Jared Zimmerman <jared.zimmerman@wikimedia.org> wrote:
…but how about setting up a google hangout or something?

done.



Jared Zimmerman  \\  Director of User Experience \\ Wikimedia Foundation               


On Tue, Apr 1, 2014 at 3:00 PM, Nikolas Everett <neverett@wikimedia.org> wrote:



On Tue, Apr 1, 2014 at 5:13 PM, Adam Baso <abaso@wikimedia.org> wrote:
My email got a little buried in the thread.

You guys on mobile-l? It would be nice to bring the conversation there if possible. Understood if not; maybe we can get mobile-tech and any other necessary lists here in that case?

I imagine the right thing is to add them to the email chain and we'll all keep reply-alling.

During the mobile quarterly planning kickoff this morning, I mentioned that I had started on a patchset for iOS and that I think it would be cool if we could try this first in apps, then hopefully roll to mobile web (to ease into load, but also to learn on any other fronts we haven't considered). Here's the WIP patchset.


See in particular the comments in the first code file in that patchset for some of my thought process.

Chad, that patchset is the thing I was talking about the other day for list=search.

It queries like the following:


It would be really neat to make the app the first place in a user's mind where s/he's going to search for factual information even when doing so via unstructured search terms. I think for people without the app, they will of course always go through more conventional channels to enter queries that aren't perfectly structured for title-starts-with; my  hope is that if we give them this goodie early on they'll be pleasantly surprised and see it as a good reason to use the official apps.


Sounds good to me.
 
Sounds like we may need to reconcile caching and general load performance items when using CirrusSearch for the backend...although if it's possible to do this fulltext magic by default on just the apps to start, without making CirrusSearch come to a halt (!), that would be totally sweet.

So prefix search is on the order of 4-5 ms for Elasticsearch to service, and it is cached.  Full text search varies from 30ms to 500ms for "acceptable" performance.  Not great, but ok.

Some queries take even longer but we're working on speeding it up.  On Thursday I'm pushing something that'll save about 25% off of particularly slow queries.  We'll get another 20%ish on top of that when we upgrade to Elasticsearch 1.1.0 next week because that'll bring to bear some work I did upstream a few weeks ago.  We'll also be able to start using some work I did back in January that can cut really nasty queries by orders of magnitude, but we'll need to make some cirrus changes for that so it'll probably hit enwiki in a few weeks.  So, yeah, we're working on it.

But the upshot is we'll have to be real careful if we want actual full text search to be fast enough for find as you type.  We can save a bunch of time by not running the "did you mean" if you don't use them.  Beyond that we'd have to look at things like the phrase match boost and highlighting the results text.  You may want to be even more careful about the number of requests you search for because once you start cutting to the bone highlighting the results starts to show up (20ms for 50 results normally, can get higher).

One idea, although less than ideal just from a coding perspective (especially if perf is not an issue), would be to make the client-side do lag detection or to observe a server-issued feature flag (there will be several of these for the app already), or both, such that if lag is unacceptable client side it would fallback to opensearch.

Probably not worth it initially.  Maybe a good idea to keep in our back pocket if we find it is just too slow.

 
I don't have it in there yet for the GUI rendering (I was just working in the confines of the existing iOS code to see how it would play), but I was thinking to put the snippet text in a smaller font below the title text in this iOS POC to help the user have a little bit more context about why a result came back...that's helpful particularly if the page title in the result set isn't obviously related to the search stem or expansion; as you know! So instead of just

    San Francisco

it would instead look like, say

    San Francisco
    ...San Francisco City and county City and County...

The client-side code could even try to opportunistically slice the snippet text in some sensible fashion to try to provide reasonable context without wrapping text, and if that fails, just start from the beginning and add the ellipsis as appropriate to not wrap the result item's snippet text to the next line.

We should talk more about this.  I've spent a bunch of time over the past two weeks working on a better highlighter then the one we are using now.  It'll be faster and require less disk space.  I wonder how stupid an idea it'd be to try to highlighter within a pixel size with a certain font.....
 

Any ideas if this is achievable? Fulltext search feels so much more natural. I guess there's maybe also this notion of search within title (it does look like srwhat=title is currently disabled for the CirrusSearch provider, at least for API), with a suggest term backing (ideally, the API would just magically augment results with suggest term autolookup, but the orchestration is obviously possible client side, too) to help deal with misspelling, which is even more likely on the mobile app.

Okay, hope that helps a bit.

Any ideas for short, medium, and longer term approach?

Got to go, but how about setting up a google hangout or something?
 

-Adam




On Tue, Apr 1, 2014 at 1:21 PM, Adam Baso <abaso@wikimedia.org> wrote:
Let me reply-all in a couple minutes.


On Tue, Apr 1, 2014 at 1:15 PM, Chad Horohoe <chorohoe@wikimedia.org> wrote:
On Tue, Apr 1, 2014 at 1:10 PM, Jared Zimmerman <jared.zimmerman@wikimedia.org> wrote:
I think that the mobile app is returning results that match user expectations (in this case RESULTS rather than NO RESULTS) so I'd urge the team to figure out how to resolve this issue even if there are technological or performance issues to overcome. 



That is not consistent with how the search box has ever worked. It's meant to
be a suggestion for page titles, not a a list of full-text results (that may contain
nothing in common between their title and what you typed). Once you complete
the search (if you don't end on a direct title match), you'll get the full-text results.

If the mobile app is presenting full-text results as suggestions I'd say that's the
wrong way to go. I'll also note our behavior is consistent with how Google works as well.

-Chad 







--
Dan Garry
Associate Product Manager for Platform
Wikimedia Foundation