*...but how about setting up a google hangout or
something?*
done.
*Jared Zimmerman * \\ Director of User Experience \\ Wikimedia
Foundation
M : +1 415 609 4043 | : @JaredZimmerman<https://twitter.com/JaredZimmerman>
On Tue, Apr 1, 2014 at 3:00 PM, Nikolas Everett <neverett(a)wikimedia.org>wrote;wrote:
On Tue, Apr 1, 2014 at 5:13 PM, Adam Baso <abaso(a)wikimedia.org> wrote:
My email got a little buried in the thread.
You guys on mobile-l? It would be nice to bring the conversation there
if possible. Understood if not; maybe we can get mobile-tech and any other
necessary lists here in that case?
I imagine the right thing is to add them to the email chain and we'll
all keep reply-alling.
During the mobile quarterly planning kickoff this morning, I mentioned
that I had started on a patchset for iOS and that I think it would be cool
if we could try this first in apps, then hopefully roll to mobile web (to
ease into load, but also to learn on any other fronts we haven't
considered). Here's the WIP patchset.
https://gerrit.wikimedia.org/r/#/c/121562/
See in particular the comments in the first code file in that patchset
for some of my thought process.
Chad, that patchset is the thing I was talking about the other day for
list=search.
It queries like the following:
https://en.m.wikipedia.org/w/api.php?action=query&list=search&srsea…
It would be really neat to make the app the first place in a user's
mind where s/he's going to search for factual information even when doing
so via unstructured search terms. I think for people without the app, they
will of course always go through more conventional channels to enter
queries that aren't perfectly structured for title-starts-with; my hope is
that if we give them this goodie early on they'll be pleasantly surprised
and see it as a good reason to use the official apps.
Sounds good to me.
Sounds like we may need to reconcile caching and
general load
performance items when using CirrusSearch for the backend...although if
it's possible to do this fulltext magic by default on *just* the apps
to start, without making CirrusSearch come to a halt (!), that would be
totally sweet.
So prefix search is on the order of 4-5 ms for Elasticsearch to service,
and it is cached. Full text search varies from 30ms to 500ms for
"acceptable" performance. Not great, but ok.
Some queries take even longer but we're working on speeding it up. On
Thursday I'm pushing something that'll save about 25% off of particularly
slow queries. We'll get another 20%ish on top of that when we upgrade to
Elasticsearch 1.1.0 next week because that'll bring to bear some work I did
upstream a few weeks ago. We'll also be able to start using some work I
did back in January that can cut really nasty queries by orders of
magnitude, but we'll need to make some cirrus changes for that so it'll
probably hit enwiki in a few weeks. So, yeah, we're working on it.
But the upshot is we'll have to be real careful if we want actual full
text search to be fast enough for find as you type. We can save a bunch of
time by not running the "did you mean" if you don't use them. Beyond that
we'd have to look at things like the phrase match boost and highlighting
the results text. You may want to be even more careful about the number of
requests you search for because once you start cutting to the bone
highlighting the results starts to show up (20ms for 50 results normally,
can get higher).
One idea, although less than ideal just from a coding perspective
(especially if perf is not an issue), would be to
make the client-side do
lag detection or to observe a server-issued feature flag (there will be
several of these for the app already), or both, such that if lag is
unacceptable client side it would fallback to opensearch.
Probably not worth it initially. Maybe a good idea to keep in our back
pocket if we find it is just too slow.
I don't have it in there yet for the GUI
rendering (I was just working
in the confines of the existing iOS code to see how it would play), but I
was thinking to put the snippet text in a smaller font below the title text
in this iOS POC to help the user have a little bit more context about
*why* a result came back...that's helpful particularly if the page
title in the result set isn't obviously related to the search stem or
expansion; as you know! So instead of just
San Francisco
it would instead look like, say
San Francisco
...San Francisco City and county City and County...
The client-side code could even try to opportunistically slice the
snippet text in some sensible fashion to try to provide reasonable context
without wrapping text, and if that fails, just start from the beginning and
add the ellipsis as appropriate to not wrap the result item's snippet text
to the next line.
We should talk more about this. I've spent a bunch of time over the
past two weeks working on a better highlighter then the one we are using
now. It'll be faster and require less disk space. I wonder how stupid an
idea it'd be to try to highlighter within a pixel size with a certain
font.....
Any ideas if this is achievable? Fulltext search feels so much more
natural. I guess there's maybe also this notion of search within title (it
does look like srwhat=title is currently disabled for the CirrusSearch
provider, at least for API), with a suggest term backing (ideally, the API
would just magically augment results with suggest term autolookup, but the
orchestration is obviously possible client side, too) to help deal with
misspelling, which is even more likely on the mobile app.
Okay, hope that helps a bit.
Any ideas for short, medium, and longer term approach?
Got to go, but how about setting up a google hangout or something?
-Adam
On Tue, Apr 1, 2014 at 1:21 PM, Adam Baso <abaso(a)wikimedia.org> wrote:
> Let me reply-all in a couple minutes.
>
>
> On Tue, Apr 1, 2014 at 1:15 PM, Chad Horohoe <chorohoe(a)wikimedia.org>wrote;wrote:
>
>> On Tue, Apr 1, 2014 at 1:10 PM, Jared Zimmerman <
>> jared.zimmerman(a)wikimedia.org> wrote:
>>
>>> I think that the mobile app is returning results that match user
>>> expectations (in this case RESULTS rather than NO RESULTS) so I'd urge
the
>>> team to figure out how to resolve this issue even if there are
>>> technological or performance issues to overcome.
>>>
>>>
>>>
>> That is not consistent with how the search box has ever worked. It's
>> meant to
>> be a suggestion for page titles, not a a list of full-text results
>> (that may contain
>> nothing in common between their title and what you typed). Once you
>> complete
>> the search (if you don't end on a direct title match), you'll get the
>> full-text results.
>>
>> If the mobile app is presenting full-text results as suggestions I'd
>> say that's the
>> wrong way to go. I'll also note our behavior is consistent with how
>> Google works as well.
>>
>> -Chad
>>
>
>