As we reach the last month of the quarter, it's a good opportunity for us
to reflect on where we want to go for the last part of our remaining time.
On the one hand, we're in quite a good place. We're just wrapping up our
work on our Q2 goal for search
which is excellent! On the other hand, the test showed minimal impact, so
our users still aren't seeing the impact of our work. Since we can continue
running A/B tests for improving language support relatively cheaply in
terms of required engineering time, let's take a look back at what we've
done previously and see if we can choose something high impact to work on!
The completion suggester is a very promising avenue for us to invest in. As
noted in our analysis of the initial test
<https://phabricator.wikimedia.org/T111858>, using the completion suggester
instead of prefixsearch significantly reduced the zero results rate. We've
not had an impact on this through other efforts, so this is interesting! In
order to more thoroughly test the suggester, we can make it a Beta Feature
<https://phabricator.wikimedia.org/T119535>. This will allow editors to
opt-in to testing it, and will gather us valuable qualitative feedback
about what use cases the completion suggester could support better. The
caveat, of course, is that the feedback will be from a specific segment of
our user base (users who test beta features) which is more specialised than
the intended audience (everyone). That said, the feedback will still be
very helpful. There's quite a bit of work to do here; our initial test of
the suggester was very hacky, but now that it's proven itself, we can be
The other avenue is using page views to influence result ranking. This is
in an earlier stage thant he completion suggester, in that it's a
relatively unproven approach for us, but it's something that's logical and
that we've been interested in for a while. But, we've repeatedly had to
deprioritise it for other work. If something is popular, it makes sense to
rank it up in search results. Obviously, we do not want to be *too* aggressive
with this in case we create feedback loops, but I think the potential
benefits are quite clear if done correctly.
I explained a lot of this more briefly in our last standup, but hopefully
this should give you all some guidance on where we're going.
Thanks, and as always, if there are any questions then please let me know.
Lead Product Manager, Discovery
A reminder to all of you who use google calendar to track your work events:
Please keep it updated, especially for the next month and a half.
1. For days you are taking off (holiday, vacation, or other), please create
an all-day event indicating that. Be sure to edit it to "busy", because
all-day events default to "available", meaning they are invisible when
someone is considering scheduling you into a meeting.
1b. For extra protection, you can also create yourself a meeting that spans
your entire workday. That makes it even more obvious that you will be out,
so makes it less likely that someone will accidentally schedule you into
something. This extra step is totally optional. (I don't do it myself.)
2. Please RSVP accurately to meetings, with either a yes or no (or maybe).
It's always very helpful to know who is planning to attend, but it's even
more important in this season of taking time off, because some meetings
can/should be canceled or rescheduled if enough people will miss them.
3. Don't forget to indicate your unavailability while traveling as well, by
blocking off entire days (remember to mark them "busy"), or specific chunks
of time. You can set those "private" to avoid exposing your specific travel
Thanks, and happy winter to all!
Agile Coach, Wikimedia Foundation
The notes from our monthly Discovery retrospective have been posted. The
action items were:
* Dan: write a goal for improving the UX of the search page on-wiki
* Dan?: Discussion of improving the relevance/sorting of results rather
than just zero results rate
* Moiz: Talk about whether we really can run A/B tests on the portal, since
it's not subject to a deployment freeze
* Dan: Follow up on the common terms query A/B test
* David: Look into listing features that affected the results set for a
query (sister project to 'query categorizer UDF')
Agile Coach, Wikimedia Foundation
The work started by Erik few month ago is finally done. Cirrus requests
are now available in the hive table wmf_raw.CirrusSearchRequestSet.
I really hope this will help us to understand the kind of queries we are
serving and start to work on query classification as Mikhail suggested.
I'm pleased to share the results of our language switching A/B test.
This was an experiment to see if language detection on failed queries
- and then rerunning in the "appropriate" language - could produce a
better outcome for users.
The long and the short of it is that it does produce a marginally
better outcome for web users, but not for API users. Our
recommendation is to enable it as the default user experience for web
users but NOT API users, if that is possible, and disable the test if
You can see the full report at