As we reach the last month of the quarter, it's a good opportunity for us to reflect on where we want to go for the last part of our remaining time.
On the one hand, we're in quite a good place. We're just wrapping up our work on our Q2 goal for search https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q2_Goals#Search, which is excellent! On the other hand, the test showed minimal impact, so our users still aren't seeing the impact of our work. Since we can continue running A/B tests for improving language support relatively cheaply in terms of required engineering time, let's take a look back at what we've done previously and see if we can choose something high impact to work on!
The completion suggester is a very promising avenue for us to invest in. As noted in our analysis of the initial test https://phabricator.wikimedia.org/T111858, using the completion suggester instead of prefixsearch significantly reduced the zero results rate. We've not had an impact on this through other efforts, so this is interesting! In order to more thoroughly test the suggester, we can make it a Beta Feature https://phabricator.wikimedia.org/T119535. This will allow editors to opt-in to testing it, and will gather us valuable qualitative feedback about what use cases the completion suggester could support better. The caveat, of course, is that the feedback will be from a specific segment of our user base (users who test beta features) which is more specialised than the intended audience (everyone). That said, the feedback will still be very helpful. There's quite a bit of work to do here; our initial test of the suggester was very hacky, but now that it's proven itself, we can be more rigorous.
The other avenue is using page views to influence result ranking. This is in an earlier stage thant he completion suggester, in that it's a relatively unproven approach for us, but it's something that's logical and that we've been interested in for a while. But, we've repeatedly had to deprioritise it for other work. If something is popular, it makes sense to rank it up in search results. Obviously, we do not want to be *too* aggressive with this in case we create feedback loops, but I think the potential benefits are quite clear if done correctly.
I explained a lot of this more briefly in our last standup, but hopefully this should give you all some guidance on where we're going.
Thanks, and as always, if there are any questions then please let me know.
Dan
Dan,
As I broached in an earlier email, the Wikisources have an interest in greater scope of the completion suggested / typeahead.
With formal titles of works often starting with "The..." or "A ..." etc. having the ability to reach further into a work's title OR to leverage the use of {{defaultsort:}} would have advantages.
We are still hampered by long titles in short suggester boxes, especially where the key words are at the end and away from the visual part of the suggestion.
Regards, Billinghurst
On Thu, 26 Nov 2015 12:47 Dan Garry dgarry@wikimedia.org wrote:
As we reach the last month of the quarter, it's a good opportunity for us to reflect on where we want to go for the last part of our remaining time.
On the one hand, we're in quite a good place. We're just wrapping up our work on our Q2 goal for search https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q2_Goals#Search, which is excellent! On the other hand, the test showed minimal impact, so our users still aren't seeing the impact of our work. Since we can continue running A/B tests for improving language support relatively cheaply in terms of required engineering time, let's take a look back at what we've done previously and see if we can choose something high impact to work on!
The completion suggester is a very promising avenue for us to invest in. As noted in our analysis of the initial test https://phabricator.wikimedia.org/T111858, using the completion suggester instead of prefixsearch significantly reduced the zero results rate. We've not had an impact on this through other efforts, so this is interesting! In order to more thoroughly test the suggester, we can make it a Beta Feature https://phabricator.wikimedia.org/T119535. This will allow editors to opt-in to testing it, and will gather us valuable qualitative feedback about what use cases the completion suggester could support better. The caveat, of course, is that the feedback will be from a specific segment of our user base (users who test beta features) which is more specialised than the intended audience (everyone). That said, the feedback will still be very helpful. There's quite a bit of work to do here; our initial test of the suggester was very hacky, but now that it's proven itself, we can be more rigorous.
The other avenue is using page views to influence result ranking. This is in an earlier stage thant he completion suggester, in that it's a relatively unproven approach for us, but it's something that's logical and that we've been interested in for a while. But, we've repeatedly had to deprioritise it for other work. If something is popular, it makes sense to rank it up in search results. Obviously, we do not want to be *too* aggressive with this in case we create feedback loops, but I think the potential benefits are quite clear if done correctly.
I explained a lot of this more briefly in our last standup, but hopefully this should give you all some guidance on where we're going.
Thanks, and as always, if there are any questions then please let me know.
Dan
-- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation _______________________________________________ discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
Hey Billinghurst,
We intend for the completion suggester beta feature to be available on all Wikimedia wikis. It will obviously be a really early prototype, so there will no doubt be plenty of problems and issues; that's why we want to roll the beta feature out, so that we can find these! I look forward to you (and others) on Wikisource testing it and giving us feedback on how it works.
Thanks, Dan
On 25 November 2015 at 23:00, billinghurst billinghurstwiki@gmail.com wrote:
Dan,
As I broached in an earlier email, the Wikisources have an interest in greater scope of the completion suggested / typeahead.
With formal titles of works often starting with "The..." or "A ..." etc. having the ability to reach further into a work's title OR to leverage the use of {{defaultsort:}} would have advantages.
We are still hampered by long titles in short suggester boxes, especially where the key words are at the end and away from the visual part of the suggestion.
Regards, Billinghurst
On Thu, 26 Nov 2015 12:47 Dan Garry dgarry@wikimedia.org wrote:
As we reach the last month of the quarter, it's a good opportunity for us to reflect on where we want to go for the last part of our remaining time.
On the one hand, we're in quite a good place. We're just wrapping up our work on our Q2 goal for search https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q2_Goals#Search, which is excellent! On the other hand, the test showed minimal impact, so our users still aren't seeing the impact of our work. Since we can continue running A/B tests for improving language support relatively cheaply in terms of required engineering time, let's take a look back at what we've done previously and see if we can choose something high impact to work on!
The completion suggester is a very promising avenue for us to invest in. As noted in our analysis of the initial test https://phabricator.wikimedia.org/T111858, using the completion suggester instead of prefixsearch significantly reduced the zero results rate. We've not had an impact on this through other efforts, so this is interesting! In order to more thoroughly test the suggester, we can make it a Beta Feature https://phabricator.wikimedia.org/T119535. This will allow editors to opt-in to testing it, and will gather us valuable qualitative feedback about what use cases the completion suggester could support better. The caveat, of course, is that the feedback will be from a specific segment of our user base (users who test beta features) which is more specialised than the intended audience (everyone). That said, the feedback will still be very helpful. There's quite a bit of work to do here; our initial test of the suggester was very hacky, but now that it's proven itself, we can be more rigorous.
The other avenue is using page views to influence result ranking. This is in an earlier stage thant he completion suggester, in that it's a relatively unproven approach for us, but it's something that's logical and that we've been interested in for a while. But, we've repeatedly had to deprioritise it for other work. If something is popular, it makes sense to rank it up in search results. Obviously, we do not want to be *too* aggressive with this in case we create feedback loops, but I think the potential benefits are quite clear if done correctly.
I explained a lot of this more briefly in our last standup, but hopefully this should give you all some guidance on where we're going.
Thanks, and as always, if there are any questions then please let me know.
Dan
-- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation _______________________________________________ discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery
discovery mailing list discovery@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/discovery