Hi everybody,
You may have seen the recent communication [1 https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes] about the product and tech tune-up which went live the week of June 5th, 2017. In that communication, we promised an update on the future of Discovery projects and we will talk about those in this email.
The Discovery team structure has now changed, but the new teams will still work together to complete the goals as listed in the draft annual plan.[2] A summary of their anticipated work, as we finalize these changes, is below. We plan on doing a check-in at the end of the calendar year to see how our goals are progressing with the new smaller and separated team structure.
Here is a list of the various projects under the Discovery umbrella, along with the goals that they will be working on:
Search Backend
Improve search capabilities:
-
Implement ‘learning to rank’ [3] and other advanced machine learning methodologies -
Improve support for languages using new analyzers -
Maintain and expand power user search functionality
Search Frontend
Improve user interface of the search results page with new functionality:
-
Implement explore similar [4] https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testing#A.2FB_test:_Add_.27explore_similar.27_pages_and_categories_for_search_results -
Update the completion suggester box [5] https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester -
Investigate the usage of a Wiktionary widget for English Wikipedia [6]
Wikidata Query Service
Expand and scale:
-
Improve ability to support power features on-wiki for readers -
Improve full text search functionality -
Implement SPARQL federation support
Portal
Create and implement automated language statistics and translation updates for Wikipedia.org
Analysis
Provide in-depth analytics support:
-
Perform experimental design, data collection, and data analysis -
Perform ad-hoc analyses of Discovery-domain data -
Maintain and augment the Discovery Dashboards,[7] which allow the teams to track their KPIs and other metrics
Maps
Map support:
-
Implement new map style -
Increase frequency of OSM data replication -
As needed, assist with individual language Wikipedia’s implementation of mapframe [8] https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps
Note: There is a possibility that we can do more with maps in the coming year; we are currently evaluating strategic, partnership, and resourcing options.
Structured Data on Commons
Extend structured data search on Commons, as part of the structured data grant [9] via:
-
Research and implement advanced search capabilities -
Implement new elements, filters, relationships
Graphs and Tabular Data on Commons
We will be re-evaluating this functionality against other Commons initiatives such as the structured data grant. As with maps, we will provide updates when we know more.
We are still working out all the details with the new team structure and there might be some turbulence; let us know if there are any concerns and we will do our best to answer them.
Best regards,
Deborah Tankersley, Product Manager, Discovery
Erika Bjune, Engineering Manager, Search Platform
Jon Katz, Reading Product Lead
Toby Negrin, Interim Vice President of Product
Victoria Coleman, Chief Technology Officer
[1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes
[2] https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/D...
[3] https://en.wikipedia.org/wiki/Learning_to_rank
[4] https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testing...
[5] https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester
[6] https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testing...
[7] https://discovery.wmflabs.org/
[8] https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps
[9] https://commons.wikimedia.org/wiki/Commons:Structured_data