Hello, Another weekly update from Discovery!

== Highlights == * A recent update to the search results page on all wikis - sister project snippets - was deployed into production on June 15; see the email for more info. [0] [1] * Added a note to the Extension:Kartographer page about mapframe deployments [2] * Sent out a communication about what the Discovery team's goals and future work will be. [3]

== Discussions ==

=== Search === * Logstash scripts are now using curator, and some standard action files (enabling / disabling shard allocation) have been deployed [4] * Deployed new versions of Wikimedia and other ElasticSearch plugins (epic task with lots of smaller subtasks [5] * Various updates to getting the search clusters up to ElasticSearch 5.3.2 [6] [7] [8] [9] * Fixed an issue where the sister project snippets were causing an weird display problem [10] * We've updated Ukrainian-language wikis with a new Ukrainian language analyzer, which should provide better search results by recognizing related forms of a word. (An example in English would be that searching for "hope", "hoped", "hopes", or "hoping" can all find each other.) [11] * We've updated Chinese-language wikis using a new Chinese language analyzer, which should provide better search results by doing a better job of breaking up Chinese text into words, and by automatically converting between Simplified and Traditional characters when searching. [12] We've updated Swedish-language wikis with a smarter configuration that recognizes å, ä, and ö as distinct letters (and not just variants of a and o). [13] * Setup testing, training and validation splits for learning to rank machine learning [14] * Worked on calculating the NDCG of click data that feeds the machine learning rank pipeline [15]

=== Wikidata Query Service === * Enabled the Mediawiki Service API which allows interacting with Mediawiki API from SPARQL. [16] * Added more federation endpoints. [17]

=== Analysis === * Finalized the migration from Vagrant to Puppet configuration for the dashboards [18] Investigated a drop in pageviews and clickthroughs on the Wikipedia.org portal - turns out summer is here [19] * Fixed a minor issue with the desktop and mobile web graphs on the external search dashboard [20]

=== Interactive === * Achieved some clarity to the phabricator board with priorities and what is in progress, needs to be in the backlog or stalled. [21]

[0] https://lists.wikimedia.org/pipermail/discovery/2017-June/001536.html [1] https://phabricator.wikimedia.org/T162276 [2] https://www.mediawiki.org/wiki/Help:Extension:Kartographer#Discovery_Maps_Up... [3] https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes/Updat... [4] https://phabricator.wikimedia.org/T166154 [5] https://phabricator.wikimedia.org/T160948 [6] https://phabricator.wikimedia.org/T163703 [7] https://phabricator.wikimedia.org/T163708 [8] https://phabricator.wikimedia.org/T167636 [9] https://phabricator.wikimedia.org/T149006 [10] https://phabricator.wikimedia.org/T167301 [11] https://phabricator.wikimedia.org/T160106 [12] https://phabricator.wikimedia.org/T158203 [13] https://phabricator.wikimedia.org/T160562 [14] https://phabricator.wikimedia.org/T162311 [15] https://phabricator.wikimedia.org/T166585 [16] https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual/MWAPI [17] https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual#Federation [18] https://phabricator.wikimedia.org/T161354 [19] https://phabricator.wikimedia.org/T167822 [20] https://phabricator.wikimedia.org/T167850 [21] https://phabricator.wikimedia.org/tag/interactive-sprint/

---

The archive of all past updates can be found on MediaWiki.org:

https://www.mediawiki.org/wiki/Discovery/Status_updates

Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator.

[1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R

Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation