Greetings,

This is the weekly update from the Search Platform team for the week starting 2018-11-26.

As always, feedback and questions welcome.

== Discussions ==

=== Search === * David worked on adding a config var to control which clusters (replica) the sanitizer works on by default, since by switching to multi-instance setup we will be adding temp clusters that the sanitizer doesn't need to run on [0] Note: this will go into production the week of December 11th, 2018 * Erik repaired the transfer_to_es analytics job which had stopped working twice in a row [1] * Erik and David worked on the effort to remedy intermittent json parse failures in completion suggester [2] Note: this will go into production the week of December 11th, 2018 * David fixed an issue where implemented LTR query features that rely on the feature vector were not compatible with the way Elastic implemented their profiling API [3] * Gehel, David, and Mathew worked on refactoring current code base to support multiple elasticsearch instances/multiple elasticsearch clusters [4] * We needed to deploy extra-analysis-surrogates & the experimental highlighter 5.5.2.4 to production before we could reindex the Chinese-language wikis [5] and [6] * An issue was found where characters in CJK extension C were being treated as U+FFFD when searching on zhWP - it's fixed now [7] * The team helped to setup two elasticsearch clusters on relforge to test multi-instance [8] * The team also helped with the Elastica dependencies that needed to be updated to v5.3.2 [9] * David worked on fixing the prefix search that had broken again when using multiple namespaces and namespaces with $wgCapitalLinks = false; [10] * David and Gehel completed work on preparing a debian package with the experimental highlighter 5.5.2.4 [11] and fixed an issue where the experimental highlighter was breaking unicode surrogate pairs when cutting the snippets [12] * David fixed an issue where a precondition failed: while trying to get token image at offset -1 [13] * Mathew worked on issues where parts of WDQS puppet module is written in old puppet and also lacks type constraints [14] * David fixed an issue where elasticsearch_hot_threads on relforge had errors (ImportError: No module named 'yaml') [15] * Mathew and Gehel refactored wdqs::gui - to separate cron tasks from the module [16] * Trey wrote a blog post about stemming, stop words, and thesauri. [17]

=== Portal === * Jan reverted a change that caused the article count to show 0 for all wikis. [18] * Robin designed and Peter and Jan implemented banners for the big English fundraising campaign. [19] * Volker also fixed some bugs on the portal [20] [21]

[0] https://phabricator.wikimedia.org/T210728 [1] https://phabricator.wikimedia.org/T210545 [2] https://phabricator.wikimedia.org/T208248 [3] https://phabricator.wikimedia.org/T205249 [4] https://phabricator.wikimedia.org/T207918 [5] https://phabricator.wikimedia.org/T209155 [6] https://phabricator.wikimedia.org/T209156 [7] https://phabricator.wikimedia.org/T168427 [8] https://phabricator.wikimedia.org/T198352 [9] https://phabricator.wikimedia.org/T208650 [10] https://phabricator.wikimedia.org/T208255 [11] https://phabricator.wikimedia.org/T209293 [12] https://phabricator.wikimedia.org/T208736 [13] https://phabricator.wikimedia.org/T209299 [14] https://phabricator.wikimedia.org/T208201 [15] https://phabricator.wikimedia.org/T209030 [16] https://phabricator.wikimedia.org/T209257 [17] https://wikimediafoundation.org/2018/11/28/the-root-of-the-problem/ [18] https://phabricator.wikimedia.org/T210401 [19] https://phabricator.wikimedia.org/T202497 [20] https://phabricator.wikimedia.org/T209018 [21] https://phabricator.wikimedia.org/T197195

----

Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update.

https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly

The archive of all past updates can be found on MediaWiki.org:

https://www.mediawiki.org/wiki/Discovery/Status_updates

Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator.

[1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R

Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation