Discovery December 2018

discovery@lists.wikimedia.org

3 participants
5 discussions

Discovery Weekly Update for the week starting 2018-12-10

by Chris Koerner

Season's greetings, This is the weekly update from the Search Platform team for the week starting 2018-12-10. As always, feedback and questions welcome. Programming note: Due to the holiday season the next update will be for the week of 2019-01-07 == Highlights== * Rare Character Search? Have you ever wanted to search for "rare" characters on-wiki—like the Ankh (☥), unusual punctuation like a ditto mark (〃), or uncommon whitespace characters like an 'en space'? We're trying to gauge interest in a potential future feature that would make finding them easier. Join the conversation on Phabricator [0], English Wikipedia or Wiktionary, or Commons (links at [1]). == Discussions == === Search === * Gehel and David worked on configuring LVS endpoints for new ElasticSearch clusters [2] * David worked on adding a noop surrogate_merger as a BC tokenfilter for WMF indices as part of the work needed for upgrading to ES6 [3] * David got some help from others and worked on adding in all three elasticsearch cluster to labs services [4] * David also worked on adding a job cirrusSearchLinksUpdate: Failed creating job from description ("Page does not exist") [5] === WDQS === * Stas worked on WDQS updating problems [6], [7] and on fixing some SPARQL bugs [8], [9], [10] [0] https://phabricator.wikimedia.org/T211824 [1] https://phabricator.wikimedia.org/T211824#4824496 [2] https://phabricator.wikimedia.org/T207195 [3] https://phabricator.wikimedia.org/T209348 [4] https://phabricator.wikimedia.org/T211526 [5] https://phabricator.wikimedia.org/T208932 [6] https://phabricator.wikimedia.org/T210901 [7] https://phabricator.wikimedia.org/T210044 [8] https://phabricator.wikimedia.org/T200612 [9] https://phabricator.wikimedia.org/T196859 [10] https://phabricator.wikimedia.org/T197598 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

5 years, 4 months

Discovery Weekly Update for the week starting 2018-12-03

by Chris Koerner

Hello, This is the weekly update from the Search Platform team for the week starting 2018-12-03. As always, feedback and questions welcome. == Discussions == === Search === * Mathew and Gehel worked on setting up ElasticSearch on the new codfw servers [0] * Erik finished up the work to analyze wbsearchentities AB test from Nov/Dec and found that clicks@1 improved from 80% to 84% with no change to abandonment rate or number of characters type. [1] [2] * Lots of conversation from the team is on this ticket about building a pipeline to transform ElasticSearch queries with 'explain' enabled to see if we can get a tf graph out of it [3] * Mathew and Gehel collaborated on writing cookbooks to support Spicerack's ElasticSearch multi cluster/instance [4] * David worked on introducing rewrite/second-try strategies for search by externalizing the language detector creation, this code will go out on the train the week of December 10th [5] * Erik and the team worked on an A/B testing plan for wbsearchentities, context=item, in order to help with investigating user click data to help better tune the Wikidata search parameters [6] [0] https://phabricator.wikimedia.org/T210265 [1] https://phabricator.wikimedia.org/F27421972 [2] https://phabricator.wikimedia.org/T211033 [3] https://phabricator.wikimedia.org/T208917 [4] https://phabricator.wikimedia.org/T207919 [5] https://phabricator.wikimedia.org/T204255 [6] https://phabricator.wikimedia.org/T209402 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

5 years, 4 months

Fwd: [Wikidata] Miniature map for coordinate properties

by Deb Tankersley

FYI, with apologies for cross-posting...! -- deb tankersley program manager, engineering Wikimedia Foundation ---------- Forwarded message --------- From: Léa Lacroix <lea.lacroix(a)wikimedia.de> Date: Mon, Dec 10, 2018 at 8:23 AM Subject: [Wikidata] Miniature map for coordinate properties To: Discussion list for the Wikidata project. <wikidata(a)lists.wikimedia.org> Hello all, After we deployed the Commons miniature pictures for the image properties, some editors suggested to do something similar with maps. From Wednesday, December 12th, we will also have a miniature map displayed for the properties containing coordinates, like coordinate location (P625) <https://www.wikidata.org/wiki/Property:P625>. This feature uses mw:Extension:Kartographer <https://www.mediawiki.org/wiki/Extension:Kartographer> and OpenStreetMap. Viewing the map directly in the item will help the editors to check quickly if the coordinates seem correct, and therefore improve the data quality. - <https://commons.wikimedia.org/wiki/File:Screenshot_Wikidata_map_feature_2.p…> View in read mode - <https://commons.wikimedia.org/wiki/File:Screenshot_Wikidata_map_feature_3.p…> View in edit mode - <https://commons.wikimedia.org/wiki/File:Screenshot_Wikidata_map_feature_1.p…> Full screen with list of external maps Some useful features: - Double-click on the miniature map: display it in full screen - On full screen view: click on “external maps” to display different map services and links to many external maps - In edit mode: when adding or editing coordinates in the field, the maps is automatically updated to fit the coordinates you entered - The coordinates are still displayed under the map and link to Geohack If you have any questions or issues with this feature, let me know. You can also see the related ticket <https://phabricator.wikimedia.org/T184933>. Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

5 years, 4 months

Upcoming Search Platform Office Hours—December 5th

by Trey Jones

The Search Platform Team <https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> holds office hours the first Wednesday of each month. Come ask us anything about Wikimedia search! We’re particularly interested in: * Opportunities for collaboration—internally or externally to the Wikimedia Foundation * Challenges you have with on-wiki search, in any of the languages we support But we're happy to talk about anything search-related. Feel free to add your items to the Etherpad Agenda for the next meeting. Details for our next meeting: Date: Wednesday, December 5th, 2018 Time: 16:00-17:00 GMT / 08:00-9:00 PST / 11:00-12:00 EST / 17:00-18:00 CET Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link: https://meet.google.com/vyc-jvgq-dww *N.B.:* Google Meet System Requirements <https://support.google.com/meet/answer/7317473> —Trey Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation

5 years, 4 months

Discovery Weekly Update for the week starting 2018-11-26

by Chris Koerner

Greetings, This is the weekly update from the Search Platform team for the week starting 2018-11-26. As always, feedback and questions welcome. == Discussions == === Search === * David worked on adding a config var to control which clusters (replica) the sanitizer works on by default, since by switching to multi-instance setup we will be adding temp clusters that the sanitizer doesn't need to run on [0] Note: this will go into production the week of December 11th, 2018 * Erik repaired the transfer_to_es analytics job which had stopped working twice in a row [1] * Erik and David worked on the effort to remedy intermittent json parse failures in completion suggester [2] Note: this will go into production the week of December 11th, 2018 * David fixed an issue where implemented LTR query features that rely on the feature vector were not compatible with the way Elastic implemented their profiling API [3] * Gehel, David, and Mathew worked on refactoring current code base to support multiple elasticsearch instances/multiple elasticsearch clusters [4] * We needed to deploy extra-analysis-surrogates & the experimental highlighter 5.5.2.4 to production before we could reindex the Chinese-language wikis [5] and [6] * An issue was found where characters in CJK extension C were being treated as U+FFFD when searching on zhWP - it's fixed now [7] * The team helped to setup two elasticsearch clusters on relforge to test multi-instance [8] * The team also helped with the Elastica dependencies that needed to be updated to v5.3.2 [9] * David worked on fixing the prefix search that had broken again when using multiple namespaces and namespaces with $wgCapitalLinks = false; [10] * David and Gehel completed work on preparing a debian package with the experimental highlighter 5.5.2.4 [11] and fixed an issue where the experimental highlighter was breaking unicode surrogate pairs when cutting the snippets [12] * David fixed an issue where a precondition failed: while trying to get token image at offset -1 [13] * Mathew worked on issues where parts of WDQS puppet module is written in old puppet and also lacks type constraints [14] * David fixed an issue where elasticsearch_hot_threads on relforge had errors (ImportError: No module named 'yaml') [15] * Mathew and Gehel refactored wdqs::gui - to separate cron tasks from the module [16] * Trey wrote a blog post about stemming, stop words, and thesauri. [17] === Portal === * Jan reverted a change that caused the article count to show 0 for all wikis. [18] * Robin designed and Peter and Jan implemented banners for the big English fundraising campaign. [19] * Volker also fixed some bugs on the portal [20] [21] [0] https://phabricator.wikimedia.org/T210728 [1] https://phabricator.wikimedia.org/T210545 [2] https://phabricator.wikimedia.org/T208248 [3] https://phabricator.wikimedia.org/T205249 [4] https://phabricator.wikimedia.org/T207918 [5] https://phabricator.wikimedia.org/T209155 [6] https://phabricator.wikimedia.org/T209156 [7] https://phabricator.wikimedia.org/T168427 [8] https://phabricator.wikimedia.org/T198352 [9] https://phabricator.wikimedia.org/T208650 [10] https://phabricator.wikimedia.org/T208255 [11] https://phabricator.wikimedia.org/T209293 [12] https://phabricator.wikimedia.org/T208736 [13] https://phabricator.wikimedia.org/T209299 [14] https://phabricator.wikimedia.org/T208201 [15] https://phabricator.wikimedia.org/T209030 [16] https://phabricator.wikimedia.org/T209257 [17] https://wikimediafoundation.org/2018/11/28/the-root-of-the-problem/ [18] https://phabricator.wikimedia.org/T210401 [19] https://phabricator.wikimedia.org/T202497 [20] https://phabricator.wikimedia.org/T209018 [21] https://phabricator.wikimedia.org/T197195 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Relations Specialist Wikimedia Foundation

5 years, 4 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Discovery December 2018