Discovery August 2019

discovery@lists.wikimedia.org

2 participants
4 discussions

Upcoming Search Platform Office Hours—September 4th

by Trey Jones

The Search Platform Team <https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> usually holds office hours the first Wednesday of each month. Come talk to us about anything related to Wikimedia search! Feel free to add your items to the Etherpad Agenda for the next meeting. Details for our next meeting: Date: Wednesday, Sep 4th, 2019 Time: 15:00-16:00 GMT / 08:00-9:00 PDT / 11:00-12:00 EDT / 17:00-18:00 CEST Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link: https://meet.google.com/vyc-jvgq-dww Hope to talk to you in a week! Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation UTC-4 / EDT

4 years, 7 months

Discovery Weekly Update for the week starting 2019-08-05

by Chris Koerner

Hello, This is the weekly update from the Search Platform team for the week starting 2019-08-05. As always, feedback and questions are welcome. == Discussions == === Search === * Erik discovered that Prometheus was not collecting cloudelastic metrics...so, he defined cloudelastic as a cluster in hieradata [0] * The team worked on fixing documentation of boolean operators [1] and you can read about it. [2] * We needed to help get cloudelastic working with the request streams that we need, so, we fixed LVS IPv6 addresses [3] * There was a spike of errors on eventlogging events that needed to be addressed [4] * We evaluated druid+turnilo+superset for search metrics going forward and found it to be reasonable; so, now we'll build a superset dashboard for search satisfaction 'did you mean' metrics [5] * Erik finished yup the work to initialize CirrusSearch on cloudelastic [6] * *borked* — Icinga check defined from LVS configuration for cloudelastic and now fixed- cloudelastic checks for all ports have been generated on icinga. However, only IPv4 checks were generated and this is Ok for now. If there's need to generate IPv6 checks, please reopen this task [7] * David found an issue where labweb mw hosts were unable to contact any of the elasticsearch clusters and fixed it [8] === Wikidata Query Service === * Stas added support for SDC MediaInfo URIs in WDQS [9] * Stas moved DCAT-AP namespace to https://dcatap.wmflabs.org/ [10] [0] https://phabricator.wikimedia.org/T229937 [1] https://phabricator.wikimedia.org/T228925 [2] https://www.mediawiki.org/wiki/Help:CirrusSearch/Logical_operators [3] https://phabricator.wikimedia.org/T224324 [4] https://phabricator.wikimedia.org/T229614 [5] https://phabricator.wikimedia.org/T229268 [6] https://phabricator.wikimedia.org/T220625 [7] https://phabricator.wikimedia.org/T229621 [8] https://phabricator.wikimedia.org/T230994 [9] https://phabricator.wikimedia.org/T229608 [10] https://phabricator.wikimedia.org/T228297 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner (he/him) Community Relations Specialist Wikimedia Foundation

4 years, 8 months

Discovery Weekly Update for the week starting 2019-07-15

by Chris Koerner

Hello, This is the weekly update from the Search Platform team for the week starting 2019-07-15. As always, feedback and questions are welcome. == Discussions == === Search === * Search of wikidata string property values using haswbstatement is case sensitive - after a bit of discussion, we added a patch for adding a case-insenstitive subfield for statement field [0], and re-indexing happened (with a minor issue) to get it into production [1] * Implemented support for haslabel:* [2] * We received a request to delete search indices for now-deleted zerowiki from production and did the needful [3] * Glent work: create oozie workflow for glent m0 prep has been finished [4] * The team did some work with the Multimedia team to index captions as description fields not label [5] * David worked on adapting IndexLookupFallbackMethod for glent requirements [6] * The team worked on evaluating DYM metrics that are available in current search satisfaction logging [7] with a follow up in [8] ** Metrics we should use moving forward *** % of search shown a [auto / non-auto] dym **** Target: Increase % without significantly reducing the other metrics *** % of people shown non-auto dym that click through to dym results **** Target: Increase % of clickthrough *** % of searches shown dym search results [auto / non-auto] dym results that clicked a result **** Target: Increase % of clickthrough * There was a bug where there was an unexpected result set returned by Elasticsearch [9], we went ahead and closed it because it looks like the same root cause is fixed in [10] * CirrusSearch will now provide a query dispatcher (once the train finishes week of July 30) [11] * We did a spike: load search data into turnilo to test whether exploratory data can do away with some of the dashboards and decided that with respect to our 'did you mean' suggestions and this seems like a plausible path forward [12] * WikibaseLexemeCirrusSearch started to fail for no particular reason (seems to be due to missing class exported by WikibaseLexeme) and will be fixed in this week's train (July 30) [13] === Wikidata Query Service === * Finished reindexing database [14] which fixes resource problems with WDQS [15] * Wikidata RDF dumps no longer have BETA marker [16] * Fixed continuation support for MWAPI in WDQS [17] * Implemented support for wdtn: prefix for WDQS GUI [18] * Fixed incorrect VIAF URIs in WDQS data [19] * Implemented support for ChronologyProtection in WDQS Updater [20] * Refactored label service to support more complex queries [21] === Portal === * The weekly update to the wikipedia.org portal had been failing for a few weeks, Hashar helped out and fixed it, thanks! [22], but we still have another ticket to be fixed [23] [0] https://phabricator.wikimedia.org/T206613 [1] https://phabricator.wikimedia.org/T227136 [2] https://phabricator.wikimedia.org/T224611 [3] https://phabricator.wikimedia.org/T227718 [4] https://phabricator.wikimedia.org/T216783 [5] https://phabricator.wikimedia.org/T226722 [6] https://phabricator.wikimedia.org/T227262 [7] https://phabricator.wikimedia.org/T228226 [8] https://phabricator.wikimedia.org/T216058 [9] https://phabricator.wikimedia.org/T220637 [10] https://phabricator.wikimedia.org/T228063 [11] https://phabricator.wikimedia.org/T216429 [12] https://phabricator.wikimedia.org/T216058 [13] https://phabricator.wikimedia.org/T229215 [14] https://phabricator.wikimedia.org/T228122 [15] https://phabricator.wikimedia.org/T213210 [16] https://phabricator.wikimedia.org/T226153 [17] https://phabricator.wikimedia.org/T209034 [18] https://phabricator.wikimedia.org/T205777 [19] https://phabricator.wikimedia.org/T218522 [20] https://phabricator.wikimedia.org/T212550 [21] https://phabricator.wikimedia.org/T175840 [22] https://phabricator.wikimedia.org/T228639 [23] https://phabricator.wikimedia.org/T213806 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner (he/him) Community Relations Specialist Wikimedia Foundation

4 years, 8 months

Upcoming Search Platform Office Hours—August 5th

by Trey Jones

The Search Platform Team <https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> usually holds office hours the first Wednesday of each month. Come talk to us about anything related to Wikimedia search! Feel free to add your items to the Etherpad Agenda for the next meeting. Details for our next meeting: Date: Wednesday, Aug 7th, 2019 Time: 15:00-16:00 GMT / 08:00-9:00 PDT / 11:00-12:00 EDT / 17:00-18:00 CEST Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link: https://meet.google.com/vyc-jvgq-dww *N.B.:* Google Meet System Requirements <https://support.google.com/meet/answer/7317473> Hope to talk to you in a week! Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation UTC-4 / EDT

4 years, 8 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Discovery August 2019