Discovery April 2018

discovery@lists.wikimedia.org

2 participants
7 discussions

Discovery Weekly Update for the week starting 2018-04-16

by Chris Koerner

Here is another update from the Search Platform team. This one includes the week of 2018-04-16 and 2018-04-23 As always, feedback and questions welcome. == Highlights== * The Search team is looking for a volunteer to help with this easy ticket: T192458 Enable search result ranking in MySQL full text search. [0] == Discussions == === Search === * Serbian-language wikis have been updated with a new language analyzer for search that does some stemming (so that related forms of a word can find each other) and handles the transliteration between the Serbian Cyrillic and Latin alphabets, which are both used in Serbian. [1] *...and work on the plugin and re-indexing the wikis to allow the new Serbian alphabets to be used is also done [2] [3] [4] * Stas fixed some bugs for deep category search [5] [6] * Completion search for Lexemes merged [7] * Fixed some indexing errors [8] * Cirrus profile management has been updated to provide a standard way to override profile content [9] * Fixed issues with ElasticSearch latency alerts on ~15 wikis that had the issue [10] [0] https://phabricator.wikimedia.org/T192458 [1] https://en.wikipedia.org/wiki/Serbian_language#Writing_system [2] https://phabricator.wikimedia.org/T189239 [3] https://phabricator.wikimedia.org/T183015 [4] https://phabricator.wikimedia.org/T189265 [5] https://phabricator.wikimedia.org/T192270 [6] https://phabricator.wikimedia.org/T188350 [7] https://phabricator.wikimedia.org/T190970 [8] https://phabricator.wikimedia.org/T192609 [9] https://phabricator.wikimedia.org/T191876 [10] https://phabricator.wikimedia.org/T191236 --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

5 years, 12 months

Fwd: [Maps-l] Map internationalization: more on how the new maps work with OSM data

by Deborah Tankersley

...with apologies for cross-posting. -- deb tankersley Program Manager, Engineering Wikimedia Foundation ---------- Forwarded message ---------- From: Joe Matazzoni <jmatazzoni(a)wikimedia.org> Date: Fri, Apr 27, 2018 at 2:38 AM Subject: [Maps-l] Map internationalization: more on how the new maps work with OSM data To: maps-l(a)lists.wikimedia.org Following up on yesterday’s announcement encouraging people to try out internationalized maps on testwiki [1], I added more information today [2] about how the feature works. In particular, in yesterday’s post we introduced the new map parameter and value, lang=“local”, which causes a map to display in the language of the territory mapped—essentially turning off internationalization and reverting to the current (soon to be former) map behavior, which displays maps only in the language(s) of the territory mapped. The new post dives into the OpenStreetMap data quirks that make such a feature necessary and explains when you might want to use it. So if you’re interested in wiki maps, have a look. You can see the internationalization features in action on a sample page I put on testwiki [3]. And please remember to leave your feedback on the Map Improvements 2018 talk page [4]. We’re listening! [1] https://www.mediawiki.org/wiki/Map_improvements_2018# April_25,_2018,_You_can_now_try_out_internationalization_(on_testwiki) [2] https://www.mediawiki.org/wiki/Map_improvements_2018# April_26,_2018:_OSM_name_data_quirks_and_the_uses_of_lang=% E2%80%9Clocal%E2%80%9D [3] https://test2.wikipedia.org/wiki/Map_internationalization_examples [4] https://www.mediawiki.org/wiki/Talk:Map_improvements_2018 _____________________ *Joe Matazzoni* Product Manager, Collaboration Wikimedia Foundation, San Francisco *"Imagine a world in which every single human being can freely share in the sum of all knowledge." * _______________________________________________ Maps-l mailing list Maps-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l

5 years, 12 months

Fwd: [Maps-l] Maps in the language of your choice— try it out now on testwiki

by Deborah Tankersley

...with apologies for cross-posting. -- deb tankersley Program Manager, Engineering Wikimedia Foundation ---------- Forwarded message ---------- From: Joe Matazzoni <jmatazzoni(a)wikimedia.org> Date: Wed, Apr 25, 2018 at 11:12 PM Subject: [Maps-l] Maps in the language of your choice— try it out now on testwiki To: maps-l(a)lists.wikimedia.org You can now display maps in languages of your choice on testwiki. I made two pages to demonstrate the new features, on testwiki [1] and testwiki2 [2] (embedded maps on test2 are dynamic; those on test are static until you click to pop up an enlargement). By default, internationalized maps display in the language of the wiki (which is English for the testwikis). So to experiment with these features, you’ll want to use the two new mapframe parameters we’ve added. Just insert them into your map frame tags. - lang=”xx” Shows map labels in the language you specify with the short language codes used for each wiki. - lang=“local” Shows map labels in the languages of the territory mapped (essentially opting out of internationalization). Right now, internationalization works only with mapframe, not maplink (which should be working some time next week). You can read more about this new feature and how to use it on the Map Improvements 2018 project page, under Updates [3]. Our plan is to wait a week or two and assess user comments about the feature. At that point, we’ll decide whether to move forward with a general release or keep making fixes. So please try the new features out and leave feedback on the Map Improvements 2018 talk page [4] . We’re listening! Yours, Joe [1] https://test.wikipedia.org/wiki/Map_internationalization_examples [2] https://test2.wikipedia.org/wiki/Map_internationalization_examples [3] https://www.mediawiki.org/wiki/Map_improvements_2018#Project_Updates [4] https://www.mediawiki.org/wiki/Talk:Map_improvements_2018 _____________________ *Joe Matazzoni* Product Manager, Collaboration Wikimedia Foundation, San Francisco *"Imagine a world in which every single human being can freely share in the sum of all knowledge." * _______________________________________________ Maps-l mailing list Maps-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l

6 years

Fwd: [Maps-l] An update on map internationalization

by Deborah Tankersley

...with apologies for cross-posting. Cheers, Deb -- deb tankersley Program Manager, Engineering Wikimedia Foundation ---------- Forwarded message ---------- From: Joe Matazzoni <jmatazzoni(a)wikimedia.org> Date: Fri, Apr 20, 2018 at 10:45 PM Subject: [Maps-l] An update on map internationalization To: maps-l(a)lists.wikimedia.org This is to let you know that Collaboration Team is planning to release map internationalization next week for testing on testwiki. When it’s ready, we’ll post a note to confirm. Meanwhile, you might like to check out the detailed post I added last night to the Map Improvements 2018 project board: Special Update on Map Internationalization[1]. It includes a lot of information on the feature's status, how the it will work, how we imagine it might be useful, what the known limitations are, etc. I’m looking forward to getting your input on this challenging but important feature (the best place to leave your ideas and questions is on the project talk page [2]). Yours, Joe [1] https://www.mediawiki.org/wiki/Map_improvements_2018# April_18,_2018,_Special_Update_on_Map_Internationalization [2] https://www.mediawiki.org/wiki/Talk:Map_improvements_2018 _____________________ *Joe Matazzoni* Product Manager, Collaboration Wikimedia Foundation, San Francisco *"Imagine a world in which every single human being can freely share in the sum of all knowledge." * _______________________________________________ Maps-l mailing list Maps-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/maps-l

6 years

Discovery Weekly Update for the week starting 2018-04-09

by Chris Koerner

Heya, Another week, another update from the Search Platform team. This one is for the week starting 2018-04-09. As always, feedback and questions welcome. == Discussions == === Search === * New search code fully deployed & enabled on Wikidata. ==Events and News == * Erik and Trey went to the "OpenSource Connections Haystack Search Relevance Conference" and "Tom Tom Founders Festival Machine Learning Conference", which were back-to-back in Charlottesville, VA. Erik presented on how we use clickstream information to create training data for our learning to rank models at Haystack. [1] Trey wrote up trip notes—with lots of links—on MediaWiki. [2] == Other Noteworthy Stuff == * Fix for CirrusSearchCheckerJob errors rolled out. [3] * Stas implemented indexing Lexemes & Forms for WikibaseLexeme extension. [4] == Did you know? == *The English verb "to be" is kind of weird—the infinitive "be" and participles "being, been" start with "b-", while the preterite forms "was, were" start with "w-", and the present forms "am, is, are" start with vowels. The conjugations originally come from three or four different verbs! Why "three or four"? Wiktionary disagrees with itself a bit, listing four on the etymology of "is" [5] and three on the etymology of "be". [6] The conflation goes back at least to Proto-Germanic, [7] so German is similarly weird. [8] Dutch has a greatly simplified paradigm, but still shows some trace of the multiple sources. [9] Other languages, including ASL, Arabic, Bengali, Hawaiian, Hebrew, Indonesian, Japanese, Russian, Turkish, and Ukrainian at least partly avoid this mess by having a zero copula. [10] For search on-wiki, we deal with this problem in part with stemming [11] and stop words. [12] [0] https://www.wikidata.org/wiki/Wikidata:Project_chat#Improvements_on_the_sea… [1] https://commons.wikimedia.org/wiki/File:From_Clicks_to_Models_The_Wikimedia… [2] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/April_2018_Conferenc… [3] https://phabricator.wikimedia.org/T190958 [4] https://phabricator.wikimedia.org/T189745 [5] https://en.wiktionary.org/wiki/is#Etymology_1 [6] https://en.wiktionary.org/wiki/be#Etymology [7] https://en.wikipedia.org/wiki/Proto-Germanic_language [8] https://en.wiktionary.org/wiki/sein#Conjugation [9] https://en.wiktionary.org/wiki/zijn#Inflection [10] https://en.wikipedia.org/wiki/Zero_copula [11] https://en.wikipedia.org/wiki/Stemming [12] https://en.wikipedia.org/wiki/Stop_words --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

6 years

Discovery Weekly Update for the week starting 2018-04-02

by Chris Koerner

Heya, Well ok, that subject line is a bit of a lie. Below you'll find the weekly update from the Search Platform team for the week starting 2018-03-26 _and_ 2018-04-02. As always, feedback and questions welcome. == Discussions == === Search === * Stas finished up the work to enable the query builder hook so that using label & description index for fulltext searches work [0] * Erik worked on boosting search results with exact phrase match and updated the models to one that is significantly less expensive [1] * We completed a task for an in-place reindex to all wiki's to pickup new trigram index for title and redirect.title field [2] * Trey finished his review of Slovak Morphological Libraries [3] [4] * David completed a forceSearchIndex on testwiki, mediawikiwiki, labswiki, labtestwiki, svwiki, and everything on Beta Cluster [5] * Trey added support for external stemmer to Analyzer Analysis tools [6] * David did work to make all SearchKeywords a subclass of SimpleKeywordFeature, to remove and centralize all the parsing logic out of these classes [7] * Davis also corrected a Cirrussearch Error "An error has occurred while searching" that was due to a comma used as decimal separator [8] === Analysis === * Chelsy finished up the a report on analyzing the results of query explorer AB test [9] [10] [0] https://phabricator.wikimedia.org/T178851] [1] https://phabricator.wikimedia.org/T187240 [2] https://phabricator.wikimedia.org/T188452 [3] https://phabricator.wikimedia.org/T178929 [4] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Slovak_Stemmer_Analy… [5] https://phabricator.wikimedia.org/T189694 [6] https://phabricator.wikimedia.org/T190816 [7] https://phabricator.wikimedia.org/T186879 [8] https://phabricator.wikimedia.org/T189877 [9] https://phabricator.wikimedia.org/T189843 [10] https://analytics.wikimedia.org/datasets/discovery/reports/Evaluate_feature… --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

6 years

[Discovery] Improvements on the search results

by Deborah Tankersley

...with apologies for cross-posting, but this is cool stuff! :) -- deb tankersley Program Manager, Engineering Wikimedia Foundation On Thu, Apr 5, 2018 at 1:35 AM, Léa Lacroix <lea.lacroix(a)wikimedia.de> wrote: > Hello all, > > Some improvements have been made by the Search team at WMF on > Special:Search <https://www.wikidata.org/wiki/Special:Search> (the page > that is loaded when you type something in the search field and press enter, > for example). > > - The string that you typed is highlighted in bold in the label of the > results > - The number of bytes is no longer displayed, instead you can see the > number of sitelinks and statements > - The code is now using the same indexing as completion search, which > should improve relevancy and better control over ranking > > If you have any question or find a bug, feel free to ping user:Smalyshev > (WMF) <https://www.wikidata.org/wiki/User:Smalyshev_(WMF)>. > Cheers, > > -- > Léa Lacroix > Project Manager Community Communication for Wikidata > > Wikimedia Deutschland e.V. > Tempelhofer Ufer 23-24 > 10963 Berlin > www.wikimedia.de > > Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. > > Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg > unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt > für Körperschaften I Berlin, Steuernummer 27/029/42207. > > _______________________________________________ > Wikidata mailing list > Wikidata(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikidata > >

6 years

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Discovery April 2018