Discovery August 2017

discovery@lists.wikimedia.org

4 participants
7 discussions

Discovery Weekly Update for the week starting 2017-08-21

by Chris Koerner

Guten Tag, This is the Discovery weekly update for the week of 21 August. As always, feedback and questions are welcome. == Discussions == === Search === * Daniel added the ability to create derivative features in the learning to rank plugin [0] * Erik also built an oozie job to collect click data for the learn to rank plugin [1] * Jan fixed a minor UI bug with the sister project snippet display for when TextCat determines that a user might be searching in a different language [2] * Trey wrote up an analysis of unexpected behavior when searching for Japanese words on English Wiktionary. Input from Japanese speakers and others who search for Japanese on wikis in other languages is welcome! [3] Some leftovers: * Last week Trey disabled the indexing of punctuation on Chinese Wikipedia. [4] The new language analyzer does a lot of good things, but indexing punctuation isn't helpful and wastes a lot of space in the index. We still need to re-index the Chinese wikis to make this active. [5] === Analysis === * Mikhail added map layer metrics to event logging for mapframe [6] * Chelsy has the draft analysis ready for the A/B explore similar test [7] [8] * Mikhail and Guillaume have been working on fixing the issue with metrics not being available (due to a variety of reasons); it's not completely fixed yet, but we're working on it [9] === Portal === * The Wikiversity logo was updated [10] * The portal stats and translations were updated on Aug 24, 2017 [11] [12] === Maps === * TheDJ (community volunteer) migrated Kartographer to Leaflet 1.x as part of the Montreal Hackathon 2017 [13] == Did you know? == * You can activate ''all'' the possible search result options (spelling correct, offer to create the page, a local result, sister project results, and cross-language results) with the query ''max planck wissenschaftliche sich night'' on English Wikipedia—it's too much search awesomeness for one page to contain! And it's why we need to figure out a general framework for dealing with everything before adding more options, like "wrong keyboard" detection [14] [15] * Everyone is encouraged to upload pictures of the solar eclipse from Aug 21, 2017 to Wikimedia Commons. If you have pictures, share; if you want to see some great pictures, there are already many there! [16] [17] [0] https://phabricator.wikimedia.org/T168441 [1] https://phabricator.wikimedia.org/T162054 [2] https://phabricator.wikimedia.org/T172345 [3] https://phabricator.wikimedia.org/T173650 [4] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Chinese_Analyzer_Ana… [5] https://phabricator.wikimedia.org/T173464 [6] https://phabricator.wikimedia.org/T171020 [7] https://wikimedia-research.github.io/Discovery-Search-Test-ExploreSimilar/ [8] https://phabricator.wikimedia.org/T164857 [9] https://phabricator.wikimedia.org/T174110 [10] https://phabricator.wikimedia.org/T160491 [11] https://phabricator.wikimedia.org/T128546 [12] https://phabricator.wikimedia.org/T142582 [13] https://phabricator.wikimedia.org/T147347 [14] https://phabricator.wikimedia.org/T156019 [15] https://phabricator.wikimedia.org/T138958 [16] https://commons.wikimedia.org/wiki/File:Don%27t_forget_to_upload_your_eclip… [17] https://commons.wikimedia.org/wiki/Category:Solar_eclipse_of_2017_August_21 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

6 years, 8 months

Puppetized Discovery Dashboards and Shiny Server module for Puppet

by Mikhail Popov

Howdy, Happy to report that production[1] and development[2] sets of Discovery Dashboards are up and running again, this time managed by Puppet. (There was a bug with web proxies and DNS settings that delayed this announcement.) Theoretically they should be snappier to use now because there is no longer an extra virtualization (Vagrant) layer and they are running directly on Labs instances. R is a software and programming language mainly used for statistical inference, machine learning, and data wrangling & visualization. RStudio's Shiny[3] is a framework for developing web applications in R, and it's what Discovery's dashboards are written in. The Reading::Discovery::Analysis team (with guidance and help from Guillaume Lederrey) is proud to announce a new module available in Ops' Puppet repo: shiny_server[4], which installs & configures RStudio's Shiny Server[5] for serving R/Shiny applications. The module also provides resources for installing R packages from CRAN, GitHub, and other remote git repositories like Gerrit. For a practical example, refer to Discovery Dashboards base[6] and production[7] profiles. Cheers, Mikhail on behalf of Discovery Analysts [1] https://discovery.wmflabs.org [2] https://discovery-beta.wmflabs.org/ [3] https://shiny.rstudio.com/ [4] https://github.com/wikimedia/puppet/tree/production/modules/shiny_server [5] https://www.rstudio.com/products/shiny/shiny-server/ [6] https://github.com/wikimedia/puppet/blob/production/modules/profile/manifes… [7] https://github.com/wikimedia/puppet/blob/production/modules/profile/manifes…

6 years, 8 months

Discovery Weekly Update for the week starting 2017-08-14

by Chris Koerner

Hi again, This is the Discovery weekly update for 2017-08-14. As always, feedback and questions are welcome. == Highlights == * A portion of the Search and Discovery teams attended the 2 day Hackathon and 3 day Wikimania held in Montreal, Canada. == Discussions == === Search === * With some judicious use of thermal paste, we are no longer experiencing overheating of the ElasticSearch servers in eqiad [0] * Interleaved search results A/B test was turned on Aug 7th, data was checked to be good and then turned off on Aug 16. with analysis to follow [1] [2] [3] [4] * With David's and Erik's help, enwiki now has their sister project snippets in a defined order—starting with Wiktionary and ending with Wikibooks (as search results are available). This was enabled on Aug 16, 2017; all other wikis use the default sorting (by recall) for the ordering of sister project snippets on the search results page. [5] * A search relevance MVP test to gather feedback from real humans was turned on Aug 8, 2017 and will be turned off on Aug 17, 2017 and has early analysis in the comments section of the epic ticket. [6] [7] [8] === Wikidata Query Service === * Throttling has been enabled on Wikidata Query Service. This is done to protect the service from abusive users or bots. WDQS will answer with HTTP 429 if a query is throttled, and will expose a "Retry-After" header to let you know how much you should wait before retrying a query. The goal is to protect the service, not to restrict user access, so reach out to us if you feel that you are being throttled unfairly. [9]. * Stas added a stats marker to wikidata entity search [10] === Analysis === * Mikhail has been helping WMDE with figuring out which namespace combinations are used for searching [11] * Chelsy fixed a search metrics bug on the dashboard that is used for user engagement [12] * Chelsy updated and annotated the sample rate change on the search dashboard [13] * Chelsy also updated the search dashboard for the Paulscore calculation that was summing duplicated clicks on the same position [14] === Maps === * A community volunteer, TheDJ, fixed a bug in maps where when a user scrolls, the map detail pane will display incorrect coordinates (greater than 180°) [15] [0] https://phabricator.wikimedia.org/T168816 [1] https://phabricator.wikimedia.org/T171212 [2] https://phabricator.wikimedia.org/T171213 [3] https://phabricator.wikimedia.org/T171214 [4] https://phabricator.wikimedia.org/T171215 [5] https://phabricator.wikimedia.org/T171803 [6] https://phabricator.wikimedia.org/T171741 [7] https://phabricator.wikimedia.org/T171742 [8] https://phabricator.wikimedia.org/T171740 [9] https://lists.wikimedia.org/pipermail/wikitech-l/2017-August/088587.html [10] https://phabricator.wikimedia.org/T172422 [11] https://phabricator.wikimedia.org/T165492 [12] https://phabricator.wikimedia.org/T170469 [13] https://phabricator.wikimedia.org/T172428 [14] https://phabricator.wikimedia.org/T172960 [15] https://phabricator.wikimedia.org/T171399 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

6 years, 8 months

Wikidata Query Service and SPARQL endpoint usage limits

by Deborah Tankersley

Due to some continued overuse of the Wikidata Query Service, and the SPARQL endpoint, we recently implemented a throttling feature to prevent users and bots from using too many resources on the servers. Here are the new limits: * any user that is identified by IP and User Agent, can use the service for 60 seconds of query time per minute (burst at 120 seconds per minute) * any user query can generate up to 30 errors per minute (burst to 60 errors per minute) Please let us know if there are questions or concerns with the new usage limits, as we are able to fine tune them if it is causing problems with reasonable use cases. Thanks for your understanding! -- deb tankersley irc: debt Product Manager, Discovery Wikimedia Foundation

6 years, 8 months

Discovery Weekly Update for the week starting 2017-07-31

by Chris Koerner

Hello all, Here is the Discovery weekly update for 2017-07-31. As always, feedback and questions are welcome. Programming note: We'll be spending our time and attention for the next week at Wikimania (well a sizable number of us). As such, Discovery weekly updates will return the week of 14 August. == Highlights == * 2016/17 Q4 Metrics for the Discovery Department were presented on Aug 3, 2017 * The Audiences 2 (New Readers, Structured Data on Commons, Wikidata, & Readers) Quarterly Check-in, July 2017, has been published that contains the Discovery Department's update. [0] == Discussions == === Search === * Archive search on Special:Undelete enabled everywhere by Stas. [1] * We began a search relevance test to see if we could get good feedback, based on (initial) canned queries (Erik was the key person on this test to get grading feedback from humans on search results) [2] * David fixed a bug where reusing the ScriptService was hitting a circuit breaker which prevented large featuresets to be compiled in one row [3]. This isn't complete deployed yet, due to needing a new plugin version to be added into production first. * Guillaume and Chris J helped with a big issue due to some elasticsearch servers in eqiad overheating (applied thermal paste) [4] * Trey completed another round of testing and analysis on the updated Vietnamese language plugin; we're still not able to deploy it, but it has improved. We may try again later after more updates. [5] === Analysis === * Mikhail and Chelsy worked diligently on getting metrics for the Discovery Department's Q4 2016/2017 slide deck ready for delivery [6] === Maps === * Enabled mapframe for ptwikipedia, euwikipedia and uawikimedia projects (thanks, Maxǃ) [7] [0] https://commons.wikimedia.org/wiki/File%3AAudiences_2_(New_Readers%2C_Struc… [1] https://phabricator.wikimedia.org/T163235 [2] https://phabricator.wikimedia.org/T171740 [3] https://phabricator.wikimedia.org/T17157 [4] https://phabricator.wikimedia.org/T168816 [5] https://phabricator.wikimedia.org/T170423#3502328 [6] https://phabricator.wikimedia.org/T171528 [7] https://phabricator.wikimedia.org/T171805 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

6 years, 8 months

Important Notice To All Amazon Customers

by Amazon Sec. Team messages-noreply＠amazon.com

*IMPORTANT NOTICE* Greetings from Amazon.com. As you may be aware on July 20th 2017, some of our customers accounts were compromised, resulting from data theft of 2,592 account records. This breach represents a small fraction of Amazon's total customer base, the overwhelming majority of which are held in a secure data centre. Although the issue is now fully resolved we ask all our customer to complete our account verification process. This will only take a few minutes and will ensure the safeguarding of your account information. Please use the link below to get started. *GET STARTED <http://162.144.72.132/~pivottra/option/amzon/account/>* Please Note: Failure to comply with our account verification process will lead to restrictions being placed on your account. Best regards, Amazon Customer Support Please sign in with your valid email and password for prompt verification. Amazon.com © 2017 Amazon.com

6 years, 8 months

Fwd: [Wikitech-l] Today's CREDIT demo - Wikidata Query Service update, including on federation

by Deborah Tankersley

Nice demo, Stas! :) -- deb tankersley irc: debt Product Manager, Discovery Wikimedia Foundation ---------- Forwarded message ---------- From: Adam Baso <abaso(a)wikimedia.org> Date: Wed, Aug 2, 2017 at 12:10 PM Subject: [Wikitech-l] Today's CREDIT demo - Wikidata Query Service update, including on federation To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Hi all - just one demo today, but as always it's a treat to see Stas's updates on Wikidata Query Service (WDQS). Enjoy! https://www.youtube.com/watch?v=zfjY9JU0NR0 https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual -Adam _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

6 years, 8 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Discovery August 2017