Discovery

discovery@lists.wikimedia.org

1 participants
756 discussions

Discovery Weekly Update for the week starting 2017-09-11
by Chris Koerner 18 Sep '17

18 Sep '17

Hello, Here are the updates from the Discovery team for last week. As always, feedback and questions are welcome. Reminder: There is a new way to follow these weekly updates.You can subscribe to recieve on-wiki (or opt-in email) notifications of the Discovery weekly update. Subscribe to be notified. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly ==Discussions== === Search === * Gehel did quite a bit of work to implement the new version of Logstash on our servers, thanks for the help, RobH! [0] * David, Gehel and Moritz updated the elasticsearch deployment plugins to use debian packages instead of salt [1] * Stas and David fixed an issue where Wikidata Elastic search drops results with matches with different language labels (e.g., you search in English but Spanish label matches) [2] * The explore similar test for language links A/B test was turned on Sep 14 and will run for a week. [3] [4] === Analysis === * Mikhail finished up a new dashboard metric for dwell time on SERP [5] [6] * Chelsy finished up the final analyzation of the results of the swap2and3 search test and it's up on Commons. [7] [8] [0] https://phabricator.wikimedia.org/T175045 [1] https://phabricator.wikimedia.org/T158560 [2] https://phabricator.wikimedia.org/T173231 [3] https://phabricator.wikimedia.org/T175647 [4] https://phabricator.wikimedia.org/T175648 [5] https://discovery.wmflabs.org/metrics/#spr_surv [6] https://phabricator.wikimedia.org/T170468 [7] https://phabricator.wikimedia.org/T136017 [8] https://commons.wikimedia.org/wiki/File:Swap2and3_Search_Test_Analysis.pdf ---- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Discovery Weekly Update for the week starting 2017-09-04
by Chris Koerner 11 Sep '17

11 Sep '17

Hello, A few updates this week from across the Discovery team. Programming note: There is a new way to follow these weekly updates. You can subscribe to recieve on-wiki (or opt-in email) notifications of the Discovery weekly update. Subscribe to be notified. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly == Discussions == * Paul conducted a cartographic review of the new map styles for the Design team on 6 Sep 2017. === Search === * Gehel updated logstash to stop deploying custom elasticsearch plugins that are no longer used and did a rolling restart of elasticsearch/logstash [0] * David upgraded an extra plugin as part of the upgrade to elastic 5.5.x [1] * Erik turned on a third test for the Search Relevance Survey (graded by humans), the test is expected to take a week. [2] * Erik trained MLR models for all wikis with >= 1% of search traffic, in preparation for AB test next week * Erik investigating potential memory leaks in MLR model training which makes training for large wikis (de, en) error prone === Portal === * Jan fixed a bug where the typeahead wasn't working for a single character [3] * Jan also updated the Wikipedia portal statistics and translations on 6 Sep 2017 [4] [5] === Maps === * Gehel finished up reimaging the maps-test servers [6] [0] https://phabricator.wikimedia.org/T174933 [1] https://phabricator.wikimedia.org/T174652 [2] https://phabricator.wikimedia.org/T175046 [3] https://phabricator.wikimedia.org/T173885 [4] https://phabricator.wikimedia.org/T128546 [5] https://phabricator.wikimedia.org/T142582 [6] https://phabricator.wikimedia.org/T169011 ---- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

An essay about searching for "how-tos"
by Chris Koerner 07 Sep '17

07 Sep '17

Hello, Here's a fun visual essay by Xaquín G.V. in collaboration with Google News Lab. The premise is that many people search for information on how to do everyday activities. From fixing things around the house, to cooking, to getting rid of hiccups. There's some neat (albeit vague) interactivity to see how search is different across different demographics. How did we get anything done before search? :) http://how-to-fix-a-toilet.com Yours, Chris Koerner Community Liaison Wikimedia Foundation

2 1

Discovery Weekly Update for the week starting 2017-08-28
by Chris Koerner 05 Sep '17

05 Sep '17

Hi, Here is the weekly status update from the Discovery team. Feedback and questions are welcome. == Discussions == === Search === * Erik turned off the interleaved search results test and the search relevance test [0] [1] * Jan completed a minor update to the UI of the sister project snippets [2] * Erik continued work on mitigating future occurrences of search latency degradation [3] === Analysis === * Mikhail finished up the analysis for the first two A/B tests for the search relevance testing (grading by humans) [4] [5] * Chelsy put the final touches on the explore similar A/B test analysis [6] [7] * Mikhail reviewed an upcoming patch to add purge info for the Kartographer schema [8] === Maps === * Gehel aligned the maps* and maps-test* configurations to be the same [9] * After many discussions on what is a reasonable per-IP ratelimit for maps, decisions have been made and was promoted to production on Sep 4 [10] == Other Noteworthy Stuff == * Trey wrote a post for the WIkimedia Blog: "Wikipedia, search, and the 'Цкщтп' keyboard" [11] [0] https://phabricator.wikimedia.org/T171214 [1] https://phabricator.wikimedia.org/T171742 [2] https://phabricator.wikimedia.org/T171804 [3] https://phabricator.wikimedia.org/T169498 [4] https://wikimedia-research.github.io/Discovery-Search-Adhoc-SurveyMVP/ [5] https://phabricator.wikimedia.org/T171740 [6] https://wikimedia-research.github.io/Discovery-Search-Test-ExploreSimilar/ [7] https://phabricator.wikimedia.org/T164857 [8] https://phabricator.wikimedia.org/T171622 [9] https://phabricator.wikimedia.org/T169082 [10] https://phabricator.wikimedia.org/T169175 [11] https://blog.wikimedia.org/2017/08/28/wikipedia-search-phonetic-keyboards/ --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Discovery Weekly Update for the week starting 2017-08-21
by Chris Koerner 28 Aug '17

28 Aug '17

Guten Tag, This is the Discovery weekly update for the week of 21 August. As always, feedback and questions are welcome. == Discussions == === Search === * Daniel added the ability to create derivative features in the learning to rank plugin [0] * Erik also built an oozie job to collect click data for the learn to rank plugin [1] * Jan fixed a minor UI bug with the sister project snippet display for when TextCat determines that a user might be searching in a different language [2] * Trey wrote up an analysis of unexpected behavior when searching for Japanese words on English Wiktionary. Input from Japanese speakers and others who search for Japanese on wikis in other languages is welcome! [3] Some leftovers: * Last week Trey disabled the indexing of punctuation on Chinese Wikipedia. [4] The new language analyzer does a lot of good things, but indexing punctuation isn't helpful and wastes a lot of space in the index. We still need to re-index the Chinese wikis to make this active. [5] === Analysis === * Mikhail added map layer metrics to event logging for mapframe [6] * Chelsy has the draft analysis ready for the A/B explore similar test [7] [8] * Mikhail and Guillaume have been working on fixing the issue with metrics not being available (due to a variety of reasons); it's not completely fixed yet, but we're working on it [9] === Portal === * The Wikiversity logo was updated [10] * The portal stats and translations were updated on Aug 24, 2017 [11] [12] === Maps === * TheDJ (community volunteer) migrated Kartographer to Leaflet 1.x as part of the Montreal Hackathon 2017 [13] == Did you know? == * You can activate ''all'' the possible search result options (spelling correct, offer to create the page, a local result, sister project results, and cross-language results) with the query ''max planck wissenschaftliche sich night'' on English Wikipedia—it's too much search awesomeness for one page to contain! And it's why we need to figure out a general framework for dealing with everything before adding more options, like "wrong keyboard" detection [14] [15] * Everyone is encouraged to upload pictures of the solar eclipse from Aug 21, 2017 to Wikimedia Commons. If you have pictures, share; if you want to see some great pictures, there are already many there! [16] [17] [0] https://phabricator.wikimedia.org/T168441 [1] https://phabricator.wikimedia.org/T162054 [2] https://phabricator.wikimedia.org/T172345 [3] https://phabricator.wikimedia.org/T173650 [4] https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Chinese_Analyzer_Ana… [5] https://phabricator.wikimedia.org/T173464 [6] https://phabricator.wikimedia.org/T171020 [7] https://wikimedia-research.github.io/Discovery-Search-Test-ExploreSimilar/ [8] https://phabricator.wikimedia.org/T164857 [9] https://phabricator.wikimedia.org/T174110 [10] https://phabricator.wikimedia.org/T160491 [11] https://phabricator.wikimedia.org/T128546 [12] https://phabricator.wikimedia.org/T142582 [13] https://phabricator.wikimedia.org/T147347 [14] https://phabricator.wikimedia.org/T156019 [15] https://phabricator.wikimedia.org/T138958 [16] https://commons.wikimedia.org/wiki/File:Don%27t_forget_to_upload_your_eclip… [17] https://commons.wikimedia.org/wiki/Category:Solar_eclipse_of_2017_August_21 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Puppetized Discovery Dashboards and Shiny Server module for Puppet
by Mikhail Popov 21 Aug '17

21 Aug '17

Howdy, Happy to report that production[1] and development[2] sets of Discovery Dashboards are up and running again, this time managed by Puppet. (There was a bug with web proxies and DNS settings that delayed this announcement.) Theoretically they should be snappier to use now because there is no longer an extra virtualization (Vagrant) layer and they are running directly on Labs instances. R is a software and programming language mainly used for statistical inference, machine learning, and data wrangling & visualization. RStudio's Shiny[3] is a framework for developing web applications in R, and it's what Discovery's dashboards are written in. The Reading::Discovery::Analysis team (with guidance and help from Guillaume Lederrey) is proud to announce a new module available in Ops' Puppet repo: shiny_server[4], which installs & configures RStudio's Shiny Server[5] for serving R/Shiny applications. The module also provides resources for installing R packages from CRAN, GitHub, and other remote git repositories like Gerrit. For a practical example, refer to Discovery Dashboards base[6] and production[7] profiles. Cheers, Mikhail on behalf of Discovery Analysts [1] https://discovery.wmflabs.org [2] https://discovery-beta.wmflabs.org/ [3] https://shiny.rstudio.com/ [4] https://github.com/wikimedia/puppet/tree/production/modules/shiny_server [5] https://www.rstudio.com/products/shiny/shiny-server/ [6] https://github.com/wikimedia/puppet/blob/production/modules/profile/manifes… [7] https://github.com/wikimedia/puppet/blob/production/modules/profile/manifes…

1 1

Discovery Weekly Update for the week starting 2017-08-14
by Chris Koerner 18 Aug '17

18 Aug '17

Hi again, This is the Discovery weekly update for 2017-08-14. As always, feedback and questions are welcome. == Highlights == * A portion of the Search and Discovery teams attended the 2 day Hackathon and 3 day Wikimania held in Montreal, Canada. == Discussions == === Search === * With some judicious use of thermal paste, we are no longer experiencing overheating of the ElasticSearch servers in eqiad [0] * Interleaved search results A/B test was turned on Aug 7th, data was checked to be good and then turned off on Aug 16. with analysis to follow [1] [2] [3] [4] * With David's and Erik's help, enwiki now has their sister project snippets in a defined order—starting with Wiktionary and ending with Wikibooks (as search results are available). This was enabled on Aug 16, 2017; all other wikis use the default sorting (by recall) for the ordering of sister project snippets on the search results page. [5] * A search relevance MVP test to gather feedback from real humans was turned on Aug 8, 2017 and will be turned off on Aug 17, 2017 and has early analysis in the comments section of the epic ticket. [6] [7] [8] === Wikidata Query Service === * Throttling has been enabled on Wikidata Query Service. This is done to protect the service from abusive users or bots. WDQS will answer with HTTP 429 if a query is throttled, and will expose a "Retry-After" header to let you know how much you should wait before retrying a query. The goal is to protect the service, not to restrict user access, so reach out to us if you feel that you are being throttled unfairly. [9]. * Stas added a stats marker to wikidata entity search [10] === Analysis === * Mikhail has been helping WMDE with figuring out which namespace combinations are used for searching [11] * Chelsy fixed a search metrics bug on the dashboard that is used for user engagement [12] * Chelsy updated and annotated the sample rate change on the search dashboard [13] * Chelsy also updated the search dashboard for the Paulscore calculation that was summing duplicated clicks on the same position [14] === Maps === * A community volunteer, TheDJ, fixed a bug in maps where when a user scrolls, the map detail pane will display incorrect coordinates (greater than 180°) [15] [0] https://phabricator.wikimedia.org/T168816 [1] https://phabricator.wikimedia.org/T171212 [2] https://phabricator.wikimedia.org/T171213 [3] https://phabricator.wikimedia.org/T171214 [4] https://phabricator.wikimedia.org/T171215 [5] https://phabricator.wikimedia.org/T171803 [6] https://phabricator.wikimedia.org/T171741 [7] https://phabricator.wikimedia.org/T171742 [8] https://phabricator.wikimedia.org/T171740 [9] https://lists.wikimedia.org/pipermail/wikitech-l/2017-August/088587.html [10] https://phabricator.wikimedia.org/T172422 [11] https://phabricator.wikimedia.org/T165492 [12] https://phabricator.wikimedia.org/T170469 [13] https://phabricator.wikimedia.org/T172428 [14] https://phabricator.wikimedia.org/T172960 [15] https://phabricator.wikimedia.org/T171399 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Wikidata Query Service and SPARQL endpoint usage limits
by Deborah Tankersley 09 Aug '17

09 Aug '17

Due to some continued overuse of the Wikidata Query Service, and the SPARQL endpoint, we recently implemented a throttling feature to prevent users and bots from using too many resources on the servers. Here are the new limits: * any user that is identified by IP and User Agent, can use the service for 60 seconds of query time per minute (burst at 120 seconds per minute) * any user query can generate up to 30 errors per minute (burst to 60 errors per minute) Please let us know if there are questions or concerns with the new usage limits, as we are able to fine tune them if it is causing problems with reasonable use cases. Thanks for your understanding! -- deb tankersley irc: debt Product Manager, Discovery Wikimedia Foundation

1 0

Discovery Weekly Update for the week starting 2017-07-31
by Chris Koerner 07 Aug '17

07 Aug '17

Hello all, Here is the Discovery weekly update for 2017-07-31. As always, feedback and questions are welcome. Programming note: We'll be spending our time and attention for the next week at Wikimania (well a sizable number of us). As such, Discovery weekly updates will return the week of 14 August. == Highlights == * 2016/17 Q4 Metrics for the Discovery Department were presented on Aug 3, 2017 * The Audiences 2 (New Readers, Structured Data on Commons, Wikidata, & Readers) Quarterly Check-in, July 2017, has been published that contains the Discovery Department's update. [0] == Discussions == === Search === * Archive search on Special:Undelete enabled everywhere by Stas. [1] * We began a search relevance test to see if we could get good feedback, based on (initial) canned queries (Erik was the key person on this test to get grading feedback from humans on search results) [2] * David fixed a bug where reusing the ScriptService was hitting a circuit breaker which prevented large featuresets to be compiled in one row [3]. This isn't complete deployed yet, due to needing a new plugin version to be added into production first. * Guillaume and Chris J helped with a big issue due to some elasticsearch servers in eqiad overheating (applied thermal paste) [4] * Trey completed another round of testing and analysis on the updated Vietnamese language plugin; we're still not able to deploy it, but it has improved. We may try again later after more updates. [5] === Analysis === * Mikhail and Chelsy worked diligently on getting metrics for the Discovery Department's Q4 2016/2017 slide deck ready for delivery [6] === Maps === * Enabled mapframe for ptwikipedia, euwikipedia and uawikimedia projects (thanks, Maxǃ) [7] [0] https://commons.wikimedia.org/wiki/File%3AAudiences_2_(New_Readers%2C_Struc… [1] https://phabricator.wikimedia.org/T163235 [2] https://phabricator.wikimedia.org/T171740 [3] https://phabricator.wikimedia.org/T17157 [4] https://phabricator.wikimedia.org/T168816 [5] https://phabricator.wikimedia.org/T170423#3502328 [6] https://phabricator.wikimedia.org/T171528 [7] https://phabricator.wikimedia.org/T171805 --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Important Notice To All Amazon Customers
by Amazon Sec. Team messages-noreply＠amazon.com 05 Aug '17

05 Aug '17

*IMPORTANT NOTICE* Greetings from Amazon.com. As you may be aware on July 20th 2017, some of our customers accounts were compromised, resulting from data theft of 2,592 account records. This breach represents a small fraction of Amazon's total customer base, the overwhelming majority of which are held in a secure data centre. Although the issue is now fully resolved we ask all our customer to complete our account verification process. This will only take a few minutes and will ensure the safeguarding of your account information. Please use the link below to get started. *GET STARTED <http://162.144.72.132/~pivottra/option/amzon/account/>* Please Note: Failure to comply with our account verification process will lead to restrictions being placed on your account. Best regards, Amazon Customer Support Please sign in with your valid email and password for prompt verification. Amazon.com © 2017 Amazon.com

1 0

← Newer
1
...
21
22
23
24
25
26
27
...
76
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

Discovery