Wikitech-l September 2018

wikitech-l@lists.wikimedia.org

63 participants
74 discussions

Translations on hold until further notice
by Greg Grossmeier 10 Oct '18

10 Oct '18

tl;dr: Please refrain from adding new strings that require translations to MediaWiki and deployed extensions until further notice. Due to changes we are making in the way our translation systems process data we will be temporarily disabling the auto deployment of newly translated content until further notice. The goal of these changes will be to improve the evaluation of content submitted to our translation services. This means developers should not merge new translatable strings into MediaWiki Core nor Wikimedia deployed extensions until we re-enable the system. This is to prevent longer term issues where local wiki admins overwrite a string and thus prevent updates from new and improved translations. We apologize for any inconvenience this delay in service may cause and will alert you as soon as this maintenance is completed. Greg Grossmeier -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

13 19

Datacenter switchover and switchback
by Alexandros Kosiaris 05 Oct '18

05 Oct '18

Hello everyone, This is to inform you that there will be a datacenter switchover and switchback on the next few weeks. The timeline's are Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC Switchback: Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC For the duration of the switchover (1 month), deployers are kindly requested to refrain from large db schema changes and avoid deploying any kind of new feature that requires creation of tables. There will be a train freeze in the week of Sept 10th and Oct 8th. The net effect of the switchover and switchback for volunteers is expected to be some minutes of inability to save an edit. For readers, everything will be as usual. The tracking task for interested parties is https://phabricator.wikimedia.org/T199073 Regards, -- Alexandros Kosiaris <akosiaris(a)wikimedia.org>

7 10

Upcoming Search Platform Office Hours—October 3rd
by Trey Jones 03 Oct '18

03 Oct '18

The Search Platform Team <https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> will be holding office hours the first Wednesday of each month, starting next week. Come ask us anything about Wikimedia search! We’re particularly interested in: * Opportunities for collaboration—internally or externally to the Wikimedia Foundation * Challenges you have with on-wiki search, in any of the languages we support But we're happy to talk about anything search-related. Details for our next meeting: Date: Wednesday, October 3rd, 2018 Time: 15:00 GMT / 08:00 PDT / 11:00 EDT / 17:00 CEST Google Meet link: https://meet.google.com/vyc-jvgq-dww *N.B.:* Google Meet System Requirements <https://support.google.com/meet/answer/7317473> See you there! —Trey Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation

1 2

Improving the mediawiki.org frontpage
by Andre Klapper 30 Sep '18

30 Sep '18

Hi, I'd like to discuss potential improvements to www.mediawiki.org (the front page itself), both regarding content and presentation, as I see several issues with the current page. I propose to discuss this in several stages: * Defining audiences, * defining content per group, * defining presentation and layout. My main intention are improvements to allow our audiences to better find the relevant information that they are looking for. We might not be able to make everything perfect but we can certainly try to make it better? If this sounds interesting, please take a look at https://www.mediawiki.org/wiki/MediaWiki/Homepage_improvements_2018 and let's discuss on the corresponding discussion page and work on this? Thank you! andre -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

1 0

Fwd: [Wikimedia-l] Wikipedia in an abstract language
by Pine W 29 Sep '18

29 Sep '18

Forwarding because this (ambitious!) proposal may be of interest to people on other lists. I'm not endorsing the proposal at this time, but I'm curious about it. Pine ( https://meta.wikimedia.org/wiki/User:Pine ) ---------- Forwarded message --------- From: Denny Vrandečić <vrandecic(a)gmail.com> Date: Sat, Sep 29, 2018 at 6:32 PM Subject: [Wikimedia-l] Wikipedia in an abstract language To: Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org> Semantic Web languages allow to express ontologies and knowledge bases in a way meant to be particularly amenable to the Web. Ontologies formalize the shared understanding of a domain. But the most expressive and widespread languages that we know of are human natural languages, and the largest knowledge base we have is the wealth of text written in human languages. We looks for a path to bridge the gap between knowledge representation languages such as OWL and human natural languages such as English. We propose a project to simultaneously expose that gap, allow to collaborate on closing it, make progress widely visible, and is highly attractive and valuable in its own right: a Wikipedia written in an abstract language to be rendered into any natural language on request. This would make current Wikipedia editors about 100x more productive, and increase the content of Wikipedia by 10x. For billions of users this will unlock knowledge they currently do not have access to. My first talk on this topic will be on October 10, 2018, 16:45-17:00, at the Asilomar in Monterey, CA during the Blue Sky track of ISWC. My second, longer talk on the topic will be at the DL workshop in Tempe, AZ, October 27-29. Comments are very welcome as I prepare the slides and the talk. Link to the paper: http://simia.net/download/abstractwikipedia.pdf Cheers, Denny _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

1 0

SMWCon Fall 2018 - December 12th to 14th, Regensburg, Germany
by planetenxin 28 Sep '18

28 Sep '18

Dear users, developers and all people interested in semantic wikis, We are happy to announce SMWCon Fall 2018 - the 15th Semantic MediaWiki Conference: Dates: December 12th to December 14th 2018 (Wednesday to Friday). Location: TechBase Regensburg, Franz-Mayer-Straße 1, 93053 Regensburg, Germany Conference page: https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2018 Ticket Shop: https://en.xing-events.com/SMWCon_Fall_2018 SMWCon Fall 2018 will be supported by gesinn.it GmbH & Co. KG [0]. Due to a challenging conference room situation, SMWCon Fall will take place very late this year. What's good about it: participants will have the opportunity to enjoy winter atmosphere (including "Christkindlmarkt") in Regensburg :-) This time, we start Wednesday with two conference days (including a short introduction to SMW) followed by a Tutorial / Hackathon Day on Friday. TICKET SHOP IS ALREADY OPEN Contributing to the conference: If you want to present your work in the conference, please go to the conference page and add your talk there. To create an attractive program for the conference, we will later ask you to give further information about your proposals. Looking forward to meet you in Regensburg! /Alexander Gesinn on behalf of the gesinn.it organization team [0] http://gesinn.it, http://semantic.wiki

1 0

Re: [Wikitech-l] Translations on hold until further notice
by John Bennett 27 Sep '18

27 Sep '18

Hello, This update is to add some additional information we are now able to share in regard to why translation updates are on hold. The Security team and others at the Wikimedia Foundation are engaged in a security event involving our translation services. No Wikimedia users or their data are currently affected. We made the decision to temporarily disable translation updates until suitable countermeasures can be applied and at this point reinstatement of translation updates is to be determined. At the resolution of this event the Security team will publish a summary blog post (https://phabricator.wikimedia.org/phame/blog/view/13/) including additional details as appropriate. Thank you for your patience and understanding while we work to better protect the community. John Bennett Director of Security, Wikimedia Foundation

2 1

TechCom Radar 2018-09-26
by Kate Chapman 27 Sep '18

27 Sep '18

Hi All, Here are the minutes from this week's TechCom meeting: * RFC Approved: Harmonise the identification of requests across our stack <https://phabricator.wikimedia.org/T201409> * Hosted IRC Meeting on: RFC: Modern Event Platform: Scalable Event Intake Service <https://phabricator.wikimedia.org/T201963> This RFC outlines the current state of our event intake systems and some possible ways forward. It was agreed in the meeting that research will be done into refactoring EventBus before looking at other options and if GET event intake needs to still be supported. * Minutes: <https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.20…> * Log: <https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.20…> * Hosting IRC discussion on 3 October 2pm PST(21:00 UTC, 23:00 CET) in #wikimedia-office: Spec for representing multiple content objects per revision (MCR) in XML dumps <https://phabricator.wikimedia.org/T199121> * Looking for feedback on the latest draft of Architecture Principles prior to TechConf, we will be incorporating this feedback prior to the session on the Architecture Principles at TechConf. * Principles: <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/MediaWiki_Plat…> * Talk page for comments: <https://www.mediawiki.org/wiki/Talk:Wikimedia_Technical_Committee/MediaWiki…> * Phabricator ticket: <https://phabricator.wikimedia.org/T204142> You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. If you prefer you can subscribe to our newsletter here: <https://www.mediawiki.org/wiki/Newsletter:TechCom_Radar> Thanks, Kate -- Kate Chapman TechCom Facilitator (Contractor)

1 0

RFC Discussion Reminder: RFC: Modern Event Platform: Scalable Event Intake Service
by Kate Chapman 25 Sep '18

25 Sep '18

Hi All, A friendly reminder that TechCom is hosting an IRC meeting tomorrow (Wednesday 26 September) regarding RFC: Modern Event Platform: Scalable Event Intake Service. <https://phabricator.wikimedia.org/T201963> This RFC outlines the current state of our event intake systems and some possible ways forward. The outcome of this RFC should be to choose an implementation option. This RFC is not intended as a design document. The meeting is scheduled for Wednesday 26 September at 2pm PST(21:00 UTC, 23:00 CET) in #wikimedia-office. If you haven't joined a #wikimedia-office meeting before more information can be found here: <https://meta.wikimedia.org/wiki/IRC_office_hours#How_to_participate> More information regarding the TechCom RFC process is available here: <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Processes#RFC_…> Thanks, Kate -- Kate Chapman TechCom Facilitator (Contractor)

1 0

Production Excellence: September 2018
by Krinkle 25 Sep '18

25 Sep '18

📘 Read this post on Phabricator at https://phabricator.wikimedia.org/phame/live/1/post/119/ ________________________________ How’d we do in our strive for operational excellence last month? Read on to find out! - Month in numbers. - Current problems. - Highlighted stories. ## Month in numbers * 1 documented incident since August 9. [1] * 113 Wikimedia-prod-error tasks closed since August 9. [2] * 99 Wikimedia-prod-error tasks created since August 9. [3] ## Current problems Frequent: * [MediaWiki-Logging] Exception from Special:Log (public GET). – https://phabricator.wikimedia.org/T201411 * [Graph] Warning "data error" from ApiGraph in gzdecode. – https://phabricator.wikimedia.org/T184128 * [RemexHtml] Exception "backtrack_limit exhausted" from search index jobs. – https://phabricator.wikimedia.org/T201184 Other: * [MediaWiki-Redirects] Exception from NS_MEDIA redirect (public GET). – https://phabricator.wikimedia.org/T203942 This is an oldie: (Well..., it's an oldie where I come from... 🎸) * [FlaggedRevs] Exception from Special:ProblemChanges (since 2011). – https://phabricator.wikimedia.org/T176232 Terminology: * An Exception (or fatal) causes user actions to be aborted. For example, a page would display "Exception: Unable to render page", instead the article content. * A Warning (or non-fatal, or error) can produce page views that are technically unaware of a problem, but may show corrupt or incomplete information. For example, an article would display the word "null" instead of the actual content. Or, a user may be told "You have (null) new messages." The combined volume of infrequent non-fatal errors is high. This limits our ability to automatically detect whether a deployment caused problems. The “public GET” risks in particular can (and have) caused alerts to fire that notify Operations of wikis potentially being down. Such exceptions must not be publicly exposed. With that behind us... Let’s celebrate this month’s highlights! ## *️⃣ Quiz defect – "0" is not nothing! Tyler Cipriani (Release Engineering) reported an error in Quiz. Wikiversity uses Quiz for interactive learning. Editors define quizzes in the source text (wikitext). The Quiz program processes this text, creates checkboxes with labels, and sends it to a user. When the sending part failed, "Error: Undefined index" appeared in the logs. Volunteer Umherirrender investigated. A line in the source text can: define a question, or an answer, or nothing at all. The code that creates checkboxes needs to decide between "something" and "nothing". The code utilised the PHP "if" statement for this, which compares a value to True and False. The answers to a quiz can be any text, which means PHP first transforms the text to one of True or False. In doing so, values like "0" became False. This meant the code thought "0" was not an answer. The code responsible for sending checkboxes did not have this problem. When the code tried to access the checkbox to send, it did not exist. Hence, "Error: Undefined index". Umherirrender fixed the problem by using a strict comparison. A strict comparison doesn't transform a value first, it only compares. – https://phabricator.wikimedia.org/T196684 ## *️⃣ PageTriage enters JobQueue for better performance Kosta Harlan (from Audiences's Growth team) investigated a warning for PageTriage. This extension provides the New Pages Feed tool on the English Wikipedia. Each page in the feed has metadata, usually calculated when an editor creates a page. Sometimes, this is not available. Then, it must be calculated on-demand, when a user triages pages. So far, so good. The information was then saved to the database for re-use by other triagers. This last part caused the serious performance warning: "Unexpected database writes". Database changes must not happen on page views. The database has many replicas for reading, but only one "master" for all writing. We avoid using the master during page views to make our systems independent. This is a key design principle for MediaWiki performance. [5] It lets a secondary data centre build pages without connecting to the primary (which can be far away). Kosta addressed the warning by improving the code that saves the calculated information. Instead of saving it immediately, an instruction is now sent via a job queue, after the page view is ready. This job queue then calculates and saves the information to the master database. The master synchronises it to replicas, and then page views can use it. – https://phabricator.wikimedia.org/T199699 / https://gerrit.wikimedia.org/r/455870 ## *️⃣ Tomorrow, may be sooner than you think After developers submit code to Gerrit, they eagerly await the result from Jenkins, an automated test runner. It sometimes incorrectly reported a problem with the MergeHistory feature. The code assumed that the tests would finish by "tomorrow". It might be safe to assume our tests will not take one day to finish. Unfortunately, the programming utility "strtotime", does not interpret "tomorrow" as "this time tomorrow". Instead, it means "the start of tomorrow". In other words, the next strike of midnight! The tests use UTC as the neutral timezone. Every day in the 15 minutes before 5 PM in San Francisco (which is midnight UTC), code submitted to Code Review, could have mysteriously failing tests. – Continue at https://gerrit.wikimedia.org/r/452873 ## *️⃣ Continuous Whac-A-Mole In August, developers started to notice rare and mysterious failures from Jenkins. No obvious cause or solution was known at that time. Later that month, Dan Duvall (Release Engineering team) started exploring ways to run our tests faster. Before, we had many small virtual servers, where each server runs only one test at a time. The idea: Have a smaller group of much larger virtual servers where each server could run many tests at the same time. We hope that during busier times this will better share the resources between tests. And, during less busy times, allow a single test to use more resources. As implementation of this idea began, the mysterious test failures became commonplace. "No space left on device", was a common error. The test servers had their hard disk full. This was surprising. The new (larger) servers seemed to have enough space to accommodate the number of tests it ran at the same time. Together with Antoine Musso and Tyler Cipriani, they identified and resolved two problems: 1) Some automated tests did not clean up after themselves. 2) The test-templates were stored on the "root disk" (the hard drive for the operating system), instead of the hard drive with space reserved for tests. This root disk is quite small, and is the same size on small servers and large servers. – https://phabricator.wikimedia.org/T202160 / https://phabricator.wikimedia.org/T202457 ## 🎉 Thanks! Thank you to everyone who has helped report, investigate, or resolve production errors. Including: Tpt Ankry Daimona Legoktm Volker_E Pchelolo Dan Duvall Gilles Dubuc Daniel Kinzler Umherirrender Greg Grossmeier Gergő Tisza (Tgr) Sam Reed (Reedy) Giuseppe Lavagetto Brad Jorsch (Anomie) Tim Starling (tstarling) Kosta Harlan (kostajh) Jaime Crespo (jcrespo) Antoine Musso (hashar) Roan Kattouw (Catrope) Adam WMDE (Addshore) Stephane Bisson (SBisson) Niklas Laxström (Nikerabbit) Thiemo Kreuz (thiemowmde) Subramanya Sastry (ssastry) This, that and the other (TTO) Manuel Aróstegui (Marostegui) Bartosz Dziewoński (matmarex) James D. Forrester (Jdforrester-WMF) Thanks! Until next time, – Timo Tijhof ________________________________ Further reading: * August 2018 edition. – https://lists.wikimedia.org/pipermail/wikitech-l/2018-August/090594.html * July 2018 edition. – https://lists.wikimedia.org/pipermail/wikitech-l/2018-July/090363.html Footnotes: [1] Incidents. – https://wikitech.wikimedia.org/wiki/Special:AllPages?from=Incident+document… [2] Tasks closed. – https://phabricator.wikimedia.org/maniphest/query/wOuWkMNsZheu/#R [3] Tasks opened. – https://phabricator.wikimedia.org/maniphest/query/6HpdI76rfuDg/#R [4] Quiz on Wikiversity. – https://en.wikiversity.org/wiki/How_things_work_college_course/Conceptual_p… [5] Operate multiple datacenters. – https://www.mediawiki.org/wiki/Requests_for_comment/Master-slave_datacenter…

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2018