Wikitech-l October 2017

wikitech-l@lists.wikimedia.org

53 participants
75 discussions

Did something change in beta cluster configuration around September 16 2017?
by Željko Filipin 27 Oct '17

27 Oct '17

TLDR: Did something change in beta cluster configuration around September 16 2017? For HTML version of this e-mail go to https://phabricator.wikimedia.org/T179157 RelatedArticles repository has Selenium tests (tests/selenium). They are running: - every time code is pushed to gerrit targeting a MediaWiki on a Jenkins slave virtual machine, - daily targeting beta cluster (en.wikipedia.beta.wmflabs.org). A reported in T176315, until about September 16 2017 selenium-RelatedArticles-jessie daily job was able to create pages at beta cluster. The pages are created via action API using nodemw NPM package. Code is in mediawiki/core. Since about September 16 2017, selenium-RelatedArticles-jessie started failing with 😳 Edit failed: Error: Error returned by API: Wikipedia has restricted the ability to create new pages. You can go back and edit an existing page, or [[Special:UserLogin|log in or create an account]]. We only keep jobs for the last 30 days so there is no record of failures available in Jenkins. 🙁 The workaround for the failure is to log in before creating a page. I have implemented it on October 3 2017. I have talked with @hashar about it and he says: In T176315#3702865, @hashar wrote: wmf-config/InitialiseSettings.php 'groupOverrides' => [ 'enwiki' => [ '*' => [ 'createpage' => false ], // See P2059 P2059 Anonymous users have been prevented from creating new pages since 2005! What is strange is that RelatedArticles tests were not failing until about September 16. I have implemented logging in via the API on October 3, so I am pretty sure the tests did not log in before that date. I do not even know where to start looking. Any help is appreciated. Željko Filipin

1 0

2017-10-18 Scrum of Scrums meeting notes
by Grace Gellerman 25 Oct '17

25 Oct '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-10-18 *= 2017-10-18=* == Callouts == * https://phabricator.wikimedia.org/T177765 Reading Web -> Security == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: Apps offsite this week, 5.7.0 in public beta, will be released next week ( https://phabricator.wikimedia.org/project/view/2899/ ) - Visual updates to onboarding, iOS 11 support, iPhone X support ==== Android native app ==== * Blocked by: * Blocking: * Updates: ==== Reading Web ==== * Blocked by: https://phabricator.wikimedia.org/T177765 [security] * Blocking: * Updates: - Investigating ChromeDriver for PDF generation. Security review has been requested. OCG is off. - Print styles have been deployed to Vector. ==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ==== Multimedia ==== * Blocked by: * Blocking: * Updates: MediaInfo work progressing, ramping up over the next few months. 3D is stalled currently on consideration of changes to the upload pipeline to handle patent/weapons concerns. ==== Discovery ==== * Blocked by: * Blocking: * Updates: - Working on serving SVGs on wikipedia.org portal - Making progress on the portal automation tool (to automate portal builds) ====== Maps ===== * Blocked by: N/A * Blocking: N/A * Updates: Hanging updates solved Most of team will be at SOTM (State of The Map) US next week === Community Tech === * Blocked by: * Blocking: * Updates: === Contributors === ==== Editing ==== * Blocked by: * Blocking: * Updates: ==== Global Collaboration ==== * Blocked by: No one * Blocking: No one (we think) * Updates: ** One cause of ORES-on-watchlist slowness fixed (removed STRAIGHT_JOIN at Bawolff's recommendation) https://phabricator.wikimedia.org/T176456 ** Compact Language Links deployed on dewiki ** Continuing work on porting ContentTranslation to VE, and addressing bugs/feedback from RCFilters release ** Team out at an offsite next week (Oct 23-27), some people traveling on Friday (Oct 20) ==== UI Standardization ==== * Blocked: icon feedback appreciated, see below ** OOUI v0.24.0 released, https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… *** 5 breaking changes, **** Dropped 'MediaWiki' backwards-compatibility theme, replaced by 'WikimediaUI' and moved, deleted different icons * Ongoing: ** OOUI: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432 ** Aligning arbitrary, historically grown colors across products to WikimediaUI color palette https://phabricator.wikimedia.org/T148708 === Community Tech === * Working on GlobalPreferences * Unicode sections going live very soon == Technology == === Analytics === * Blocked by: * Blocking: * Updates: - ** Developed new editing- and content-related metrics for AQS together with Services team (for Wikistats2 back-end) - ** Worked on Wikistats2 front-end: topic selector, views for new AQS metrics - ** Replacement boxes for eventlogging databases are racked and have been reimaged, migration soon - ** Assisted Reading team with refinement of EventLogging:Popups data into Hive for scalable A/B test === Cloud Services === * Blocked by: * Blocking: * Updates: === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Reconstructing banner and landing page impression data from Hive ** May have fixed our replication lag (mysql's automatic temp table drops were being replicated out of order, leaving lots of open temp tables on replicas. Explicit drops seem to be fine). ** Using contact_id from email links (and adding a verification hash) to avoid creating duplicate contacts ** Fixing issues caused by Safari's new privacy policies *** Payment providers that require cookies can't be in an iframe *** Using a different method to record banner impressions ** Adding more stats to grafana ** Logging client-side errors on donation forms === MediaWiki Platform === * Blocked by: * Blocking: * Updates: === Performance === * Blocked by: * Blocking: * Updates: === Release Engineering === * Blocked by: None * Blocking: None * Updates: None === Research === * Blocked by: * Blocking: * Updates: === Scoring Platform === * Blocked by: * Blocking: * Updates: === Search Platform === * Blocked by: none * Blocking: none * Updates: * Discussing how to improve relevance surveys and A/B tests ( https://phabricator.wikimedia.org/T171215#3677637) * A/B test for using MLR on 18 wikis finished, analysis next ( https://phabricator.wikimedia.org/T177490) * Running A/B test with relaxing match requirements on search with multiple terms (https://phabricator.wikimedia.org/T177502) * Wikidata configured for indexing P31 and P279, reindexing and working on integrating them to search * Working on porting Selenium tests from Ruby to JS * Working on upgrade to Elastic 5.5 * Working on indexing Wikidata descriptions and adding them to fulltext search * WDQS had brief partial outage due to overactive client, working on some fixes ( https://wikitech.wikimedia.org/wiki/Incident_documentation/20171018-wdqs) === Security === * Blocked by: None * Blocking: Whoever is waiting for security review * Updates: * Reviews: * pdfrw * Extension:WikipediaExtracts * New vega library === Services === * Blocked by: none * Blocking: none * Updates: ** Deployed analytics edit metrics REST API *** Check it out! https://wikimedia.org/api/rest_v1/ ** Removed deprecated /revision hierarchy from REST API ** Deployed reading lists API to beta for Reading to test ** Started testing summary endpoint move to Cassandra 3 in production on ruwiki === Technical Operations === * Blocked by: * Blocking: * Updates: == Wikidata == * QUnit 2 update unblocked for real now: https://phabricator.wikimedia.org/T177764 * Investigating workarounds and performance improvements to enable RecentChanges integration again: https://phabricator.wikimedia.org/T177707 * Working on persisting all elements a Lexeme is made of (Senses and Forms): https://phabricator.wikimedia.org/T173744 == German Technical Wishlist == * Blocked by: * Blocking: * Updates:

1 0

PHPUnit error: Cannot access the database: Unknown database
by Sebastian Berlin 25 Oct '17

25 Oct '17

I'm having trouble running PHPUnit for the extension I'm developing (Wikispeech), in Vagrant. I followed the instructions on https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_f…, i.e. `tests/phpunit/phpunit.php extensions/Wikispeech/tests`, and got the error: > Fatal error: Uncaught exception 'Wikimedia\Rdbms\DBConnectionError' with > message 'Cannot access the database: Unknown database > 'extensions/Wikispeech/tests/phpunit/ApiWikispeechTest.php' (127.0.0.1)' in > /vagrant/mediawiki/includes/libs/rdbms/database/Database.php:800 > For full log, see https://phabricator.wikimedia.org/P6176. Does anyone know what's causing this? Previously I've run the tests with `phpunit extensions/Wikispeech/tests` which I realize isn't the correct way, after reading the Wiki-page again. However, this didn't work after I reinstalled Vagrant (due to various other problems). All the best, *Sebastian Berlin* Utvecklare/*Developer* Wikimedia Sverige (WMSE) E-post/*E-Mail*: sebastian.berlin(a)wikimedia.se Telefon/*Phone*: (+46) 0707 - 92 03 84

2 2

Upgrade of QUnit from 1.x to 2.x is underway
by Krinkle 25 Oct '17

25 Oct '17

TL;DR: MediaWiki core is upgrading its version of QUnit from 1.x to 2.x. This means extensions or skins with QUnit tests must now be compatible with 2.x. See https://phabricator.wikimedia.org/T170515 and https://qunitjs.com/upgrade-guide-2.x/. Hi all, ### Deprecated API In 2014, QUnit started to overhaul its API, to be more robust and better support async workflows. The most notable change was the removal of global and static functions, in favour of more contextual methods. The first part of this released in 1.15, and more was gradually introduced in later releases. The vast majority of our codebases are already using the new interfaces. In fact, the vast majority of our QUnit tests were written after 2014 and never used the old interfaces in the first place. For a short list of removed functions, see https://phabricator.wikimedia.org/T170515. If you find a QUnit Jenkins job for a MediaWiki extension or skin repo starts failing, it is most likely due to this. Look for errors such as "QUnit.start undefined", "test.callback is not a function", "QUnit.asyncTest is undefined", and "QUnit.push is undefined", There are also some methods that have been deprecated over the past few years. These still work in QUnit 2. Please take a moment to familiarise yourself with the renamed methods and new methods. Doing so will avoid confusion when reading new code that uses them. See https://qunitjs.com/upgrade-guide-2.x/. ### New features The 'setup' and 'teardown' module hooks are now called 'beforeEach' and 'afterEach'. The old names still work, but the new names clarify that these hooks are run for each QUnit.test(). QUnit 2.0 also adds new 'before' and 'after' hooks, which run only once per module. This is somewhat analogous to use of setUpBeforeClass() in PHPUnit. Since QUnit 1.16, QUnit.test() supports returning a Promise from the test callback. This automatically attaches an assert.async() handler and waits for the promise to complete. It also asserts that the Promise will be resolved, and fails the test if rejected. This helps avoid a common pitfal where a test could timeout when forgetting to attach a promise.fail() handler. ## Upgrade The version used on Special:JavaScriptTest has already been upgraded. – https://gerrit.wikimedia.org/r/365757. The copy used for command-line usage (grunt-karma) and Jenkins will be upgraded by https://gerrit.wikimedia.org/r/367838 -- Timo

1 1

Disabling Messaging Fallbacks for Language Analysis
by Trey Jones 24 Oct '17

24 Oct '17

SUMMARY: The Search Platform team (formerly part of Discovery) is planning to fix a long-standing search bug on many wiki projects by disabling the code in CirrusSearch that re-uses the “fallback” languages (which are specified for user interface or system messages) for the language analysis modules (which are used to index words in search). Deployment is planned to start the week of October 9, 2017. Messaging fallbacks specify what language to show a message in when there is no message available in the language of a given wiki. A language analysis module is language-specific software that processes text to improve searching—so that, for example, searching for a given word will find related forms of that word, like "hope, hopes, hoping, hoped" or "resume, resumé, résumé" on English-language wikis. Fallback languages for system messages make sense for historical and cultural reasons—a reader of the Chechen Wikipedia is more likely to understand a user interface or system message in Russian than in French, Greek, Hindi, Italian, or Japanese—but the fallbacks don't necessarily make any linguistic sense. Chechen and Russian, for example, are from unrelated language families; while the languages have undoubtedly influenced one another, their grammars are completed different. We will deploy the software change that disables using messaging fallbacks for language analysis fallbacks in about two weeks (targeting the week of October 9, 2017), with any cross-language analysis exceptions explicitly configured in a new manner. Changes will not immediately happen to all affected wikis because each wiki in each language will need to be re-indexed, which is a separate process that takes time. There may also be other delays caused by Elasticsearch upgrades or other changes that need immediate attention. You can also track progress of the tasks on Phabricator[1] or read more, see examples, and get the full list of languages affected on MediaWiki.[2] [1] https://phabricator.wikimedia.org/T147959 [2] https://www.mediawiki.org/wiki/Wikimedia_Discovery/Disabling_Messaging_Fall… Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation

1 1

Discovery Weekly Update for the week starting 2017-10-16
by Deborah Tankersley 24 Oct '17

24 Oct '17

This is the weekly update for the week starting '''2017-10-16''' ==Highlights== * Wikidata Query Service had a partial outage on 18 Oct 2017: a single client started to run an unusually high number of queries on WDQS and the overload was not prevented by our current throttling nor was it automatically detected and isolated—detailed information is on the mailing list. [0] * Discovery and Legal chatted on Oct 18 about supporting only CC0 data in the .map data namespace on Commons, in response to a community conversation; a detailed response will be prepared and released next week. [1] [2] ==Discussions== === Search === * We added more namespaces to the search results for Bengali Wikisource (Special:Search—add) [3] * David fixed an issue where users couldn't click on svwikisource search results when running a crosswiki search on svwiki; this will be in production during the week of Oct 24th [4] * David was also working on finishing up deployment-prep elasticsearch cluster to 5.5.x [5] * David launched the A/B test for testing the relaxing of the retrieval query filter using two different configurations (the test is expected to last a week) [6] * We wrapped up a series of adding content namespaces to wikis that were accidentally excluded from the default results of Special:Search (thanks to TheDJ for discovering this bug and to David for his quick fixing of the issues once the community consultation was completed by Chris K) [7] * Erik deployed a fix for when the rebuildtextindex fails when searchindex on InnoDB (no support for MyISAM) and it went into production on Oct 19 [8] === Analysis === * Mikhail finished up tasks related to establishing the baseline of metrics for SDoC—looking at querying databases [9] === Portal === * The Wikipedia.org portal page stats and translations were updated on Oct 19, 2017 [10] [11] === Maps === * MaxSem and Gehel fixed a weird bug where tileshell wasn't exiting after after being notified of updates, effectively stopping the OSM data in production from being regenerated; data updates are now back and running once a day [12] [0] https://lists.wikimedia.org/pipermail/wikidata/2017-October/011280.html [1] https://commons.wikimedia.org/wiki/Commons:Village_pump/Proposals#Proposal_… [2] https://phabricator.wikimedia.org/T154071 [3] https://phabricator.wikimedia.org/T178041 [4] https://phabricator.wikimedia.org/T177913 [5] https://phabricator.wikimedia.org/T178413 [6] https://phabricator.wikimedia.org/T177502 [7] https://phabricator.wikimedia.org/T170473 [8] https://phabricator.wikimedia.org/T177477 [9] https://phabricator.wikimedia.org/T177356 [10] https://phabricator.wikimedia.org/T128546 [11] https://phabricator.wikimedia.org/T142582 [12] https://phabricator.wikimedia.org/T177389 --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R -- deb tankersley Product Manager, Discovery Wikimedia Foundation

1 0

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 24 Oct '17

24 Oct '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow 3-4 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/ wiki/Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting are: @addshore & @CFisch_WMDE. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

MediaWiki-CodeSniffer 14.1.0 released
by Kunal Mehta 24 Oct '17

24 Oct '17

Hello! MediaWiki-CodeSniffer 14.1.0 is now available for use in your MediaWiki extensions and other projects. This release features improves to the documentation-related sniffs and new upstream releases. 14.0.0 was tagged, but 14.1.0 should be used as it includes a newer upstream version. Here's the changelog since 13.0.0 * Update PHP_CodeSniffer to 3.1.1 (Paladox) * Add sniff for @params instead of @param (Umherirrender) * Better distinguish "one space before brace" and "brace on same line" (Florian Schmidt) * Typo fix in docs (MarcoAurelio) * Unwrap types in function docs from {} (Umherirrender) * Update PHP_CodeSniffer to 3.1.0 from 3.0.2 (Paladox) * Validate doc syntax (Umherirrender) Libraryupgrader and Umherirrender submitted patches for most Gerrit repositories :) Thanks, -- Legoktm

1 0

New Developers Quarterly Report's first edition
by Srishti Sethi 22 Oct '17

22 Oct '17

Hello everyone, I would like to share the first edition of the New Developers Quarterly Report <https://www.mediawiki.org/wiki/New_Developers/Quarterly/2017-10> that the Developer Relations team has produced. This report covers metrics, survey analysis and lessons learned from new developers focused activities in the previous quarter (July-September 2017). If you have questions and feedback that you would like to share with us, please add them on the discussion <https://www.mediawiki.org/wiki/Talk:New_Developers/Quarterly/2017-10> page. To receive a notification when a new report is published, subscribe here <https://www.mediawiki.org/wiki/Special:Newsletter/11/subscribe>. We plan to release a report every quarter and take action items identified from the key findings for improving our existing methods and processes. The next release will be in January 2018. If you have any questions, comments, and concerns, we will be more than happy to hear them! Thanks, Srishti -- Srishti Sethi Developer Advocate Technical Collaboration team Wikimedia Foundation https://www.mediawiki.org/wiki/User:SSethi_(WMF)

5 4

need a URL encoding expert
by Ryan Kaldari 21 Oct '17

21 Oct '17

Could someone knowledgeable about URL encoding take a look at this pull request? Thanks! https://github.com/wikimedia/DeadlinkChecker/pull/26/files

4 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2017