Wikitech-l November 2017

wikitech-l@lists.wikimedia.org

60 participants
56 discussions

Re: [Wikitech-l] [AI] New ORES FAQ
by Aaron Halfaker 22 Nov '17

22 Nov '17

Rainbow colors! Here's a plain text one for those who can't read the fancy colors: Hi Everyone, I've been working together with Adam and Aaron from the Scoring Platform team to identiy technical documentation in need of improvement. This past quarter, the team and I worked together on a FAQ for ORES[1], which is now up and running. If you are interested in ORES, AI, or technical documentation in general, ORES documentation is a great place to contribute. We've got answers for most of the common questions, but we haven't finished 100% yet. You can help us by asking new questions on the discussion page for the FAQ[2]. I hope you can join us in editing BOLDly[3] by working on documentation that makes our projects more accessible and easier to understand. Documentation like this is critical for our open software environment to work. I'm organizing a special interest group[4] to address documentation needs. If you have any questions or comments, please dont hesitate to reach out and share! [1] https://mediawiki.org/wiki/ORES/FAQ [2] https://mediawiki.org/wiki/Talk:ORES/FAQ [3] https://en.wikipedia.org/wiki/Wikipedia:BOLD,_revert,_discuss_cycle [4] https://wikitech.wikimedia.org/wiki/SIGDOCS With much Kindness, Sarah R. Rodlund Senior Project Coordinator-Product & Technology, Wikimedia Foundation srodlund(a)wikimedia.org

1 0

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 21 Nov '17

21 Nov '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/ wiki/Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting are: @Lucas_WMDE & @Thiemo_WMDE. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Discovery Weekly Update for the week starting 2017-11-13
by Chris Koerner 20 Nov '17

20 Nov '17

Hello, Here is the Discovery weekly update for 2017-10-02. As always, feedback and questions are welcome. *Programing note*: Due to the US holiday there will be no status update for the week of 20 November. Updates will resume the week of 27 November. == Highlights == * There is a survey on French Wikipedia asking if they should replace static maps with dynamic maps and to define best practices. [0] * An update on the Discovery team and projects was shared [1] == Discussions == === Search === * Erik did quite a bit of research into whether we should expand the rescore windows for learning to rank — it was decided it wouldn't be helpful [2] * David fixed a bug where the JobQueue was trying to report update metrics when response was not valid [3] * Gehel paid down some technical debt by creating a dedicated elastic component in the APT repository — which will help get the next upgrade more smoothly [4] * Stas fixed bug where removed Wikidata description was still showing when searching [5] === Analysis === * Mikhail provided analysis of external & internal WDQS traffic as preparation for sizing of internal WDQS cluster. [6] === WDQS === * Stas reloaded WDQS database to enable external links and performance improvements [7] [0] https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Sondage/Remplacement_des_carte… [1] https://lists.wikimedia.org/pipermail/discovery/2017-November/001601.html [2] https://phabricator.wikimedia.org/T162369 [3] https://phabricator.wikimedia.org/T180298 [4] https://phabricator.wikimedia.org/T179964 [5] https://phabricator.wikimedia.org/T180382 [6] https://phabricator.wikimedia.org/T179850 [7] https://phabricator.wikimedia.org/T176593 --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Help test longer edit summaries (and other comments) on Beta Cluster
by Brad Jorsch (Anomie) 20 Nov '17

20 Nov '17

I've just now enable the feature flag on the Beta Cluster[1] that allows MediaWiki to store comments longer than 255 bytes. The web UI has not been updated to allow longer comments in places where it enforces a limit, such as the edit summary box. But if you use the API to edit, or perform page moves or do other things where long comments could be entered and were truncated, you should now find that they're truncated at 1000 Unicode characters rather than 255 bytes. Please test it out! If you find errors, or places in core features (not comments in extensions such as SecurePoll, AbuseFilter, CheckUser, or Flow) where *new* comments are still being truncated to 255 bytes, or places where comments aren't showing up at all, please let me know. You can reply to this message or post a task in Phabricator and add me as a subscriber. If things go well, we'll look at rolling this out to production wikis once the schema changes to the production databases are complete. See https://phabricator.wikimedia.org/T174569 to follow progress there. If anyone is interested in submitting patches for the web UI to reflect the changed length limits, please do. I'll try to review them if you add me as a reviewer. [1]: https://deployment.wikimedia.beta.wmflabs.org/wiki/Main_Page -- Brad Jorsch (Anomie) Senior Software Engineer Wikimedia Foundation

2 1

Thoughts for handy features
by John Elliot V 18 Nov '17

18 Nov '17

Hey there. I love MediaWiki and have been using it everywhere for years. Recently I've been doing some rather major documentation and I realised there were three features which would be really handy for me (if these or similar already exist I would love to know!): 1. For links to articles in sections on the same page it would be really handy if we had syntax like: [[#unity|]] which would auto-complete to [[#unity|unity]] for you. 2. For duplicated content it would be handy if you could define a bunch of "variables" down the bottom of a page and then reference them from elsewhere. I am aware of templates, but those are overkill and difficult to maintain per my use case (my use case is documenting the "purpose" of a computer, I duplicate this in various places, but don't want to maintain templates for that). 3. It would be cool if for any given wiki page an "estimated reading time" could be provided. Along with maybe a word count, character count, etc. Since I'm here, quick thanks to the MediaWiki community for creating such wonderful wiki software! Regards, John Elliot V -- E: jj5(a)jj5.net P: +61 4 3505 7839 W: https://www.jj5.net/

6 5

2017 Community Wishlist Survey starts today!
by Danny Horn 18 Nov '17

18 Nov '17

Hi everyone, The third annual Community Wishlist Survey starts today, and you're invited to post proposals for projects that you'd like WMF's Community Tech team to work on: https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey The Community Tech team builds features and makes changes that active Wikimedia contributors want, and the Wishlist Survey sets the team's agenda for the year. The Wishlist Survey starts with a two-week proposal period, when contributors from all Wikimedia projects are invited to post, discuss and improve propsals. After that, there's a two-week voting period, when everyone can post support-votes on the proposals that they think are worthwhile. We end up with a ranked list of wishes, measured by the participants' enthusiasm for each idea. Community Tech is responsible for addressing the top 10 wishes on the list, as well as some wishes from smaller groups and projects that are doing important work, but don't have the numbers to get their proposal into the top 10. The Wishlist is also used by volunteer developers and other teams, who want to find projects to work on that the community really wants. So I hope that everybody comes and participates; it's an opportunity to set the agenda for a Wikimedia Foundation product team. We would also ask that you help us spread the word. Please do post on your wikis and tell others this is happening, and that if they don't feel comfortable writing in English, proposals are welcome in any language. I hope to see everyone there! Danny Horn WMF Community Tech

3 3

MediaWiki train for 1.31.0-wmf.8 is on hold due to a database failure
by Greg Grossmeier 17 Nov '17

17 Nov '17

Hello, There was a badly timed database failure today that has caused all deployments to be on hold until the situation is resolved. See: https://phabricator.wikimedia.org/T180714 "db1063 crashed" The MW Train blocker task, https://phabricator.wikimedia.org/T178635, will be updated when we move forward. Sorry for the inconvenience, Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

1 1

TechCom Radar, 2017-11-15
by Daniel Kinzler 17 Nov '17

17 Nov '17

Hello all! Here are the minutes from this week's meeting: * Marko Obrovac joined the Technical Committee. We are looking forward to working with him. Welcome Marko! * Last Call: RFC on imported user names <https://phabricator.wikimedia.org/T179832>. After the public discussion on November 15, it was agreed that this RFC should enter the last call period. If no pertinent concerns remain unaddressed by November 29, this RFC will be approved for implementation. * Next week’s RFC discussion: Bump PHP requirement to 7.0 in MW 1.31 <https://phabricator.wikimedia.org/T172165>. This is a reiteration of the RFC discussion from a few weeks ago, which ended with a last call for the proposal to go to PHP 5.6 for MW 1.31. The last call did not lead to approval, due to a suggestion made by Anomie, namely to go for a subset of PHP 7 that is supposed by hhvm in default mode, without the PHP7 flag set. As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday, at 2pm PST / 23:00 CET / 22:00 UTC. * Roan is experimenting with RemexHTML for tag stripping You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Security release: 1.29.2 / 1.28.3 / 1.27.4
by Sam Reed 15 Nov '17

15 Nov '17

I would like to announce the release of MediaWiki 1.29.2, 1.28.3 and 1.27.4! These releases fix nine security issues in core and one related issue in the vendor folder. Download links are given at the end of this email. Patches will be pushed to gerrit after this email is sent, and will land into the relevant branches as fast as our CI infrastructure allows. Git tags will follow soon after. All related tasks will be made public in phabricator too in the following few hours. Please note that this month is the End-Of-Life date for MediaWiki 1.28. This means that MediaWiki 1.28.3 will be the last security release for that version, barring any unforeseen issues. We would strongly encourage users of MediaWiki 1.28 to upgrade to MediaWiki 1.29, released in July 2017, or a yet newer version as soon as possible. MediaWiki 1.29 will be supported until July 2018. See <https://www.mediawiki.org/wiki/Version_lifecycle> for more information. This release also serves as a maintenance release for these branches. == Security fixes == * (T128209) Reflected File Download from api.php. Reported by Abdullah Hussam. (CVE-2017-8809) * (T165846) BotPasswords doesn't throttle login attempts. * (T134100) On private wikis, login form shouldn't distinguish between login failure due to bad username and bad password. (CVE-2017-8810) * (T178451) XSS when $wgShowExceptionDetails = false and browser sends non-standard url escaping. (CVE-2017-8808) * (T176247) It's possible to mangle HTML via raw message parameter expansion. (CVE-2017-8811) * (T125163) id attribute on headlines allow raw >. (CVE-2017-8812) * (T124404) language converter can be tricked into replacing text inside tags by adding a lot of junk after the rule definition. (CVE-2017-8814) * (T119158) Language converter: unsafe attribute injection via glossary rules (CVE-2017-8815) The following only affects 1.29: * (T180488) (T125177) "api.log contains passwords in plaintext" wasn't correctly fixed in all branches in the previous security release. (CVE-2017-0361) The following only affects 1.27 and 1.28: * (T180231) composer.json has require-dev versions of PHPUnit with known security issues. Reported by Tom Hutchison. (CVE-2017-9841) It is recommended to run `composer update --no-dev` after upgrading to MW 1.27.4 or 1.28.3 if you installed MediaWiki via git. If you are using the tarball, you are not affected, and you do not need to run this command. This will remove developer dependancies that production wikis do not require. If you require developer dependancies, run `composer update` which will update to a version of PHPUnit without known RCE. If you cannot run `composer update` for any reason, it is recommended that you delete the offending file as a minimum yourself using the following command: `rm -rf vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php` == Links to all mentioned tasks == https://phabricator.wikimedia.org/T128209 https://phabricator.wikimedia.org/T165846 https://phabricator.wikimedia.org/T134100 https://phabricator.wikimedia.org/T178451 https://phabricator.wikimedia.org/T176247 https://phabricator.wikimedia.org/T125163 https://phabricator.wikimedia.org/T180231 https://phabricator.wikimedia.org/T125163 https://phabricator.wikimedia.org/T124404 https://phabricator.wikimedia.org/T119158 https://phabricator.wikimedia.org/T180488 https://phabricator.wikimedia.org/T125177 == Release notes == Full release notes for 1.27.4: https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_27/RELEASE-NOTES… https://www.mediawiki.org/wiki/Release_notes/1.27 Full release notes for 1.28.3: https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_28/RELEASE-NOTES… https://www.mediawiki.org/wiki/Release_notes/1.28 Full release notes for 1.29.2: https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_29/RELEASE-NOTES… https://www.mediawiki.org/wiki/Release_notes/1.29 For information about how to upgrade, see <https://www.mediawiki.org/wiki/Manual:Upgrading> ********************************************************************** Download: https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.4.tar.gz Download without bundled extensions: https://releases.wikimedia.org/mediawiki/1.27/mediawiki-core-1.27.4.tar.gz Patch to previous version (1.27.3): https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.4.patch.gz GPG signatures: https://releases.wikimedia.org/mediawiki/1.27/mediawiki-core-1.27.4.tar.gz.… https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.4.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.4.patch.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html ********************************************************************** Download: https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.3.tar.gz Download without bundled extensions: https://releases.wikimedia.org/mediawiki/1.28/mediawiki-core-1.28.3.tar.gz Patch to previous version (1.28.2): https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.3.patch.gz GPG signatures: https://releases.wikimedia.org/mediawiki/1.28/mediawiki-core-1.28.3.tar.gz.… https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.3.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.28/mediawiki-1.28.3.patch.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html ********************************************************************** Download: https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.2.tar.gz Download without bundled extensions: https://releases.wikimedia.org/mediawiki/1.29/mediawiki-core-1.29.2.tar.gz Patch to previous version (1.29.1): https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.2.patch.gz GPG signatures: https://releases.wikimedia.org/mediawiki/1.29/mediawiki-core-1.29.2.tar.gz.… https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.2.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.29/mediawiki-1.29.2.patch.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html

5 4

TechCom Radar, 2017-11-08
by Daniel Kinzler 15 Nov '17

15 Nov '17

Hello all! Here are the minutes from this week's meeting: * Previous week’s RFC discussion: How to modify all preferences? <https://phabricator.wikimedia.org/T178449>. Many details seem unclear, so this RFC went back to “under discussion”. * This week’s discussion on IRC was about the RFC process itself. Log at <https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.20…> * Next week’s RFC discussion: Handling of Imported Usernames <https://phabricator.wikimedia.org/T179832>. As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday - however, this time, it will be two hours earlier! Please join us at 12 noon PST / 21:00 CET / 20:00 UTC. You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

2 2

[MediaWiki-announce] Security release: 1.29.2 / 1.28.3 / 1.27.4
by Sam Reed 15 Nov '17

15 Nov '17

1 0

Discovery Weekly Update for the week starting 2017-10-30
by Chris Koerner 14 Nov '17

14 Nov '17

Hello there, This is the Discovery weekly update for the week starting 2017-10-30 _and_ 2017-11-06 - so the last two weeks. == Highlights == * Blog post about searching using dashes and exclamation points when they're not part of the expected search syntax (by Trey Jones): ''"So -happy to meet you: Advanced searching techniques on Wikimedia sites"''. [0] == Discussions== === Search === * Trey wrote a blog post about advanced searching techniques, particularly dealing with dashes and exclamation points when they are *not* search syntax. [0] * Trey finished an analysis of enabling mapping hiragana and katakana for English- and Japanese-language wikis. There was less enthusiasm for the mapping and more problems on the Japanese side, so it is not going to be deployed there at this time. However, we're looking to enable it for other languages. [1] * David finished up refactoring the Searcher class and the SearchContext to allow easy code reuse [2] * Eric fixed an issue with a searchmatch span that was altering searches [3] * Eric and David finished up the work needed for evaluating the training speed and accuracy for 1M and 30M sample training sets and updated the resource usage docs [4] * Stas enabled disambiguation page demoting in Wikidata search and fixed several regressions in prefix search [5] [6] [7] [8]. === Analysis === * Chelsy finished up the analysis of the A/B test to test relaxing the retrieval query filter and the report has been published [9] [10] * After the quarterly metrics meeting at the end of October, there was a follow-up question that Chelsy answered [11] * Mikhail and Gehel finished up the creation of a Puppet profile/role for doing R-based heavy stats/ML on Wikimedia Cloud [12] === Portal === * The Wikipedia.org portal page was updated with new translations and new stats on October 30, 2017 [13] [14] === WDQS === * Stas fixed bug in RDF formatting of coordinates [15] * Stas implemented ordinal variable in WDQS calls to MWAPI [16] [0] https://blog.wikimedia.org/2017/11/06/searching-techniques/ [1] https://phabricator.wikimedia.org/T176197 [2] https://phabricator.wikimedia.org/T178906 [3] https://phabricator.wikimedia.org/T178522 [4] https://phabricator.wikimedia.org/T170009 [5] https://phabricator.wikimedia.org/T148411 [6] https://phabricator.wikimedia.org/T179061 [7] https://phabricator.wikimedia.org/T179045 [8] https://phabricator.wikimedia.org/T179130 [9] https://phabricator.wikimedia.org/T177957 [10] https://analytics.wikimedia.org/datasets/discovery/reports/AB_test_to_test_… [11] https://phabricator.wikimedia.org/T179449 [12] https://phabricator.wikimedia.org/T178096 [13] https://phabricator.wikimedia.org/T142582 [14] https://phabricator.wikimedia.org/T128546 [15] https://phabricator.wikimedia.org/T179228 [16] https://phabricator.wikimedia.org/T177275 --- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 14 Nov '17

14 Nov '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 4-5 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_ Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting are: @addshore & @CFisch_WMDE. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Fwd: Security pre-release announcement: 1.29.2 / 1.28.3 / 1.27.4
by Sam Reed 14 Nov '17

14 Nov '17

Hi all, Tomorrow we will be issuing a security release to all supported branches of MediaWiki. The new releases will be: 1.29.2 1.28.3 1.27.4 This will resolve 9 issues in MediaWiki core. Fixes will be available in these respective release branches, the currently unreleased 1.30.0 and also master. Tarballs will be available for the above mentioned point releases as well. 1.28 was previously announced as end of life, but 1.28.3 will be the final security and maintenance release barring any unforeseen issues. This security release includes fixes for MediaWiki core. --- Sam Reed

1 0

New print to pdf feature for mobile web readers
by Olga Vasileva 13 Nov '17

13 Nov '17

The Readers web team will be deploying a new feature this week to make it easier to download PDF versions of articles on the mobile website. [0] Providing better offline functionality was one of the highlighted areas of focus based on the generative research done by the New Readers team in Mexico, Nigeria, and India. Over the past year, the New Readers and Readers Web teams created a prototype for mobile PDFs which was evaluated by user research and community feedback [1]. As the prototype evaluation received positive feedback and results, we went forward with development. [2] For the initial deployment, the feature will be available to Google Chrome browsers on Android with support for other mobile browsers to come in the future. For Chrome, the feature will use the native Android print functionality where users can choose to download a webpage as a PDF. Mobile print styles will be used for these PDFs to ensure optimal readability for smaller screens. [3] The feature will be available starting Wednesday, Nov 15. For more information, see the project page on MediaWiki.org. [0] [0] https://www.mediawiki.org/wiki/Reading/Web/Projects/Mobile_PDFs [1] https://meta.wikimedia.org/wiki/New_Readers/Offline [2] https://meta.wikimedia.org/wiki/New_Readers/Offline#Concept_testing_for_mob… [3] https://www.mediawiki.org/wiki/Reading/Web/Projects/Print_Styles#Mobile_Pri… -- Olga Vasileva // Product Manager // Reading Web Team https://wikimediafoundation.org/

1 0

2017-11-08 Scrum of Scrums meeting notes
by Grace Gellerman 09 Nov '17

09 Nov '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-11-08 = 2017-11-08 = == Callouts == * No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week). * RelEng: Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained). [https://phabricator.wikimedia.org/T139740 task] * Tidy replacement: if you are an editor on some wiki, please get the word out to your wikis to fix the high-priority linter issues. We want to get Tidy removed from the cluster sooner than later.A bunch of wikis (including mediawiki) no longer have Tidy, and a few 100 small wikis can get Tidy replaced based on stats about linter issues ( https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/Linter/Stats/2017-11-… ). But would be good to get some largish wikis moved over so we can identify any lingering problems sooner than later. == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** 5.7.1 in the app store today (minor bug fixes) https://phabricator.wikimedia.org/project/view/3047/ ** Continuing work on 5.8 - https://phabricator.wikimedia.org/project/view/2913/ (faster article loads, synced reading lists) ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Released version 2.6.206 to production. ** Continuing to test / integrate with Reading List service. ==== Reading Web ==== * Blocked by: ** Ops&Release engineering - How to use Chromium in Puppeteer [ https://phabricator.wikimedia.org/T178570]+ https://phabricator.wikimedia.org/T178189 *** Currently investigating on path forward in https://phabricator.wikimedia.org/T180037. * Blocking: * Updates: ** Working on the Chromium-based PDF renderer service. *** Starting performance testing within two weeks. ** Researching behaviour around the "Download to PDF" button [ https://phabricator.wikimedia.org/T177969, https://phabricator.wikimedia.org/T177969#3742056] ** Working product and technical documentation for Marvin ==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ** Continuing work on MCS summary, media (gallery) endpoints. ** Continuing with Reading Lists ==== Multimedia ==== * Blocked by: * Blocking: * Updates: ==== Discovery ==== * Blocked by: * Blocking: * Updates: ** continuing work on portal automation. going well. ===== Maps ===== * Blocked by: N/A * Blocking: N/A * Updates: None === Contributors === ==== Editing ==== * Blocked by: * Blocking: * Updates: ==== Parsing ==== * section + figure-inline + html5-ids changes aren't going out this week after all -- reviews not yet finalized + we need thorough testing in beta cluster. We are still hoping to get it out next week. * Tidy replacement: if you are an editor on some wiki, please get the word out to your wikis to fix the high-priority linter issues. We want to get Tidy removed from the cluster sooner than later. A bunch of wikis (including mediawiki) no longer have Tidy, and a few 100 small wikis can get Tidy replaced based on stats about linter issues ==== Global Collaboration ==== * Blocked by: nobody * Blocking: ops maybe? There's the Flow dumps issue but not sure how much of a blocker that is * Updates: ** Going to roll out RCFilters to remaining (FlaggedRevs) wikis on Nov 15 ** Early work on using wikitext instead of HTML for storing Flow content, need something like pcache except 1) it's Parsoid HTML not PHP HTML 2) Flow comments aren't pages ==== UI Standardization ==== ** OOUI v0.24.1 & v0.24.2 released (missed last week's Scrum of Scrums), https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…; among changes: *** UI design **** WikimediaUI theme with 3 changes: SVG markup (see below), MenuSectionOptionWidget clearer on long lists and ComboBoxInputWidget providing stronger user feedback on open * Ongoing: ** OOUI & relying products: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432 ** Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867 === Community Tech === * Not blocking * Not blocked * Our annual survey is underway: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey * Fixing issues uncovered by Unicode sections rollout to Russian projects == Technology == === Analytics === * New kafka Jumbo cluster is up, for now mirror-ing topics from analytics cluster. * New DB hosts for eventlogging back in place, purging data out of compliance at a faster rate due to better hardware. * We have spark2 in cluster! * Still troubleshooting metric difference wikistats 1.0 on wikistats 2.0. Take a look edit metrics available now (some of them have a 5% difference, so not final yet): https://stats.wikimedia.org/v2/#/am.wikipedia.org * Work on refining eventlogging data in hadoop continues so hadoop can be a easy to query backend for eventlogging data === Cloud Services === * Blocked by: * Blocking: * Updates: === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Deploying new English thank you letter ** Payments form validation improvements ( https://phabricator.wikimedia.org/T176501) ** Better API error handling for backup card processor ( https://phabricator.wikimedia.org/T177893) ** More stats reporting to prometheus ** CiviCRM UI improvements (https://phabricator.wikimedia.org/T142549, https://phabricator.wikimedia.org/T178003, https://phabricator.wikimedia.org/T177873) ** CentralNotice: allow delaying impression reporting ( https://phabricator.wikimedia.org/T176334) === MediaWiki Platform === * Blocked by: * Blocking: * Updates: === Performance === * Blocked by: * Blocking: * Updates: === Release Engineering === * Blocking ** ??? * Blocked ** Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained). *** See: https://phabricator.wikimedia.org/T139740 *** Notably: Global Collaboration Team, Fundraising Tech, Wikibase, and Mulitmedia * Updates ** No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week). ** [TechDebt program] First pass of the service levels for component ownership was shared with the Code Health Group last week; feedback on-going. ** [TechDebt program] The next blog post should be posted Real Soon Now™ (done on our side). ** [SSD Program] Working on getting the mathoid tests running on submit. ** [SSD Program] A new release of Blubber is on the horizon. *** [SSD Program] Aside: what it is: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Deploymen… *** [SSD Program] Bubber: https://phabricator.wikimedia.org/source/blubber/ ** [scap tech debt] Working to support both pre git-2.11 and post git 2.11 hosts (namely Trusty and Jessie (with backports)/Stretch, respectively) to let us use newer functionality (notably `--jobs`). ** [CI] Most tox jobs are moved to Docker containers ** [CI] Investigating why many docker containers are left behind after a SIGTERM === Research === * Blocked by: * Blocking: * Updates: === Scoring Platform === * Blocked by: * Blocking: * Updates: **frwiki -- ongoing issue. https://phabricator.wikimedia.org/T179711 (should be fixed by SOS) **Draft quality data is coming to English Wikipedia (ping Commtech and Global Collab) ***You can use this to help patrollers sort new pages by (Spam, Attach, Vandalism/Hoax, or OK). OK == not immediately concerning but not necessarily good. ***[Reponse from Roan: we don't plan to use draftquality directly at this point] **Wikilabels getting some improvements. ***Links to on-wiki documentation for each campaign ("info" link) ***Statistics about labelers will have usernames **Amir is working on Icelandic language support. === Search Platform === * Blocked by: * Blocking: * Updates: ** Added hiragana<->katakana translations for English wiki (not for Japanese one) https://phabricator.wikimedia.org/T176197 ** Implemented demoting disambig pages for Wikidata prefix search https://phabricator.wikimedia.org/T148411 ** Improved reusability of Searcher code https://phabricator.wikimedia.org/T178906 ** Improving completion suggester https://phabricator.wikimedia.org/T178474 ** Working on porting Selenium tests from Ruby to JS ** Working on upgrade to Elastic 5.5 ** Working on indexing Wikidata descriptions and adding them to fulltext search === Security === * Blocked by: None * Blocking: Everyone waiting for security reviews * Updates: ** Reviews: *** BeautifulSoup (done, comments will be added shortly) *** vue.js (done, same) *** mediawiki-services-chromium-render ** Preparing for MW security release === Services === * Blocked by: none * Blocking: none * Updates: ** Parsoid storage switched to new storage model for all but wikipedias ** Definitions storage switched to new model ** Planning on switching RecordLintJob to kafka-based JobQueue === Technical Operations === * Blocked by: ** Flow isAllowed gets actual revision text before it is needed https://phabricator.wikimedia.org/T172025 * Blocking: * Updates: ** labsdb1001 storage crashed; Manuel got the dbs back up, but we are leaving in read-only - https://phabricator.wikimedia.org/T179464 ** Turning up s8 databases for wikidata https://phabricator.wikimedia.org/T177208 ** wikidiff2 1.5.1 fully rolled out, enabled for group0 wikis, additional will be enabled by Adam Shoreland over the next weeks == Wikidata == * Blocked by: * Blocking: * Updates: ** getting rid of the Wikidata build (to deploy Wikibase etc. as normal extensions): https://phabricator.wikimedia.org/T173818 *** changed how JS libraries are included (as submodules, instead of via composer) * work on Lexeme (persistence) * work on multi-content revisions * Coordinates in RDF exports are going to use xsd:double: https://phabricator.wikimedia.org/T179228 == German Technical Wishlist == * Blocked by: * Blocking: * Updates:

1 0

Simple overview image about how MW loads resources in clients
by Joaquin Oltra Hernandez 09 Nov '17

09 Nov '17

Hi, We were having a session where we talked about resource loading, code entry points for the front-end, and how things work on MediaWiki, and we came up with a small pic to explain the lifecycle for people newer to MediaWiki. Maybe it could help some people get a better grasp about where files are coming from and what why the load.php urls are as they are. Please, forgive any missing details, and if there is something very wrong I'd love to correct it, please let me know. Also to clarify, "Magic" is used as "Dynamic, runtime based, dependent on the state of your code/client cache/server state & extensions" to shorten things and in a humorous key. Links: - Phab: https://phabricator.wikimedia.org/M232 - Imgur: https://i.imgur.com/DYLqtQf.png

9 12

TechCom discussion in 90 minutes: Tweaking the RFC process
by Daniel Kinzler 08 Nov '17

08 Nov '17

Hi all! As mentioned in the TechCom Radar mail earlier this week, today's TechCom IRC meeting will be about the RFC process itself. There is no concrete proposal to discuss (yet), this is intended to be a session for gathering questions, comments and suggestions. The meeting will start at 22:00 UTC (2pm PST, 23:00 CET) on #wikimedia-office. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Try out the commit message validator
by Kunal Mehta 08 Nov '17

08 Nov '17

Hi! tl;dr <https://www.mediawiki.org/wiki/commit-message-validator> Commit messages are pretty important - they make it easy to figure out what a change does without reading the entire diff. And because of that, we have some guidelines[1] on how to format your commit message in the standard way. But sometimes people aren't familiar with the guidelines, or more likely we make a typo somewhere. Here's where the commit-message-validator[2] comes in handy! $ git commit <type in commit message and save> commit-message-validator v0.5.1 The following errors were found: Line 1: First line should be <=80 characters Line 3: Bug: value must be a single phabricator task ID Please review <https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines> and update your commit message accordingly Or if you did a good job: commit-message-validator v0.5.1 Commit message is formatted properly! Keep up the good work! Installing it as a git hook is the recommended usage, and instructions on the wiki page[2] explain how to do so, even for people who aren't familiar with Python. The tool was written by Bryan Davis, Fabian Neundorf, and myself. Bugs/feature requests can be filed in the Phabricator project. [1] https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines [2] https://www.mediawiki.org/wiki/Commit-message-validator -- Legoktm

9 8

Readers monthly update for October 2017
by Chris Koerner 08 Nov '17

08 Nov '17

Hello, Here is the monthly update for October from the Readers department at the Wikimedia Foundation. **Reminder**: You can now subscribe to receive theses updates as on-wiki notifications or opt-in email! Handy if you subscribe to the digest of his mailing list or don't visit the archive frequently. Subscribe on MediaWiki.org: https://www.mediawiki.org/wiki/Newsletter:Readers_Monthly Thanks for reading, as always, feedback and questions are welcome. === Apps=== * The iOS team released version 5.7.0. This version includes an updated share-a-fact, improved on-boarding for new users and lots of fixes for iPhone X. [0] * The iOS app was named an "Editors Choice" by Apple. This is the highest recognition Apple gives apps, and in their decision they particularly called out the improved quality of the app, and focus on accessibility. * The Android app went to beta with their latest update. This version adds app shortcuts, image dimming, theming improvements for Light and Dark mode, bug fixes for Simplified and Traditional Chinese and many other refinements. [1] * The apps teams participated in a 3 day "Offsite" in Philadelphia. Read about it on MediaWiki.org. [2] === Web === * Updated Print styles arelive [3] * Regrettably, we turned off OCG. Fortunately, this unblocked Ops's decommission of the service and servers. * We're working hard on a replacement service for the Electron-based PDF rendering service. [4] Follow our progress on Phabricator. [5] === Reading Infrastructure === * the API backend for Reading Lists is in beta [6] [7] === New Readers === * New Reader updates can be found at Meta. [8] === Multimedia === * Support for uploading and viewing 3D models is coming soon to Wikimedia Commons. The feature will support the .STL file format. [9] === Discovery === The Discovery team provides weekly updates. Here are their updates for the month. * 2017-10-02 [10] * 2017-10-09 [11] * 2017-10-16 [12] * 2017-10-23 [13] == Other Noteworthy Stuff == * Sam Smith: I'm currently investigating adding up/downstream bandwidth configuration to my micro-device-lab side project. [14] [0] https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=32471523… [1] https://play.google.com/store/apps/details?id=org.wikipedia.beta [2] https://www.mediawiki.org/wiki/Wikimedia_Apps/Philadelphia_offsite/Offsite_… [3] https://twitter.com/jdlrobson/status/918264297343422465 [4] https://github.com/wikimedia/mediawiki-services-electron-render [5] https://phabricator.wikimedia.org/T176627 [6] https://www.mediawiki.org/wiki/Reading/Reading_Lists [7] http://readinglists.wmflabs.org/wiki/VagrantRoleReadingLists [8] https://meta.wikimedia.org/wiki/New_Readers/Updates [9] https://www.mediawiki.org/wiki/Help:Extension:3d [10] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-10-02 [11] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-10-09 [12] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-10-16 [13] https://www.mediawiki.org/wiki/Discovery/Status_updates/2017-10-23 [14] https://github.com/phuedx/micro-device-lab --- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Reading/Status_updates Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

3D model support coming soon to Commons
by Chris Koerner 07 Nov '17

07 Nov '17

Hello! Just a quick note to let you all know that support for 3D models is coming to Commons! [0] The plan is to have the feature enabled by the end of November. There are a few technical items left to address. The feature will support the STL file format, one of the most common file formats for 3D models. [1] You can see an example of how a 3D model will appear on the test wiki. [2] (.gif) [3] This feature has been discussed in the Wikimedia community for a long time. Most recently the feature was #11 on the 2015 Community Tech Wishlist and was previously discussed on Commons in April as development progressed. [4] [5] There will be a blog post on the Wikimedia Blog once the feature is enabled. You can read more about the technical details of the feature and the help documentation on MediaWiki.org. [6] [7] [0] https://commons.wikimedia.org/wiki/Commons:Village_pump#3D_model_support_co… [1] https://wikipedia.org/wiki/Special:MyLanguage/STL_(file_format) [2] https://test.wikipedia.org/wiki/File:Programmatically_created_crystal.stl [3] https://commons.wikimedia.org/wiki/File:Demo_of_3D_model_in_MediaViewer.gif [4] https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Commons#3D_m… [5] https://commons.wikimedia.org/wiki/Commons:Village_pump/Archive/2017/05#Pre… [6] https://www.mediawiki.org/wiki/Extension:3D [7] https://www.mediawiki.org/wiki/Help:Extension:3d Yours, Chris Koerner Community Liaison Wikimedia Foundation

1 0

Important news about the November dumps run!
by Ariel Glenn WMF 07 Nov '17

07 Nov '17

As was previously announced on the xmldatadumps-l list, the sql/xml dumps generated twice a month will be written to an internal server, starting with the November run. This is in part to reduce load on the web/rsync/nfs server which has been doing this work also until now. We want separation of roles for some other reasons too. Because I want to get this right, and there are a lot of moving parts, and I don't want to rsync all the prefetch data over to these boxes again next month after cancelling the move: ******** If needed, the November full run will be delayed for a few days. If the November full run takes too long, the partial run, usually starting on the 20th of the month, will not take place. ********* Additionally, as described in an earlier email on the xmldatadumps-l list: ********* files will show up on the web server/rsync server with a substantial delay. Initially this may be a day or more. This includes index.html and other status files. ********* You can keep track of developments here: https://phabricator.wikimedia.org/T178893 If you know folks not on the lists in the recipients field for this email, please forward it to them and suggest that they subscribe to this list. Thanks, Ariel

2 4

TechCom Radar, 2017-11-01
by Daniel Kinzler 07 Nov '17

07 Nov '17

Hello all! Here are the minutes from this week's meeting: * TechCom would like to thank Kevin Smith for his help with facilitating and note taking. We hope to see you around, Kevin! * Previous IRC meeting: How to modify all preferences? <https://phabricator.wikimedia.org/T178449>. Outcome: Introducing a PreferenceFactory service seems like a good idea, but there are several issues to resolve with the implementation as proposed. RFC remains under discussion. * Last Call canceled: Bump PHP requirement to 5.6 in 1.31 <https://phabricator.wikimedia.org/T178538>. The RFC goes back to “under discussion” due to an issue raised by Anomie <https://phabricator.wikimedia.org/T172165#3699606>. * Discussion on wikitech-l: Proposal regarding the handling of imported usernames <https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/089047.html>. * Next IRC meeting: Gathering input on the RFC process. We want to hear from you what you like or dislike about the RFC process. If you have criticism or suggestions, please let us know. Note that we have diverged quite a bit from the process as documented at Requests_for_comment/Process. This session is part of a drive to overhaul our processes and improve documentation. As always, the discussion will take place in the IRC channel #wikimedia-office on Wednesday 21:00 UTC (2pm PDT, 23:00 CEST). * Daniel has published a rant about how unclear ownership makes it hard to address technical debt <https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/Avoiding_the_Tech…>. This topic was brought up during the discussion about FauxRequest <https://phabricator.wikimedia.org/T169266>. You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 1

Tomorrow: Weekly Technical Advice IRC Meeting
by Michael Schönitzer 07 Nov '17

07 Nov '17

Sorry for cross-posting! Reminder: Technical Advice IRC meeting again **tomorrow 4-5 pm UTC** on #wikimedia-tech. The Technical Advice IRC meeting is open for all volunteer developers, topics and questions. This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting This meeting is an offer by WMDE’s tech team. Hosts of tomorrows meeting are: @addshore & @CFisch_WMDE. Hope to see you there! Michi (for WMDE’s tech team) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

Upcoming holidays/events and their impact on the deployment schedule
by Greg Grossmeier 06 Nov '17

06 Nov '17

Hello all, It's that time of year again where the density of holidays increases and at the same time our plans for fundraising also increase. Per our usual practice we will be not doing deployments at various points in the next few months. Here's the full outline: Reminder for all on we did last year: * No MW train the week of Thanksgiving (but SWAT deploys were open for high-priority things). * No deploys (at all) the last two weeks of December. People were happy with this. * The first week of January was normal (minus Monday being Jan 2nd, our observed New Year's Day holiday) deployment wise. * The second week was weird due to Dev Summit/All Hands: No MediaWiki train, only SWATs as needed on Mon/Tues/Wed. No SWATs/deploys during All Hands. I imagine we'll do similarly. In that case (looking at the calendar....) * No MW train the week of Thanksgiving (Nov 20th), SWATs open for high priority things * No deploys weeks of Dec 18th and 25th (last two weeks) * Normal week week of Jan 1st (minus no deploys that Monday) * The Dev Summit and WMF All Hands is the week of January 22nd, so that will be a "No Train but SWATs OK on Mon/Tues/Wed" week. * The following week (week of January 29th) the Release Engineering team will be on an offsite, so a week of "No Train, but SWATs and service deploys OK". This is now on-wiki at: https://wikitech.wikimedia.org/wiki/Deployments#Upcoming Best, Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Release Team Manager A18D 1138 8E47 FAC8 1C7D |

1 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l November 2017