Wikitech-l June 2019

wikitech-l@lists.wikimedia.org

54 participants
57 discussions

Quod facit beatus hic sabbati? / What's making you happy this week? (Week of 2 June 2019)

by Pine W

As usual, I am grateful for the volunteer contributors to the most recent issue <https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/Archives/2019-05…> of The Signpost. I was happy to see that Wikimedia Canada <https://ca.wikimedia.org/wiki/Main_Page> and Library and Archives Canada <https://en.wikipedia.org/wiki/Library_and_Archives_Canada> announced a collaboration <https://lists.wikimedia.org/pipermail/glam/2019-June/001596.html>. Thanks to Brooke Storm (WMF Cloud Services) and Bryan Davis (WMF Technical Engagement), I found an amusing xkcd depiction <https://xkcd.com/1319/> of successful task automation and unsuccessful task automation. <https://xkcd.com/1319/> Here are some recent stories that were published by WMF: * Four open design methods we used to improve Wikipedia’s iOS app <https://wikimediafoundation.org/2019/05/28/four-open-design-methods-we-used…> * With Wikipedia in the classroom, a former student has become the teacher <https://wikimediafoundation.org/2019/05/22/with-wikipedia-in-the-classroom-…> * Wikimedia Foundation announces tenth transparency report <https://wikimediafoundation.org/2019/05/20/wikimedia-foundation-announces-t…> * Wikimedia Foundation urges Chinese authorities to lift block of Wikipedia in China <https://wikimediafoundation.org/2019/05/17/wikimedia-foundation-urges-chine…> * MediaWiki is the software that underpins Wikipedia. This conference shows all the other ways it can be used. <https://wikimediafoundation.org/2019/05/01/mediawiki-is-the-software-that-u…> * Wikimedia Argentina and the National University of La Plata partner to promote free knowledge <https://wikimediafoundation.org/2019/04/30/wikimedia-argentina-and-the-nati…> The Word of the Day for English Wiktionary on June 2nd was "rubicon". I was familiar with a fictional ship named <https://en.wikipedia.org/wiki/Runabout_(Star_Trek)>Rubicon <https://en.wikipedia.org/wiki/Runabout_(Star_Trek)> and I guessed that the name referred to a river, which it does <https://en.wikipedia.org/wiki/Rubicon>, but I did not know that more than one river <https://en.wikipedia.org/wiki/Rubicon_River_(disambiguation)> is named Rubicon, and I did not know about many other uses for the name <https://en.wikipedia.org/wiki/Rubicon_(disambiguation)>. Thanks to Wiktionary, I learned that the word "rubicon" (with a lowercase "r") has its own meanings <https://en.wiktionary.org/wiki/Wiktionary:Word_of_the_day/June_2> and an interesting etymology <https://en.wiktionary.org/wiki/rubicon> that refers to the the Italian river as a location of a notable event in history <https://en.wikipedia.org/wiki/Crossing_the_Rubicon>, and possibly also refers the red color of the river <https://en.wiktionary.org/wiki/Reconstruction:Proto-Indo-European/h%E2%82%8…> . What's making you happy this week? You are welcome to comment in any language. Pine ( https://meta.wikimedia.org/wiki/User:Pine )

4 years, 10 months

Upcoming Search Platform Office Hours—June 5th

by Trey Jones

The Search Platform Team <https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> usually holds office hours the first Wednesday of each month. Come talk to us about anything related to Wikimedia search! Feel free to add your items to the Etherpad Agenda for the next meeting. Details for our next meeting: Date: Wednesday, June 5th, 2019 Time: 15:00-16:00 GMT / 08:00-9:00 PDT / 11:00-12:00 EDT / 17:00-18:00 CEST Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link: https://meet.google.com/vyc-jvgq-dww *N.B.:* Google Meet System Requirements <https://support.google.com/meet/answer/7317473> Hope to talk to you in a week! —Trey Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation UTC-4 / EDT

4 years, 10 months

How large is Wikipedia?

by Jaime Crespo

Hi, I cannot actually answer this question (it is not easy), but I sometimes get this kind of question related to our main relational database storage (MariaDB). I am preparing some slides for a presentation, and took some numbers and wanted to share those with you (as of June 2019): * There is approximately 550 TB of used data in the MariaDB-related servers along the Wikimedia infrastructure (mostly compressed in some way- InnoDB, gzip, etc.) * If we do not account for redundancy, 60TB of data is unique (average of 9x redundancy, which seems about right) ** Of that, 24TB is for insert-only highly-compressed content (External Storage) ** The rest is metadata, local content, misc services, disc cache, analytics, cloud dbs, and backups. Please note this doesn't have into account storage in other mediums or technologies (search, maps, analytics, REST, file storage, etc.). Also content compression be very efficient so uncompressed data can be much larger. We are in fact aiming at reducing even more the storage footprint over the next months. If someone is interested on seeing size evolution, you can get the latest up to date metrics on Grafana: https://grafana.wikimedia.org/d/000000607/cluster-overview?orgId=1&var-data… -- Jaime Crespo <http://wikimedia.org>

4 years, 10 months

FYI: Continuous Integration now running node 10 for MW, extensions, and skins

by James Forrester

Hey all, A quick heads-up: the continuous integration tests for MediaWiki core, MediaWiki extensions, and MediaWiki skins are now all using node 10, replacing node 6, which is end-of-life. CI jobs were replaced by new ones which should run faster, without any disruption for developers' work. For a brief period, selenium tests were disabled; they've now been re-enabled for all repos.[0] There are still a number of repos running node6 CI, which will need work to convert over in future.[1] This work also unblocks a number of issues, including: * committing package-lock, making for faster and more secure development[2], * using the newest version of stylelint (e.g. [3]), and * upgrading browser tests to webdriverio 5+ [4]. Antoine and I think we've checked for issues, but if you run into any, please report them on Phabricator.[0] [0] – https://phabricator.wikimedia.org/T222406 [1] – https://phabricator.wikimedia.org/T211784 [2] – https://phabricator.wikimedia.org/T179229 [3] – https://gerrit.wikimedia.org/r/c/510822 [4] – https://phabricator.wikimedia.org/T213268 J. -- *James D. Forrester* (he/him <http://pronoun.is/he> or they/themself <http://pronoun.is/they/.../themself>) Wikimedia Foundation <https://wikimediafoundation.org/>

4 years, 10 months

VE submodule path update

by James Montalvo

Could this change [1] be backported to VE release branches (REL1_32, REL1_31, etc)? Without it initializing submodules fails due to the gerrit path change. This may affect other extensions/projects as well, but this VE error has been reported to me by several people. Until recently it had only happened within within specific cloud hosting environments, but now I just got the failure within Travis CI [2] which worked fine yesterday. [1] https://github.com/wikimedia/mediawiki-extensions-VisualEditor/commit/6f6d8… [2] https://travis-ci.org/enterprisemediawiki/meza/jobs/540726316#L2646-L2652 Thanks, James

4 years, 10 months

FlaggedRevs icons

by Amir Sarabadani

Hey, This is not big enough to have its own announcement but since it'll go live this week and this hasn't made it to this week's Tech news, here's a quick heads up. Icons of FlaggedRevs extension (Pending changes) [0] were introduced 12 years ago and hasn't been changed since. I started to improve it and make it aligned with WikimediaUI design style guide [1] on it a while ago and it finally got merged. This will go live this week with the train and you can see an example of the new icons in [2] and you can see pictures of before and after (plus more information) in [3] Still so many things need to be done to improve it's design but for now, this is a starting point. [0] https://www.mediawiki.org/wiki/Extension:FlaggedRevs (or Pending changes: https://en.wikipedia.org/wiki/Wikipedia:Pending_changes) [1]: https://design.wikimedia.org/style-guide/visual-style_icons.html [2]: https://en.wikipedia.beta.wmflabs.org/wiki/PC1 [3]: https://phabricator.wikimedia.org/T155878 Best -- Amir (he/him)

4 years, 10 months

Separating MediaWiki core unit and integration tests — help welcome!

by Máté Szabó

Hey all, The idea of separating PHPUnit unit and integration/system tests in MediaWiki core has been around for some time[1][2][3]. Currently, the tests assume the presence of valid MediaWiki settings and a database connection, meaning one must install & configure MediaWiki and an RDBMS in their local development environment to be able to run the tests. The fact we use a non-standard entry point (phpunit.php) also makes these tests incompatible with existing tooling such as IDE integrations.[4] At the 2019 hackathon I worked with Amir Sarabadani, Michael Große and Kosta Harlan to perform some preliminary investigation into separating unit tests (that can be run without a database and MediaWiki configuration) into a separate PHPUnit configuration that could be run via the official phpunit binary. After some additional work, this has evolved into a patch that separates 5301 unit tests into a dedicated suite that can be executed via vendor/bin/phpunit in 15 seconds (on my machine!). By contrast, running the same 5301 tests via phpunit.php takes around 30 seconds.[5] Not using phpunit.php here also allows for integrating with e.g. Intellij/PHPStorm’s test execution and code coverage functionality—here’s a screenshot of the execution and coverage information of PathRouterTest via IntelliJ.[6] I feel that these benefits would be felt both by developers and CI maintainers—developers could iterate more rapidly by running the unit tests, and Jenkins would have to spend less time executing the test suite. I’d like to thank everyone who has supported this enterprise so far—Amir (Ladsgroup) for creating a script to identify the initial set of tests that do not rely on a database[7][8] and providing assistance throughout, Kosta Harlan for highlighting this old and forgotten issue and bringing it to the hackathon, Bartosz Dziewoński (MatmaRex) for providing a solution to scoping issues around MediaWiki core files,[9] Michael Große for demonstrating the feasibility of this approach in the Wikibase extension test suite,[10] and James Forrester for reviewing the changes and outlining possible next steps. However, the work is far from done yet! The patch is not yet merged, so any reviews, comments and suggestions would be very welcome there! And if it does get merged, we will have to think about how to bring this separation to extensions’ test suites as well as potentially port more core tests to the unit test suite. So if you have any ideas on how to improve this patch or would like to add to the next steps, don’t hesitate to leave a note :) [1] https://phabricator.wikimedia.org/T89432 [2] https://phabricator.wikimedia.org/T87781 [3] https://phabricator.wikimedia.org/T84948 [4] https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Running_the_unit_tes… [5] https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513106 [6] https://phabricator.wikimedia.org/F29316124 [7] https://phabricator.wikimedia.org/T87781#5194643 [8] https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/510226/ [9] https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513106#message-32b2a171a7… [10] https://gerrit.wikimedia.org/r/511035 Yours, --- Máté Szabó Software Engineer +36 30 947 5903 WIKIA sp. z o.o. z siedzibą w Poznaniu, ul. Abp. A. Baraniaka 6 Sąd Rejonowy Poznań – Nowe Miasto i Wilda w Poznaniu, VIII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000254365 NIP: 5252358778 Kapitał zakładowy: 50.000,00 złotych

4 years, 10 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l June 2019