Wikitech-l October 2018

wikitech-l@lists.wikimedia.org

71 participants
69 discussions

TechCom Radar 2018-10-17
by Kate Chapman 19 Oct '18

19 Oct '18

Hi All, Here are the minutes from this week's TechCom meeting: * No TechCom meeting on October 24 due to many from TechCom attending TechConf * No IRC meeting next week * Moved Last Call to end Wednesday 31 October 2pm PST (21:00 UTC, 22:00 CET) Spec for representing multiple content objects per revision (MCR) in XML dumps <https://phabricator.wikimedia.org/T199121> * On Last Call ending Wednesday 31 October 2pm PST (21:00 UTC, 22:00 CET) RfC: Release notes automation <https://phabricator.wikimedia.org/T200392> You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. If you prefer you can subscribe to our newsletter here: <https://www.mediawiki.org/wiki/Newsletter:TechCom_Radar> Thanks, Kate -- Kate Chapman TechCom Facilitator (Contractor)

1 0

URL-addressable Predicate Calculus
by Adam Sobieski 18 Oct '18

18 Oct '18

I would like to share, for discussion, some knowledge representation ideas with respect to a URL-addressable predicate calculus. In the following examples, we can use the prefix “mw” for “https://machine.wikipedia.org/” as per xmlns:mw="https://machine.wikipedia.org/" . mw:P1 → https://machine.wikipedia.org/P1 mw:P1(arg0, arg1, arg2) → https://machine.wikipedia.org/P1?A0=arg0&A1=arg1&A2=arg2 mw:P2 → https://machine.wikipedia.org/P2 mw:P2<t0, t1, t2> → https://machine.wikipedia.org/P2?T0=t0&T1=t1&T2=t2 mw:P2<t0, t1, t2>(arg0, arg1, arg2) → https://machine.wikipedia.org/P2?T0=t0&T1=t1&T2=t2&A0=arg0&A1=arg1&A2=arg2 Some points: 1. There is a mapping between each predicate calculus expression and a URL. 2. Navigating to mapped-to URLs results in processing on servers, e.g. PHP scripts, which generates outputs. 3. The outputs vary per the content types requested via HTTP request headers. 4. The outputs may also vary per the languages requested via HTTP request headers. 5. Navigating to https://machine.wikipedia.org/P1 generates a definition for a predicate. 6. Navigating to https://machine.wikipedia.org/P2?T0=t0&T1=t1&T2=t2 generates a definition for a predicate after assigning values to the parameters T0, T1, T2. That is, a definition of a predicate is generated by a script, e.g. a PHP script, which may vary its output based on the values for T0, T1, T2. 7. The possible values for T0, T1, T2, A0, A1, A2 may be drawn from the same set. T0, T1, T2 need not be constrained to be types from a type system. 8. The values for T0, T1, T2, A0, A1, A2, that is t0, t1, t2, arg0, arg1, arg2, could also each resolve to URLs. Best regards, Adam Sobieski http://www.phoster.com/contents/

3 3

2018-10-17 Scrum of Scrums meeting notes
by Grace Gellerman 17 Oct '18

17 Oct '18

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-10-17 =2018-10-17= == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar * Release Engineering ** Train blocked: (MediaWiki-General-or-Unknown) T207288 Text in the Sidebar does no longer show the message text, only the message name == Audiences == === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** Waiting for "Execute the schema change for Partial Blocks" to release Partial Blocks https://phabricator.wikimedia.org/T204006 ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** ==== Growth ==== * Blocked by: Editing on the details of instrumentaton for cross collaboration * Blocking: * Updates: **Copyvio logging is in progress on enwiki, so users of Special:NewPagesFeed?copyvio=1 can see drafts and articles which potentially have copyvio ==== Language ==== * Blocked by: * Blocking: * Updates: ** === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** 6.1 with wikidata description editing & revert notifications in beta ( https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/) ** 6.1.1 ( https://phabricator.wikimedia.org/tag/ios-app-v6.1.1-narwhal-on-a-magic-pum…) will be a bug fix & tech debt release ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** ==== Readers Web ==== * Blocked by: * Blocking: * Updates: ** Mobile website (MinervaNeue / MobileFrontend): *** Invest in the MobileFrontend & MinervaNeue frontend architecture https://www.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFro… **** Webpackify Skins and Overlay management T205593 **** Count production JavaScript client Errors in statsv T205582 **** Remove ESLint from Grunt tasks T206069 **** Fix QUnit appendChild bug with Browser.test.js T203818 **** Port PageGateway tests to node-qunit T206226 **** Use NPM for Hogan v2.0.0 T205128 **** Add code coverage testing and increase test coverage for some existing mobile.startup files T203818 *** Page issues https://www.mediawiki.org/wiki/Reading/Web/Projects/Mobile_Page_Issues **** A/B test will conclude soon T200792 **** Remove page issues A/B instrumentation T206178 *** Add share button T181195 *** Make the watchlist accessible to non-JavaScript browsers T196893 *** Maintenance and bug fixes T204835 *** Advanced mobile contributions https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions **** Design continuing special pages work in Minerva ** PDF rendering (Proton) https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality *** Remaining work tracked in T186748 *** Rewrite Queue to Promises T204055 *** Rollback Puppeteer to v1.5.0 ** SEO: *** Add Schema.org Article JSON-LD to main namespace pages T198946 (thanks for all the help cscott!) ** Management supporting Multimedia hiring processes ** Product continuing 3-5 year planning. ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** At offsite this week (not attending). ==== Multimedia ==== * Updates ** Hiring continues ** SDoC - ready to go to beta once James F comes back from his offsite ** SDoC - discussion on data modelling continues ** SDoC - usability study on multi-lingual captions ** SDoC - ... other stuff to do with allowing users to add/search on 'depicts' statements ==== Parsing ==== * Blocked by: None * Blocking: None * Updates: Nothing significant to report to SOS ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** == Technology == === Analytics === * Blocked by: someone to finish reviewing a very simple extension change: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Dashiki/+/464469/ will post on the Parsoid IRC channel, hopefully Cscott or Subbu will review and +2 (Shannon commenting here) (thank you kindly, I didn't want to bother people that are doing *important* work, this is just a piddly change) (Request submitted to parsoid IRC) * Blocking: * Updates: ** EventLogging - To - Druid automatic ingester now uses a whitelist instead of a blacklist. It's smoother and accepting more kinds of data, come talk to us if you're interested ** Various Wikistats 2 UI bug fixes, will focus on this all quarter to prepare it for a Beta release by the end of the year ** Progress on importing XML dumps, automatic job will be deployed soon and we'll start working on processing the content ** Working on the quality of Mediawiki History data in the Data Lake, addressing edge cases that we left for later ** Prototyping node Event Intake service for the Modern Event Platform ** EventLogging javascript client lost debug validation by default, and will soon no longer need Schema ResourceLoader modules, see migration info: https://phabricator.wikimedia.org/T205744 === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: **CRM tests still regularly failing due to full mysql partition on integration hosts. Possible fix noted by Eileen on https://phabricator.wikimedia.org/T205950 * Blocking: * Updates: **Email campaigns have been underway for a couple weeks now, seeing a decent amount of traffic **Fixing some payments-wiki card processor error handling **Refactoring CiviCRM imports to use drupal batch API **Still finishing up new code to import and aggregate banner and landing page stats to mysql on the FR cluster **Dusting off the internal donations dashboard for the Big English campaign === MediaWiki Core Platform === * Blocked by: * Blocking: * Updates: ** === Performance === * Blocked by: ** * Blocking: ** * Updates: ** === Release Engineering === * Blocked by: ** (MediaWiki-General-or-Unknown) T207288 Text in the Sidebar does no longer show the message text, only the message name * Blocking: ** Fundraising Tech: CRM tests still regularly failing due to full mysql partition on integration hosts. Possible fix noted by Eileen on https://phabricator.wikimedia.org/T205950 * Updates: ** Interviewing on-going for our Developer Productivity position: https://boards.greenhouse.io/wikimedia/jobs/1225258?gh_src=f15731e11 ** Train Health: *** Last week: no train, datacenter switchover T191071 1.32.0-wmf.25 deployment blockers *** This week: the last 1.32 release T191072 1.32.0-wmf.26 deployment blockers **** Still open/blocking: T207288 Text in the Sidebar does no longer show the message text, only the message name (MediaWiki-General-or-Unknown) **** Resolved: T207220 AFComputedVariable.php: Argument to getLinksFromDB() must be an instance of Article - the cause has been identified and reverted. *** Next week: 1.33 starts the next week - T191072 1.32.0-wmf.26 deployment blockers ** Log Health: *** T204871 Deployments of MediaWiki with scap cause a spam of "web request took longer than 60 seconds and timed out" ** Code Health: *** Metrics group meets weekly *** T207046 Code health metrics spike === Research === * Blocked by: * Blocking: * Updates: === Scoring Platform === * Blocked by: ** Need DBA review of a proposed link table and indexes for Extension:JADE, https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/JADE/+/456078/ * Blocking: * Updates: ** ORES cluster CPU usage has been slowly climbing, from 5% to almost 100%. This seems to be caused by a bug in mwparserfromhell, we're going to upgrade soon. No downtime so far, we have new alerts in place and will be sending out an update to wikitech-l soon. https://phabricator.wikimedia.org/T206654 ** We're switching some of our task tracking from Celery to Redis, which will help us with an upgrade to Celery 4, and reduces load on Redis. https://phabricator.wikimedia.org/T152012 === Search Platform === * Blocked by: * Blocking: * Updates: ** Finished evaluating Korean analyzers, positive result but waiting for ES 6.4 migration: https://phabricator.wikimedia.org/T178925 ** Standardize autocomplete preferences interface: https://phabricator.wikimedia.org/T205991 ** Lexemes are now supported in WDQS: https://lists.wikimedia.org/pipermail/wikidata//2018-October/012545.html ** Created auto-deployment testing server for WDQS: https://phabricator.wikimedia.org/T197187 ** Started work on “wrong keyboard layout” detection: https://phabricator.wikimedia.org/T138958 ** Working on calculating examination probabilities for completion queries: https://phabricator.wikimedia.org/T205348 ** Working on improving data and creating models from autocompletion click data: https://phabricator.wikimedia.org/T205111 ** Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654 ** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 === Security === * Blocked by: * Blocking: * Updates:https://phabricator.wikimedia.org/T200279 is moving forward to Beta ** === Services === * Blocked by: * Blocking: * Updates: ** === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Switchback completed successfully ** S8 (wikidata) eqiad missing data due to replication issues still being fully fixed ** Blubber 0.6.0 packaged and uploaded ** Apache puppet restructuring ongoing to support php7 == Wikidata == * Blocked by: * Blocking: * Updates: ** == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == Multi-Content Revisions == * Blocked by: * Blocking: * Updates: ** == SoS Meeting Bookkeeping == * Updates: **Agreement to change meeting start time to 35 mins past the hour

1 0

Collecting UI feedback for PolyGerrit - Gerrit
by Paladox 16 Oct '18

16 Oct '18

Hi, i am collecting feedback for Gerrit's New UI called PolyGerrit. It's possible to use PolyGerrit on gerrit.wikimedia.org since 2.14. The new UI has recently been made the default upstream. The Old UI is going away in the next release after 2.16. Upstream have given PolyGerrit another update that looks different to the one on gerrit.wikimedia.org. PolyGerrit now includes a dark ui. To switch to PolyGerrit either click the "New UI" button on the footer or put ?polygerrit=1 in the url. To switch back to GWTUI either click "Switch back to old ui" on the footer or put ?polygerrit=0 in the url. Non dark mode: Here's how it looks like: Dashboard: https://phabricator.wikimedia.org/F26296230 Change list: https://phabricator.wikimedia.org/F26296240 Change screen: https://phabricator.wikimedia.org/F26296242 https://phabricator.wikimedia.org/F26296257 Dark mode: https://phabricator.wikimedia.org/F26296282 And many other UI improvements across the app. You can play around the the new ui from the master branch that will become 2.16 here https://gerrit.git.wmflabs.org/r/ Please give feedback so upstream can make PolyGerrit even better! You can either file your reports at https://phabricator.wikimedia.org/project/view/330/ or reply to the email with your feedback. It has a dedicated team on the UI with a design researcher behind the scenes redesigning polygerrit constantly based on feedback.

10 21

[Python tox + Wikimedia CI] How to specify one testenv to be run for "check" Zuul pipeline
by Martin Urbanec 16 Oct '18

16 Oct '18

Hey all, I'd like to run lint testenv in wikimedia-cz/tracker for all users (whitelisted or non-whitelisted), to have jenkins vote "somehow". Lint is cheap, so I think there's no reason to have it restricted. I'm trying to figure out the way how to do it properly in Wikimedia setup. How can I get Wikimedia CI do this? Best, Martin Urbanec

1 0

Comment/summary storage changes
by Brad Jorsch (Anomie) 16 Oct '18

16 Oct '18

As you may have been aware, we've been working on changing how MediaWiki stores comments: instead of having them as fields in each revision (rev_comment), log entry (log_comment), and so on, we're storing the text in a central "comment" table and referring to them by ID from other tables (log_comment_id and so on). We've been writing to the new fields and tables since the end of February 2018, and have back-populated them for old revisions, log entries, and so on. Now we're starting to look at stopping the writes to the old fields, starting soon with testing wikis such as test.wikipedia.org and also with mediawiki.org. Other wikis will follow, likely in October after the DC switch-back. For the most part wiki users shouldn't notice any changes, however if you notice something not displaying a comment or edit summary for new entries, on these wikis let me know. For users of the Data Services replicas, such as Toolforge, the views currently simulate the old columns for you. However this may change in the future. See https://phabricator.wikimedia.org/T181650#4581384 for details. MediaWiki developers should make sure code accessing comment fields makes use of the CommentStore class that was introduced in MediaWiki 1.30. You can watch https://phabricator.wikimedia.org/T166733 (and any subtasks) for more information on the deployment process. -- Brad Jorsch (Anomie) Senior Software Engineer Wikimedia Foundation

1 2

Proposal regarding norms for meeting/deadline announcements
by Pine W 15 Oct '18

15 Oct '18

Hi Wikimedia-l and Wikitech-l, Keeping in mind the large numbers of subscribers on some Wikimedia email lists, the endless valuable uses for the time of knowledgeable volunteer Wikimedians, the significant financial costs for the time of many of the staff and contractors on these mailing lists, and how packed calendars can be, I propose that we implement a few social norms/guidelines for Wikimedia-l and Wikitech-l in particular. 1. When planning to have a one-time public meeting, announce it at least 14 days in advance to give everyone who might like to participate that much lead time to clear space on their calendars. Rarely is a one-time public meeting so urgent that it cannot wait 14 days from the day that it is announced. 2. Send a maximum of one reminder email regarding a one-time public meeting, and also send a maximum of one reminder email regarding events with deadlines such as Wikimania scholarship submissions or conference presentation proposals. More than one reminder about a meeting or deadline is excessive. 3. If extending a deadline, send only an announcement of the extension with no additional reminder. 4. Send only one email to announce a recurring weekly meeting, with no additional reminders. Meetings which recur less often, such as biweekly or monthly, may continue to be announced with one additional reminder. At this time these are proposals only. Comments are welcome. If the comments become extensive then I may request that we move the conversation to Meta. Thank you, Pine ( https://meta.wikimedia.org/wiki/User:Pine )

1 0

How much data can we plan to upload on Wikidata infrastructure? [Was: Fwd: Re: [wikidata] [glam] [Toulouse] Projet de partenariat CNES]
by Mathieu Lovato Stumpf Guntz 15 Oct '18

15 Oct '18

Hello, That might not be the most appropriate canal for this question, but I didn't have a better idea, so please let me know if you have better suggestion for my future demands. So, if you read French you can read the thread bellow, but basically to give some context to my question, we are looking at possible partnership with spatial agencies to feed the Wikimedia world with data. Depending on what we ask and achieve to make as agreement, the volume they could provide would be possibly really huge, with a given example of 1Go/minute for a single satellite. So my question is how much data should we aim at collecting, and depending on the volume, what transfer process should we use? Cheers -------- Message transféré -------- Sujet : Re: [wikidata] [glam] [Toulouse] Projet de partenariat CNES Date : Sun, 8 Apr 2018 13:06:00 +0200 De : Sébastien Dinot <sebastien.dinot(a)free.fr> Répondre à : Sébastien Dinot <sebastien.dinot(a)free.fr> Pour : Xavier Cailleau <xavier.cailleau(a)wikimedia.fr> Copie à : glam(a)lists.wikimedia.fr, toulouse(a)lists.wikimedia.fr, Liste OSM Toulouse <local-toulouse(a)listes.openstreetmap.fr>, ca(a)listes.openstreetmap.fr, wikidata(a)lists.wikimedia.fr, paris(a)lists.wikimedia.fr Sébastien Dinot a écrit : > Je dois pouvoir me libérer une demi-journée : Il est sans doute utile de préciser que je connais le projet Wikipédia depuis fort longtemps mais que mes contributions y sont fort modestes (quelques corrections d'articles et quelques photos) car on ne peut pas être sur tous les fronts à la fois (je suis un militant du logiciel libre depuis 1998 et un militant de l'open data depuis 2009, mais essentiellement dans le périmètre utile à la cartographie). En outre, je ne connais pas grand chose au climat et je peux manquer de pertinence sur le sujet. Par conséquent, je peux rencontrer vos interlocuteurs et sans doute être utile par ma connaissance du CNES et des licences, mais il me semble nécessaire d'être accompagné par quelqu'un qui connait bien mieux que moi Wikipédia et les projets connexes. Quels sont les objectifs de la fondation ? Obtenir des échantillons de données permettant d'illustrer des articles, des couvertures globales de l'Europe ou des terres émergées, de longues séries temporelles ? Quel volume de données est-il raisonnable d'envisager (dans le spatial, les volumes de données produits sont impressionnants : à ma connaissance, un seul satellite sentinel 2 transmet 12 Go de données brutes toutes les 12 minutes). Sébastien -- Sébastien Dinot, sebastien.dinot(a)free.fr http://sebastien.dinot.free.fr/ Ne goûtez pas au logiciel libre, vous ne pourriez plus vous en passer !

4 3

Wednesday: Technical Advice IRC Meeting
by Michael Schönitzer 15 Oct '18

15 Oct '18

Reminder: Technical Advice IRC meeting again **Wednesday 3-4 pm UTC** on #wikimedia-tech. Question can be asked in English & German. The Technical Advice IRC Meeting is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! Michi (for the Technical Advice IRC Meeting crew) -- Michael F. Schönitzer Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 http://wikimedia.de Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen Wissens frei teilhaben kann. Helfen Sie uns dabei! http://spenden.wikimedia.de/ Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

1 0

non-obvious uses of <nowiki> in your language
by Amir E. Aharoni 15 Oct '18

15 Oct '18

Hi, In MediaWiki, the <nowiki> tag was created for writing characters without having them interpreted as wiki syntax. An obvious and direct use case for this is writing help pages about editing wiki pages in wiki syntax, for example: Writing <nowiki>'''words between three apostrophes'''</nowiki> will show them in bold font: '''words between three apostrophes'''. Another related use case is demonstrating how templates work: <nowiki>This sentence shows the template used at the end.{{Citation needed|reason=Reliable source needed for the whole sentence|date=October 2018}}</nowiki> However, <nowiki> has less trivial use cases, that are not quite the same as demonstrating wiki syntax. One such usage I'm aware of is linking a part of a long compound German word, for example "[[Schnee]]<nowiki />reichtum". It produces the desired effect, however it is a bit of a hack: the word "nowiki" doesn't have anything to do with dividing compound words. This use is quite common in the German Wikipedia because of the nature of the German language, which has a lot of long compound words. Are there other languages where comparable hacks with <nowiki> exist, dictated by the nature of the language or by any local policies? -- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com ‪“We're living in pieces, I want to live in peace.” – T. Moore‬

16 31

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l October 2018