Wikitech-l September 2019

wikitech-l@lists.wikimedia.org

62 participants
67 discussions

Scrum of scrums/2019-09-18
by Željko Filipin 19 Sep '19

19 Sep '19

Hi, for HTML version see https://www.mediawiki.org/wiki/Scrum_of_scrums/2019-09-18 Željko -- = 2019-09-18 = == Callouts == * SRE says: various PDU upgrades throughout the next few weeks, we might reach out to specific teams. * Release Engineering - REMINDER: We're at 1.34.0-wmf.23 this week. The last branch for this release will be wmf.25 on 30 September. Teams who want to ship things for MW 1.34 should land them now. (If you do not mark code as deprecated in the next few weeks, you have to maintain it for another nine months.) * Analytics is blocking Search Platform: https://phabricator.wikimedia.org/T229882 == Product == === Editing === * Updates: ** Engaging in research to support talk pages improvements ** ApiVisualEditorEdit: Remove special handling for SpamBlacklist (task T211443) ** Add new EditAttemptStep stage: firstChange (task T229079) ** Add another looser selector for catching pasted references (task T232461) ** Implement firstChange logging (task T229079) ** Updated linters in Citoid (git #ac54ad79 ), CodeMirror (git #81ce8b33), Math (git #085ee7dc), Popups (git #9b3029e0), Score (git #dd534e19), Universal Language Selector (git #dae53b62), MinervaNeue (git #8e579c37) === Growth === * Updates: ** Homepage discovery for desktop no-JS users (task T225318) ** Standardize error handling (task T230798) ** [MW core RecentChanges] ChangesListSpecialPage: Cache the change tag list and its summary === iOS native app === * Updates: ** 6.4 released yesterday (09/17) https://phabricator.wikimedia.org/tag/ios-app-v6.4-seamonkey-on-a-hoverboar… *** iOS 13 & iPadOS updates *** New loading screen for dark mode *** Bug fixes *** external URLs > open in Safari ** 6.5 in development *** History & diffs API discussions with CorePlatform https://phabricator.wikimedia.org/tag/ios-app-v6.5/ **** Wrapping up first round of diff engine PR feedback https://phabricator.wikimedia.org/T232231 === Android native app === * Updates: ** Released a minor version to production with a timed survey https://play.google.com/store/apps/details?id=org.wikipedia&hl=en_US ** Continuing work on Suggested Edits V3 *** Bug fixes === Product Infrastructure === * Updates: ** DBA and security readiness reviews are in progress for Extension:MachineVision ** wikifeeds service (split out from mobileapps) deployed to production; not yet handling traffic ** Maps *** Production incident caused instability in eqiad cluster https://wikitech.wikimedia.org/wiki/Incident_documentation/20190913-maps === Structured Data === * Updates: ** also working on https://phabricator.wikimedia.org/T230862 (blocker from last week) ** working on - input types for structured data, machine-aided tagging, some minor campaigns stuff === Parsing === * Updates: ** Working with Marko / CPT to get Parsoid/PHP deployed and enabled on the beta cluster. ** Continuing to fix bugs found during mass roundtrip testing on scandium ( now 99.93% tests run without crashing ). == Technology == === Analytics === * Blocking: ** Search Platform: https://phabricator.wikimedia.org/T229882 === Fundraising Tech === * Updates: ** Payments-wiki *** More form behavior variants https://phabricator.wikimedia.org/T224929, https://phabricator.wikimedia.org/T231452 ** CiviCRM *** Annual totals receipt for recurring donors https://phabricator.wikimedia.org/T195907 *** Troubleshooting new screen-scraping data importer from mass mail house https://phabricator.wikimedia.org/T230509 ** CentralNotice *** Deploying last fixes for using EventLogging to track banner views *** Scheduling jobs on fundraising cluster to import EventLogging stats into local DBs === Core Platform === * Blocking: ** Search Platform: RecentChange support for SDC: https://phabricator.wikimedia.org/T230862 ** Performance: CR for rdbms/LoadMonitor changes https://gerrit.wikimedia.org/r/394430, https://phabricator.wikimedia.org/T230800 * Updates: ** API testing ** Kask testing ** New REST API interfaces === Engineering Productivity === ==== Performance ==== * Blocked by: ** SRE: Need CR and deploy for Apache SVG compression fix. – https://gerrit.wikimedia.org/r/535860 ** CPT: CR for rdbms/LoadMonitor changes https://gerrit.wikimedia.org/r/394430, https://phabricator.wikimedia.org/T230800 * Updates: ** Attended the We Love Speed conference in Lille, France. ** (Q-goal) Finished the AbuseFilter rework (Thanks to @Daimona) which will reduce Save Timing. Deployment expected in 1-2 weeks. – https://phabricator.wikimedia.org/T156095 ==== Release Engineering ==== * Blocked by: ** Search Platform: Need resolution on the Cirrus tests in mw-config; this is blocking general testing/expansion framework for this repo https://phabricator.wikimedia.org/T232691#5488675 * Updates: ** The phabricator "Report Error Code" form ( https://phabricator.wikimedia.org/maniphest/task/edit/form/46/ ) has been updated with separate fields for the stack trace and error code/request id. ** REMINDER: We're at 1.34.0-wmf.23 this week. The last branch for this release will be wmf.25 on 30 September. Teams who want to ship things for MW 1.34 should land them now. https://phabricator.wikimedia.org/T232026 (If you do not mark code as deprecated in the next few weeks, you have to maintain it for another nine months.) ** Train Health *** Last week: 1.34.0-wmf.22 - https://phabricator.wikimedia.org/T220747 *** This week: 1.34.0-wmf.23 - https://phabricator.wikimedia.org/T220748 *** Next week: 1.34.0-wmf.24 - https://phabricator.wikimedia.org/T220749 === Scoring Platform === * Updates: ** ORES deploy: https://phabricator.wikimedia.org/T232660 *** Trims metrics down: https://phabricator.wikimedia.org/T232164 *** Allow POST at some endpoint for large arguments: https://phabricator.wikimedia.org/T232500 *** Expand feature injection to all scoring endpoints: https://phabricator.wikimedia.org/T232143 ** Ongoing stability issue in WMFLabs (NOT production -- production is solid): https://phabricator.wikimedia.org/T232228 ** Discussing expanding the topic models with the Growth Team: https://www.mediawiki.org/wiki/Topic:Ub3g57qa9gflrlrc & https://phabricator.wikimedia.org/T231506 ** Built a library for detecting revert tools https://phabricator.wikimedia.org/T226426 === Search Platform === * Blocked by: ** Core Platform (or Multimedia?): RecentChange support for SDC: https://phabricator.wikimedia.org/T230862 ** SRE: dumpRDF for MediaInfo entities loads each page individually: https://phabricator.wikimedia.org/T222497 ** Analytics: https://phabricator.wikimedia.org/T229882 * Blocking: ** Release Engineering: Need resolution on the Cirrus tests in mw-config; this is blocking general testing/expansion framework for this repo https://phabricator.wikimedia.org/T232691#5488675 * Updates: ** Added explanation api for completion suggester scoring: https://phabricator.wikimedia.org/T230919 ** Migrated integration testing to node10 === Security === * Updates: ** Data Map / GDPR Compliance updates ** Security Reviews in Progress are: *** Security Review for MediaWiki REST API, assigned to Sam, https://phabricator.wikimedia.org/T230140 *** Security review of Ex:DoubleWiki, in-progress - https://phabricator.wikimedia.org/T131199 *** Parsoid-PHP, moved to active, Scott this week - https://phabricator.wikimedia.org/T227209 *** Page Content Service route /page/mobile-html, Scott will post 9/17 - https://phabricator.wikimedia.org/T227114 *** Security review of WebAuthn library dependencies, in-progress - https://phabricator.wikimedia.org/T227244 === Site Reliability Engineering === * Blocking: ** CPT on restrouter deployment ** Search Platform: dumpRDF for MediaInfo entities loads each page individually: https://phabricator.wikimedia.org/T222497 ** Performance: Need CR and deploy for Apache SVG compression fix. – https://gerrit.wikimedia.org/r/535860 * Updates: ** Helped product infrastructure team deploy wikifeeds \o/. ** Various PDU upgrades throughout the next few weeks, we might reach out to specific teams.

1 1

content_models table only contains wikitext content model on fresh MW 1.32.3 install
by Tom Schulze 19 Sep '19

19 Sep '19

Dear all, for our deployment procedure we're setting up a clean database with some prefilled content. I use MW 1.32.3 for that. When I did the fresh install (minimal installation method) I realized that the 'content_models' database table only contains one row with model_id = 1 and model_name = wikitext. I then install extensions, run update.php, and import a couple pages, templates, etc. When setting up Cargo's _pageData table using the setCargoPageData.php script at a later stage I get the following error (see backtrace at the very bottom). Failed to access name from content_models using id = 2 I get this error fixed by inserting the core content models <https://www.mediawiki.org/wiki/Content_handlers> into the 'content_models' table manually. Accessing the pages afterwards works fine (Widgets in this case). The widgets themselves work fine, too. Am I missing something that the core content models are not in the initial database table 'content_models'? Is this table not filled during the installation procedure? Is it actually bad advice to insert the rows manually? Kind regards and thanks for your support, Tom [12081d263c4cc289d8a144d8] [no req] MediaWiki\Storage\NameTableAccessException from line 42 of /var/www/mediawiki/lib/includes/Storage/NameTableAccessException.php: Failed to access name from content_models using id = 2 Backtrace: #0 /var/www/mediawiki/lib/includes/Storage/NameTableStore.php(308): MediaWiki\Storage\NameTableAccessException::newFromDetails(string, string, integer) #1 /var/www/mediawiki/lib/includes/Revision/RevisionStore.php(1625): MediaWiki\Storage\NameTableStore->getName(integer) #2 /var/www/mediawiki/lib/includes/Revision/RevisionStore.php(1671): MediaWiki\Revision\RevisionStore->loadSlotRecords(string, integer) #3 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}() #4 /var/www/mediawiki/lib/includes/Revision/RevisionSlots.php(165): call_user_func(Closure) #5 /var/www/mediawiki/lib/includes/Revision/RevisionSlots.php(107): MediaWiki\Revision\RevisionSlots->getSlots() #6 /var/www/mediawiki/lib/includes/Revision/RevisionRecord.php(192): MediaWiki\Revision\RevisionSlots->getSlot(string) #7 /var/www/mediawiki/lib/includes/Revision/RevisionRecord.php(175): MediaWiki\Revision\RevisionRecord->getSlot(string, integer, NULL) #8 /var/www/mediawiki/lib/includes/Revision.php(932): MediaWiki\Revision\RevisionRecord->getContent(string, integer, NULL) #9 /var/www/mediawiki/lib/includes/page/WikiPage.php(801): Revision->getContent(integer, NULL) #10 /var/www/mediawiki/lib/extensions/Cargo/includes/CargoPageData.php(107): WikiPage->getContent() #11 /var/www/mediawiki/lib/extensions/Cargo/maintenance/setCargoPageData.php(88): CargoPageData::storeValuesForPage(Title) #12 /var/www/mediawiki/lib/maintenance/doMaintenance.php(94): SetCargoPageData->execute() #13 /var/www/mediawiki/lib/extensions/Cargo/maintenance/setCargoPageData.php(103): require_once(string) #14 {main} -- Tom Schulze energypedia consult GmbH König-Adolf-Str. 12 65191 Wiesbaden Phone: +49 0611 18195031 Email: t.schulze(a)energypedia-consult.com Web: www.energypedia-consult.com | www.webmo.info Registergericht: Frankfurt, Eintragungs-Nr. HRB 93412 | Sitz: Eschborn | Geschäftsführung: Robert Heine

3 4

Fwd: [Wikimedia Announcements] Welcoming Wikimedia Foundation’s new CTO, Grant Ingersoll
by Pine W 18 Sep '19

18 Sep '19

Forwarding to Wikitech-l and MediaWiki-l. Pine ( https://meta.wikimedia.org/wiki/User:Pine ) ---------- Forwarded message --------- From: Katherine Maher <kmaher(a)wikimedia.org> Date: Wed, Sep 18, 2019, 10:44 Subject: [Wikimedia Announcements] Welcoming Wikimedia Foundation’s new CTO, Grant Ingersoll To: wikimediaannounce-l <WikimediaAnnounce-l(a)lists.wikimedia.org> Hi all, I’m excited to officially welcome Grant Ingersoll as our Chief Technology Officer! Grant will be starting September 23. He’ll be based in Charlotte, North Carolina. Grant comes from a long history of working on open source projects. Most recently, he served as the Chief Technical Officer of Lucidworks, an AI-powered open-source search services company which he co-founded. He is a Lucene and Solr committer, co-founder of the Apache Mahout machine learning project, and a long-standing member of the Apache Software Foundation. He’s an author, having written a book for Java developers on how to wrangle unstructured text for search, text-mining, and the like. He’s also a long-time, committed remotee, having worked with the distributed Lucidworks team from his home in North Carolina for nearly a decade. In Grant’s own words, “The power of learning and knowledge have always stood as key pillars in my career. I'm a big believer that access to free, trusted knowledge is of vital importance as society looks to tackle large scale challenges. My wife, Robin, and I recently moved from Chapel Hill, North Carolina, our home for 10 years, to Charlotte, North Carolina, where our daughter lives. We also just dropped our son off at college for his freshman year. We're adjusting to the empty nest life with our dog Allie (a black lab mix). When not traveling or exploring Charlotte, I can usually be found on a bike, out kayaking, or writing code.” Grant will be working with myself, Erika Bjune (who is transitioning to VP, Technology), and others in the leadership of the technology department to determine a set of priorities for his first year. I expect these will likely focus around evaluating our current capacities and co-creating a vision for the continued evolution of our technical platforms, supporting the staff of the department, working with the finance and operations folks on planning and budgeting for the future, and of course, getting to know our technical community and the broader movement. Under Grant’s leadership, we will continue the work of improving and modernizing our technical ecosystem to respond to our future needs, as laid out in the movement strategy. I’m thrilled to have the CTO role filled, and to bring Grant in at a time when the movement is digging into the question of what it means to “become the essential infrastructure of free knowledge.” From our first meeting, I was struck by his curiosity. He was genuinely interested in how Wikimedia works, and was willing to get into what I often think of “Wiki PhD” level conversations about the nuances of our community and values. I have generally found that the folks who bring that sort of openness and humility to their work are the folks who thrive in the challenges of our mission and movement. As our movement expands, I’m glad to have Grant’s experience, curiosity, and passion for building things on board here at the Foundation. I want to also thank Erika Bjune for her work as interim CTO. She passionately advocated for the importance of our platforms, embodied partnership and cooperation, and her expertise on the interview panel was invaluable. I’m thrilled she’ll be working so closely with Grant as we move forward. Please join me in welcoming Grant to the Wikimedia Foundation! Katherine -- Katherine Maher (she/her) Executive Director Wikimedia Foundation <https://wikimediafoundation.org/> _______________________________________________ Please note: all replies sent to this mailing list will be immediately directed to Wikimedia-l, the public mailing list of the Wikimedia community. For more information about Wikimedia-l: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l _______________________________________________ WikimediaAnnounce-l mailing list WikimediaAnnounce-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimediaannounce-l

1 0

New section in code search: Wikimedia Services
by Amir Sarabadani 17 Sep '19

17 Sep '19

Hey, Codesearch [1] now indexes Wikimedia services including Mobile apps, Kask, Parsoid, CX server, PoolCounter, and EventStream. You can also search only in services section by going to [2]. Let me know if anything doesn't work properly. [1] https://www.mediawiki.org/wiki/Codesearch [2] https://codesearch.wmflabs.org/services/ Best -- Amir (he/him)

1 0

Re: [Wikitech-l] Adding a gadget to VisualEditor
by Yaron Koren 17 Sep '19

17 Sep '19

Hi Alain, First of all, thanks for choosing the Cargo extension! As the main author of that extension, I'm happy you're trying it out. So, by an amazing coincidence, over the last month or so I've been working on creating a new extension that allows for saving and displaying BPMN diagrams within the wiki. The extension isn't released yet, and it doesn't even have an official name yet - right now it's called "Diagrams", because it's meant to support other diagram languages besides BPMN, although right now it only supports BPMN. However, you can already see it in action on this wiki: http://discoursedb.org/wiki/BPMN:Test_diagram The contents of that wiki page are simply one BPMN file - it's an approach similar to what the Scribunto extension does with Lua modules. To edit the diagram, you can edit the text directly, or click on the "Edit diagram" tab to edit it graphically, using bpmn.io: http://discoursedb.org/w/index.php?title=BPMN:Test_diagram&action=diagramed… There's even a Cargo tie-in - some of the essential information from all the diagrams is stored in a Cargo table, where it can be queried: http://discoursedb.org/wiki/Special:CargoTables/_bpmnData It's a somewhat different approach from what you are trying to do, but hopefully similar enough that it would be useful for you. I'm curious to know what you think - if you want to talk about it further, please write me back, privately or on this mailing list. -Yaron On Tue, Sep 16, 2019 at 6:38 AM Alain PERRY <alainp(a)franceactive.org> wrote: > Hello list, > > I was sent here from the #mediawiki IRC channel, I hope this is indeed the > correct list to write to and ask you to please forgive me if it is not. > > First, a bit about myself: I'm no real developer, just a tech-savvy guy > trying to build a wiki to document my employer's company inner workings. > This last bit means documenting any piece of information we handle and our > business processes in doing so. > > After testing a few wiki engines, I came to the conclusion that mediawiki > with the cargo extension was what I was looking for. I also built a small > extension to allow inclusion and edition of BPMN diagrams with the > https://bpmn.io library. All in all, this is pretty basic. > > Then I came to the realization that if I wanted any lambda-user to be > willing to contribute to the wiki content, I needed the Visual Editor. > Deploying it was easy enough, and everything seems to be working fine at > the moment. > > I however feel I should make it possible to include a BPMN diagram from > the VE toolbar. Ideally, this will take the form of a "popup" that will > include the bpmn.io modeler to edit the diagram. However, in order to > achieve this goal, I have a whole damn lot to learn and to experiment with > about VE. > > So I thought I should start with something a little humbler: a button that > would just include some basic XML in the page (a special page already > allows editing that part of the page, so this would already be convenient, > though far from perfect). Using the gadgets example from the documentation > on the mediawiki wiki, I'm able to insert content into the page. But that > content is escaped if it includes XML and I have no idea from the API doc > how to include something less basic than mere text (or than a given > template, since the example actually shows how to do that). > > What I'm actually getting at is this: would someone with good knowledge of > the API and some good old patience be willing to "tutor" me by giving > pointers where I need them ? > > My first technical question would be this: I understand that my gadget can > register a ve.ui.Command, that will in turn call a method on an > ve.ui.Action object. I guess I should stick to the ContentAction one. But > I'm not sure what the "content" parameter should contain if given an array. > Is there, somewhere, some documentation I can read on this? > > Thanks a bunch for any help. > > Regards, > > -- > Alain Perry > -- WikiWorks · MediaWiki Consulting · http://wikiworks.com

1 0

Pencils down on September 29 for REL1_34 branching on September 30
by Cindy Cicalese 17 Sep '19

17 Sep '19

Hello! We are beginning the process of working on the 1.34.0 release of MediaWiki. The release is currently scheduled for November 2019. We anticipate creating the REL1_34 branch on September 30 (aligned with the final Wikimedia alpha 1.34.0-wmf.25), so we are asking for "pencils down" on September 29. Once the REL1_34 branch is cut, any patches that need to get into the release will be in the normal patch master + backport process until the release in November. If you have any open Phabricator tasks tagged with mw-1.34-release [0], please check to see if they are indeed blockers for the release. If not, please remove the tag from them. Conversely, if there are any blockers that are not tagged with mw-1.34.release, please tag them. Please feel free to reach out to me if you have any questions about this. Thanks, Cindy [0] https://phabricator.wikimedia.org/tag/mw-1.34-release/ _____________________________ Cindy Cicalese Product Manager, MediaWiki Platform Wikimedia Foundation

1 1

TechCom Radar 2019-09-11
by Kate Chapman 16 Sep '19

16 Sep '19

Hi All, Here are the minutes from this week's TechCom meeting: * RFC Approved: Define criteria for setting explicit PHP support target for MediaWiki <https://phabricator.wikimedia.org/T228342> Patch available for RFC: Section headings should have a clickable anchor <https://phabricator.wikimedia.org/T18691> (please comment on ticket if you are interested in this) Discussed: Drop IE6 and IE7 basic compatibility and security support. <https://phabricator.wikimedia.org/T232563> You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. If you prefer you can subscribe to our newsletter here <https://www.mediawiki.org/wiki/Newsletter:TechCom_Radar> Thanks, Kate -- Kate Chapman Senior Program Manager, Core Platform Wikimedia Foundation kchapman(a)wikimedia.org

1 0

GSoC 2019 and Outreachy Round 18 are over! Call for projects and mentors for Outreachy Round 19 begins.
by Srishti Sethi 16 Sep '19

16 Sep '19

Hello everyone, Google Summer of Code 2019 and Outreachy Round 18 are over! See a complete list of 15 projects interns worked on this past summer: https://www.mediawiki.org/wiki/Google_Summer_of_Code/Past_projects#2019, https://www.mediawiki.org/wiki/Outreachy/Past_projects#Round_18 If you find a project relevant to your area of interest, you can explore more via the resources mentioned in the links above. Help test the tools or features and share your feedback or any information on bugs you encountered in a comment on the corresponding Phabricator task. Next up is Outreachy Round 19! Wikimedia is participating in the winter edition! If you are interested in mentoring a coding or non-coding (documentation, design, research, outreach, translation, etc.) project, and you are not familiar with the program, reach out to me (via email ssethi(a)wikimedia.org) to learn more about the next steps. If you are already familiar with the program and want to share your idea for a project that you would like to mentor, share in a comment on this Phabricator task: https://phabricator.wikimedia.org/T232048. The *deadline to submit* projects on the Outreachy website is *September 24th, 2019*. Some relevant links: * A recent blog post highlighting one of our intern's and mentor's experience participating in Outreachy program with Wikimedia: https://wikimediafoundation.org/news/2019/06/25/wikimedia-and-outreachy-tec… * Outreachy website: https://www.outreachy.org/. * Roles and responsibilities of an Outreachy mentor: https://www.mediawiki.org/wiki/Outreachy/Mentors Looking forward to your participation! Cheers, Srishti *Srishti Sethi* Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

1 1

Gerrit: Internet Explorer no longer supported under PolyGerrit
by Paladox 16 Sep '19

16 Sep '19

Hello All! PolyGerrit no longer supports Internet Explorer. It never really worked so now the offical status is "not supported". You can use alternative browsers like Chrome, Firefox, Opera and Microsft Edge. GWTUI will support internet explorer. Though GWTUI has been removed in newer versions of gerrit.

1 0

[Train] 1.34.0-wmf.22 status update
by Antoine Musso 16 Sep '19

16 Sep '19

Hello, Deployment of MediaWiki 1.34.0-wmf.22 on the Wikimedia cluster is currently blocked. It is blocked on a php7.2 bug that has yet to be tracked and determined. Immediately after deploying to group1 (which has commons and wikidata), I noticed an infamous error in the logs: PHP Notice: Undefined index:" Which sounds similar to known errors with php7.2 (T229433, T231089 among others) but this time it hits LBFactoryMulti.php which is for the database load balancing. So that sounds scary and I thus decided to block the train. The notice happens at: $groupLoads[ILoadBalancer::GROUP_GENERIC] The GROUP_GENERIC constant is an empty string. Brad "anomie" Jorsch quickly found out the code always set that key! https://phabricator.wikimedia.org/T232613#5485181 So what is left is a edge case issue in php 7.2, maybe in the opcache or something even scarier. Thanks to Tim Starling, we have a solution to issue a core dump of php when that issue happens. There is already available on the cluster, but it has to be investigated which is not that trivial. -- Antoine Musso

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2019