Wikitech-l July 2018

wikitech-l@lists.wikimedia.org

82 participants
73 discussions

QUnit, Promises and ArticlePlaceHolder
by Antoine Musso 23 Jul '18

23 Jul '18

Hello, The ArticlePlaceholder extension has QUnit tests run via nodejs. I am trying to port them to Special:JavaScriptTest. I am hitting a wall due to an OOUi widget not being found on the first test. Most probably because the window is opened asynchronously and the assertions are run before it opens. JavaScript Promises are not my thing and I know nothing about OOUi. If that is in your area of expertise, I could use a hand to finish up the port: https://gerrit.wikimedia.org/r/#/c/444936/6 Thank you! -- Antoine "hashar" Musso

4 3

Add modules=site.styles inline
by Nischay Nahata 21 Jul '18

21 Jul '18

Hi all, It seems the Common.css on my website is pretty small but Resource Loader loads it using a <link> tag in the head. I have read that it will be better to instead inline the styles or even load them later. What is the right way to go about this? Any suggestions are welcome. Also see https://developers.google.com/speed/docs/insights/OptimizeCSSDelivery Regards, Nischay Nahata

2 8

TechCom radar 2018-07-18
by Kate Chapman 21 Jul '18

21 Jul '18

Hi All, Below are this week's TechCom minutes: * IRC Meeting Occurred: Unify various deletion systems <https://phabricator.wikimedia.org/T20493> * Meeting minutes: <https://tools.wmflabs.org/meetbot/wikimedia-office/2018/wikimedia-office.20…> * There is the question of how Special:Contributions and Special:DeletedContributions will work and how if required to provide feature parity. Further summary here: <https://phabricator.wikimedia.org/T20493#4436769> * New RFC: Partial blocks <https://phabricator.wikimedia.org/T199917> The Anti-Harassment team is asking TechCom to review a plan for implementing Partial Blocks which seeks to allow admins to block users from specific pages and/or all articles inside a namespace. * New RFC: Add a language agnostic build step to skins/extensions to our deploy process <https://phabricator.wikimedia.org/T199004> Currently requires developers to do the build step locally and check-in the results. This RFC proposes making a universal/repeatable process. * New RFC: Extension:JADE scalability concerns due to creating a page per revision <https://phabricator.wikimedia.org/T196547> RFC proposal still being developed * New RFC: Re-evaluate librsvg as SVG renderer on Wikimedia wikis <https://phabricator.wikimedia.org/T40010> This RFC is re-evaluating the use of librsvg to generate PNGs from SVG due to rendering issues and determining if there is a better alternative. * IRC Meeting scheduled for 2018-08-01 in the #wikimedia-office channel at 2pm PST(22:00 UTC, 23:00 CET) RFC: Modern Event Platform - Choose Schema Tech <https://phabricator.wikimedia.org/T198256> This RFC is looking at if Apache Arvo or JSONSchema should be used for the event schema and serialization method. * No IRC Meeting scheduled for July 25 You can also find our meeting minutes at <https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes> See also the TechCom RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. If you prefer you can subscribe to our newsletter here: <https://www.mediawiki.org/wiki/Newsletter:TechCom_Radar> Thanks, Kate -- Kate Chapman TechCom Facilitator (Contractor)

1 0

Timeless Newsletter
by Isarra Yos 20 Jul '18

20 Jul '18

Hey all, For anyone not familiar, the Timeless skin has been available as an option on Wikimedia wikis since november or so, and development of the skin is now being funded by a Project Grant. If you're interested in receiving updates about this, I've created a newsletter which should be going out monthly. You can sign up for the newsletter here: https://meta.wikimedia.org/wiki/Timeless/Newsletter Or read the first issue: https://meta.wikimedia.org/wiki/Timeless/Newsletter/1 Timeless: https://www.mediawiki.org/wiki/Skin:Timeless?useskin=timeless The grant now funding this: https://meta.wikimedia.org/wiki/Grants:Project/Isarra/Timeless:_Post-deploy… Thank you to everyone for your interest and support thus far, and making all of this possible! -Isarra

1 0

hewiki dump to be added to 'big wikis' and run with multiple processes
by Ariel Glenn WMF 20 Jul '18

20 Jul '18

Good morning! The pages-meta-history dumps for hewiki take 70 hours these days, the longest of any wiki not already running with parallel jobs. I plan to add it to the list of 'big wikis' starting August 1st, meaning that 6 jobs will run in parallel producing the usual numbered file output; look at e.g. frwiki dumps for an example. Please adjust any download/processing scripts accordingly. Thanks! Ariel

1 0

CI jobs using npm might suffer from a 10 minutes delay
by Antoine Musso 20 Jul '18

20 Jul '18

Hello, Since June 27th, any CI job running 'npm install' might suffer from a 10 minutes extra delay. Somehow when requesting package informations from the NpmJS CDN (CloudFlare), the connection holds for ten minutes. npm just idles waiting for a reply. Then eventually it shows: npm ERR! registry error parsing json npm then retry and process as usual. The json error is due to a CloudFlare HTML page stating: The page could not be rendered due to a temporary fault. The impact is any Jenkins job using npm have a high chance of taking 10 more minutes to build. That notably impacts MediaWiki core and all its extensions. A few minutes ago, I have made a change to run npm with --loglevel=info which would give some hints about what it is doing by causing npm to emit more informations in the console. (verbose would be way too much log though). I have filled a bug to npm: https://github.com/npm/npm/issues/21101 Our task: https://phabricator.wikimedia.org/T198348 I have no idea how to mitigate the issue :-( -- Antoine "hashar" Musso

3 4

Patchsets by new Gerrit contributors waiting for code review and/or merge
by Andre Klapper 20 Jul '18

20 Jul '18

CR0: Please review and provide guidance if you are familiar with the code, and decide (CR±1 or CR±2): * https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/164049/ ** Add Tatar LanguageConverter ** 2018-May-24 (though commented by Thiemo on 2018-Jul-05) ** Maintainers/Stewards: Contributors > Parsing team * https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Drafts/+/443128/ ** Adding code standards configuration for phpcs to drafts. ** 2018-Jul-09 ** Maintainers/Stewards: ? CR+1: Please help make a decision (CR±1, CR±2) on these CR+1 patches: * https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/ApprovedRevs/+/4340… ** Add support for custom content types other than WikitextContent ** 2018-Jun-17 ** Maintainers/Stewards: ? Read https://www.mediawiki.org/wiki/Gerrit/Code_review#By_project how you can get notified of new patches in your code areas of interest. Thanks in advance for your reviews! Of last edition's 4 listed patches, 2 got merged. Thanks to Fisch and Mholloway! Maintainers/Stewards data taken from https://www.mediawiki.org/wiki/Developers/Maintainers CR0 source: https://gerrit.wikimedia.org/r/#/q/ownerin:newcomers+status:open+label:Veri… CR+1 source: https://gerrit.wikimedia.org/r/#/q/ownerin:newcomers+status:open+label:Veri… -- Andre Klapper | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

1 0

2018-07-18 Scrum of Scrums meeting notes
by Grace Gellerman 20 Jul '18

20 Jul '18

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-07-18 = 2018-07-18= == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar * [Perf] Seeking input on wmf-config variables that appear to be unused, and which we'd like to remove: https://wikitech.wikimedia.org/wiki/User:Krinkle/Unused_config == Audiences == === Readers === ==== iOS native app ==== (Natalia H) * Blocked by: Question for Analytics: can we return an error code if event logging validation fails? Right now we have to test menually using https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/TestingO… ** With the current system, no, because the endpoint you're sending to is a fire-and-forget Varnish server. This is a good thing to bring up in the Modern Event Platform Context. ^ thanks, that makes sense 👍 (let me know if you want to brainbounce the problem that this causes ~~~~ milimetric) * Blocking: * Updates: **User testing of feed redesign and navigation changes in progress ( https://phabricator.wikimedia.org/T198932) **6.0.0 ( https://phabricator.wikimedia.org/tag/ios-app-v6.0-walrus-on-a-unicycle/) feature-complete by the end of this week **Product at Wikimedia ==== Android native app ==== (Natalia H) * Blocked by: * Blocking: * Updates: **Finishing up prototypes of navigation updates: https://phabricator.wikimedia.org/project/view/3367/ **Maintenance release this week to address minor analytics issues and audio playback. https://phabricator.wikimedia.org/T198504 ==== Readers Web ==== (Stephen N) * Blocked by: ** New service request: chromium-render/deploy https://phabricator.wikimedia.org/T186748 *** We're not blocked but we're ready to help as needed on: **** https://gerrit.wikimedia.org/r/#/c/mediawiki/services/chromium-render/+/443… (+1d but needs review by Petr when available!) **** https://phabricator.wikimedia.org/T199264 (not sure what the status is) * Blocking: * Updates: ** RFC: Add a language agnostic build step to skins/extensions to our deploy process https://phabricator.wikimedia.org/T199004 ** Mobile website (MinervaNeue / MobileFrontend): *** Page issues UI and instrumentation T191532 T191528 T197932 T197931 T199005 *** Special pages preferences T186760 *** Document existing JavaScript code coverage T197637 *** Other fixes and hygiene T193172 T197133 T198930 ** WikidataPageBanner *** Banners should display the same in RTL or LTR T198818 ** PDF rendering (Proton): *** Font configuration T199264 ** Desktop website (Popups): *** Identify aborted HTTP requests logged T199482 *** Some JSDoc improvements and other fixes T198663 ** Wikipedia.org (Portals) *** Extra spacing on article counts T199337 ** Product and design at Wikimania ** Design finishing navigation prototypes for advanced contributors ==== Readers Infrastructure ==== * Blocked by: n/a * Blocking: n/a * Updates: ** Working on updating maps dependencies to use a more recent Mapnik version (https://phabricator.wikimedia.org/T188674) ** Building a Docker Compose setup for maps stack ( https://phabricator.wikimedia.org/T193232) ** content-html endpoint: *** new name tbd T199491 *** working on splitting up CollapseTable in wikimedia-page-library into client and server portions ** Working on respecting the accept-language header in MCS ( https://phabricator.wikimedia.org/T197009) ==== Multimedia ==== * Blocked by: * Blocking: * Updates: ** Half the team is incommunicado for Wikimania; progress is minimal this week I think ** Search prototype is progressing ** Designs coming through soon for other work === Contributors === ==== Community Tech ==== * Blocked by: *Security*: Need security review for TemplateWizard extension https://phabricator.wikimedia.org/T198666 * Blocking: * Updates: ** GlobalPreferences API is out on beta. There's a breaking change that fixes a 'bug' that was introduced with blacklist notifications. Should not affect anyone. ** ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** ==== Parsing ==== * Blocked by: * Blocking: * Updates: ** Completed migration from Tidy to Remex, no major issues reported ==== Growth ==== * Blocked by: * Blocking: * Updates: ** ==== Language ==== * Blocked by: https://phabricator.wikimedia.org/T199011 -> VE: https://phabricator.wikimedia.org/T196521 * Blocking: * Updates: ** Progress on 'Progress Calculation' for CX2. ** Some annoying bug fixes for CX1. ** Fixed issues with cxserver and maintainance updates. === Audiences Design === ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** OOUI v0.27.5 released last week with accessible tabs, and standard buttons in dialogs. No release this week ** Continuous work on Special:Preferences connected tasks and minor support for Special:Log to OOUI preparation ** Design Style Guide: Color section/palette visual refresh, mobile friendly: https://design.wikimedia.org/style-guide/visual-style_colors.html == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** We had a kafka main outage due to a stale testing environment for rest base, working with SRE teams to get more kafka expertise in ops teams so analytics is not the main point of contact of tier-1 service: https://wikitech.wikimedia.org/wiki/Incident_documentation/20180711-kafka-e… ** New maps on wikistats. See pageviews for portuguese wikipedia around the world: https://stats.wikimedia.org/v2/#/pt.wikipedia.org/reading/page-views-by-cou… |map|2-Year~2016060100~2018071700|~total ** Added Historic data to UI, see legacy pageviews for portuguese wikipedia: https://stats.wikimedia.org/v2/#/pt.wikipedia.org/reading/legacy-page-views… |bar|All~1980010100~2018071700|~total ** More friendly configuration on turnilo to explore datasets with numeric dimensions, this will help SRE and perf team troubleshoot issues with near-realtime data. See request-size for example: https://turnilo.wikimedia.org/#webrequest_sampled_128 === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Thanks to the core team for fixing the bug blocking CentralNotice translatable message saves! ** More public testing of new API for main CC processor ** More work on email opt-in checkbox for selected countries ** More work on donor data export and expurgation ** More work on new data pipeline for banner and landing page views ** Working on support for es-419 (LatAm Spanish) in donation pipeline === MediaWiki Core Platform === * Blocked by: * Blocking: * Updates: ** === Performance === (Ian M) * Blocked by: **SRE - Waiting on several puppet reviews/merges related to the webperf profile/roles * Blocking: ** None * Updates: **Mcrouter writes config broke things when we deployed it last week, resulting in a partial outage. Rolled back, and have since fixed that issue, but have not scheduled the rollout of the fix. Incident report at https://wikitech.wikimedia.org/wiki/Incident_documentation/20180710-MediaWi… ** Also found an issue with mcrouter not properly supporting Unicode keys. Wrote a patch to address that which has landed on master, and has been backported. ** fixed a few different unit tests that were sporadically failing ** Seeking input on wmf-config variables that appear to be unused, and which we'd like to remove: https://wikitech.wikimedia.org/wiki/User:Krinkle/Unused_config === Release Engineering === (Hashar) * Blocked by: * Blocking: ** webpack for extensions https://phabricator.wikimedia.org/T199004 ** ORES CI patches fail https://phabricator.wikimedia.org/T198201 * Updates: ** {done} Mobile CI job to Docker: https://gerrit.wikimedia.org/r/#/c/integration/config/+/442126/ ** Flappy tests blocking PageTriage changes https://phabricator.wikimedia.org/T198137 *** worked around by disabling a test on Friday 07/13 https://gerrit.wikimedia.org/r/445394 *** Test suite ongoing improvement to properly fix the underlying issue ** CI Jenkins to be upgraded to latest LTS ** Various wikimedia deployed extensions fail tests in CI. Antoine will reach out to teams. (parent task https://phabricator.wikimedia.org/T183512 ) *** Eg: ArticlePlaceholder , Cognate, intersection, LabeledSectionTransclusion, TrustedXFF, WikibaseLexeme === Research === * Blocked by: none * Blocking: none * Updates: ** Wikimania ** Analyzed the inital set of CitationUsage data. Making adjustments to the code to collect more data. ** Working on re-running the UnsourcedStatements WikiLabels campaign. === Scoring Platform === * Blocked by: ** JADE storage discussion continues, latest twist is that we should go back to TechCom. * Blocking: * Updates: ** wp10 storage enabled in euwiki ** Migrating to new library for abstraction of wikibase entities in ores is going on (from pywikibase to mwbase) === Search Platform === * Blocked by: * Blocking: * Updates: ** === Security === * Blocked by: * Blocking: * Updates: ** === Services === * Blocked by: * Blocking: * Updates: ** === Site Reliability Engineering === * Blocked by: * Blocking: * Updates: ** == Wikidata == * Blocked by: * Blocking: * Updates: ** == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == Multi-Content Revisions == * Blocked by: * Blocking: * Updates: ** == SoS Meeting Bookkeeping == * Updates: **Would it help to schedule this meeting earlier in the day? ***Participants of the meeting said yes ***Proposal to experiment with holding meeting 2 hours earlier

1 0

Announcing TechCom’s Newest Members
by Daniel Kinzler 19 Jul '18

19 Jul '18

Dear All, back in June the Wikimedia Technical Committee (TechCom) began a process to recruit two new members to broaden the committee’s area of expertise[1]. After some deliberation, we have now concluded our process and would like to welcome Niklas Laxström and Dan Andreescu to TechCom. Congratulations, we are looking forward to you bringing your expertise to the committee! For those who wonder what this is about: TechCom is the guardian of the integrity, consistency, stability, and performance of all software supporting the Wikimedia projects. It acts as the senior advisor and the convergence point of all decisions related to technical work that is strategic, cross-cutting, and/or hard to undo. You can find more information in the TechCom Charter[2]. To those who applied or were nominated, but were not selected, I want to say: thank you for offering your time and brain power! It was a difficult decision to make. We evaluated all candidates based on the following criteria: - Activity on RFCs and other Phabricator tickets - Area(s) of expertise - Having a unique engineering perspective - Experience working in and with the Wikimedia movement and community It was a challenge to balance these criteria, especially with candidates who have a lot of experience, but whose expertise and perspective is similar to that of existing members. In such cases, we aimed to improve the diversity of perspective and knowledge on the committee by picking candidates that would help us cover any blind spots. We plan to do another round of nominations again soon, and hope to have continued interested from others in joining TechCom. Thank you all, [1] https://lists.wikimedia.org/pipermail/wikitech-l/2018-June/090068.html [2] https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Charter

7 6

Wikimedia production problems (July 2018)
by Krinkle 19 Jul '18

19 Jul '18

Hi! Striving for operational excellence, here's a summary of recent issues in production that can currently affect end-users. ## Statistics * 124 fatals. (channel=fatal) * 3,986 exceptions. (channel=exception) * 34,559 PHP errors. (channel=error) * 42,222 errors in total. (channel=*, level=ERROR) >From logstash.wikimedia.org, last 7 days, having type=mediawiki. ## Higher impact These caused users (of web or api) to see errors. From logstash.wikimedia.org, ordered by frequency: * [Babel extension] https://phabricator.wikimedia.org/T199941 - MWException "Language::isValidBuiltInCode must be passed a string". * [Flow extension] https://phabricator.wikimedia.org/T70526 - InvalidDataException "Flow workflow is for different page". * [Page deletion] https://phabricator.wikimedia.org/T195692 - Undelete for certain pages aborted by IncompleteRevisionException. * [Special pages] https://phabricator.wikimedia.org/T199044 - Special:ActiveUsers aborted by DBQueryTimeoutError. * [MobileFrontend] https://phabricator.wikimedia.org/T199066 - Special:MobileContributions shows "Special:Badtitle" (Revision::ensureTitle error). * [AbuseFilter extension] https://phabricator.wikimedia.org/T187153 - Special:Abuselog throws BadMethodCallException on details/examine. ## Noise These are caused by code not behaving unexpectedly, but with limited impact due to graceful recovery by PHP or other code. These harm our ability to detect and prevent higher impact issues (through in Scap). They may also be symptoms of deeper issues. * [CirrusSearch extension] https://phabricator.wikimedia.org/T180807 - Undefined index: _score and _id in CompSuggestQueryBuilder. * [Collection extension] https://phabricator.wikimedia.org/T189636 - Undefined 'subtitle' and 'title' indexes in CollectionPageTemplate. * [WMF-JobQueue] https://phabricator.wikimedia.org/T199594 - Exception "Job queue is read-only". * [CentralAuth extension] https://phabricator.wikimedia.org/T170971 - iconv Detected an illegal character. * [Cite extension] https://phabricator.wikimedia.org/T182929 - Undefined index: "" in Cite.php. ## Thank you Thanks to the following people who recently helped investigate and/or resolve similar issues. * Jdforrester (James Forrester) * Marostegui (Manuel Aróstegui) * jcrespo (Jaime Crespo) * SBisson (Stephane Bisson) * Tpt (Tpt) * Anomie (Brad Jorsch) * Joe (Giuseppe Lavagetto) * Legoktm (Kunal Mehta) * Aaron (Aaron Schulz) * mobrovac (Marko Obrovac) And everyone else I forgot to mention, thank you! -- Timo

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2018