Wikitech-l August 2019

wikitech-l@lists.wikimedia.org

50 participants
61 discussions

Database dumps
by Byrial Jensen 17 Apr '25

17 Apr '25

Until some weeks ago http://dumps.wikimedia.org/backup-index.html used to show 4 dumps in progress at the same time. That meant that new database dumps normally was available within about 3 weeks for all databases except for enwiki and maybe dewiki where the dump process due to size took longer time. However the 4 dumps processes at one time become 3 some weeks ago. And after massive failures at June 4, only one dump has been in progress at the same time. So at the current speed it will take several months to come thru all dumps. Is it possible to speed up the process again using several dump processes at the same time? Thank you, Byrial

3 2

User-Agent:
by Domas Mituzas 17 Apr '25

17 Apr '25

Hi! from now on specific per-bot/per-software/per-client User-Agent header is mandatory for contacting Wikimedia sites. Domas

19 61

Acquiring list of templates including external links
by Takashi OTA 17 Apr '25

17 Apr '25

Hoi, This is an inquiry from my friend in academia, researching about Wikipedia. He would like to know whether there's a way to acquire a list of templates including external links. Here are some examples including external links. https://ja.wikipedia.org/wiki/Template:JOI/doc https://ja.wikipedia.org/wiki/Template:Twitter/doc Such links are stored in externallinks.sql.gz, in an expanded form. When you want to check increase/decrease of linked domains in chronological order through edit history, you have to check pages-meta-history1.xml etc. In a such case, traditional links and links by templates are mixed, Therefore, the latter ones (links by templates) should be expanded to traditional link forms. Sorry if what I am saying does not make sense. Thanks in advance, --Takashi Ota [[U:Takot]]

13 24

EBNF grammar project status?
by Steve Bennett 01 Apr '25

01 Apr '25

What's the status of the project to create a grammar for Wikitext in EBNF? There are two pages: http://meta.wikimedia.org/wiki/Wikitext_Metasyntax http://www.mediawiki.org/wiki/Markup_spec Nothing seems to have happened since January this year. Also the comments on the latter page seem to indicate a lack of clear goal: is this just a fun project, is it to improve the existing parser, or is it to facilititate a new parser? It's obviously a lot of work, so it needs to be of clear benefit. Brion requested the grammar IIRC (and there's a comment to that effect at http://bugzilla.wikimedia.org/show_bug.cgi?id=7 ), so I'm wondering what became of it. Is there still a goal of replacing the parser? Or is there some alternative plan? Steve

26 217

Missing Section Headings
by Marc Riddell 13 Sep '24

13 Sep '24

Hello, I have been a WP editor since 2006. I hope you can help me. For some reason I no longer have Section Heading titles showing in the Articles. This is true of all Headings including the one that carries the Article subject's name. When there is a Table of Contents, it appears fine and, when I click on a particular Section, it goes to that Section, but all that is there is a straight line separating the Sections. There is also no button to edit a Section. If I edit the page and remove the "== ==" markers from the Section Titles, the Title then shows up, but not as a Section Heading. Also, I don't have any Date separators on my Want List. This started 2 days ago. Any thoughts? Thanks, Marc Riddell [[User:Michael David]]

10 11

How to make commits from mirrored Wikimedia repos show up on your GitHub profile
by Bartosz Dziewoński 28 Jun '24

28 Jun '24

I know it has been annoying a couple of people other than me, so now that I've learned how to make it work I'll share the knowledge here. tl;dr: Star the repositories. No, seriously. (And yes, you need to star each extension repo separately.) (Is there a place on mw.org to put this tidbit on?) ------- Forwarded message ------- From: "Brian Levine" <support(a)github.com> (GitHub Staff) To: matma.rex(a)gmail.com Cc: Subject: Re: Commits in mirrored repositories not showing up on my profile Date: Tue, 09 Jul 2013 06:47:19 +0200 Hi Bartosz In order to link your commits to your GitHub account, you need to have some association with the repository other than authoring the commit. Usually, having push access gives you that connection. In this case, you don't have push permission, so we don't link you to the commit. The easy solution here is for you to star the repository. If you star it - along with the other repositories that are giving you this problem - we'll see that you're connected to the repository and you'll get contribution credit for those commits. Cheers Brian -- Matma Rex

3 3

Research FAQ gets a facelift
by Dario Taraborelli 25 Jun '24

25 Jun '24

We just released a new version of Research:FAQ on Meta [1], significantly expanded and updated, to make our processes at WMF more transparent and to meet an explicit FDC request to clarify the role and responsibilities of individual teams involved in research across the organization. The previous version – written from the perspective of the (now inactive) Research:Committee, and mostly obsolete since the release of WMF's open access policy [2] – can still be found here [3]. Comments and bold edits to the new version of the document are welcome. For any question or concern, you can drop me a line or ping my username on-wiki. Thanks, Dario [1] https://meta.wikimedia.org/wiki/Research:FAQ [2] https://wikimediafoundation.org/wiki/Open_access_policy [3] https://meta.wikimedia.org/w/index.php?title=Research:FAQ&oldid=15176953 *Dario Taraborelli *Head of Research, Wikimedia Foundation wikimediafoundation.org • nitens.org • @readermeter <http://twitter.com/readermeter>

2 1

bluejeans
by Jeremy Baron 22 Feb '23

22 Feb '23

Hi, On Tue, Mar 1, 2016 at 3:36 PM, David Strine <dstrine(a)wikimedia.org> wrote: > We will be holding this brownbag in 25 minutes. The Bluejeans link has > changed: > > https://bluejeans.com/396234560 I'm not familiar with bluejeans and maybe have missed a transition because I wasn't paying enough attention. is this some kind of experiment? have all meetings transitioned to this service? anyway, my immediate question at the moment is how do you join without sharing your microphone and camera? am I correct thinking that this is an entirely proprietary stack that's neither gratis nor libre and has no on-premise (not cloud) hosting option? are we paying for this? -Jeremy

9 16

New DB_REPLICA constant; DB_SLAVE deprecated
by Aaron Schulz 04 Jul '20

04 Jul '20

As of 950cf6016c, the mediawiki/core repo was updated to use DB_REPLICA instead of DB_SLAVE, with the old constant left as an alias. This is part of a string of commits that cleaned up the mixed use of "replica" and "slave" by sticking to the former. Extensions have not been mass converted. Please use the new constant in any new code. The word "replica" is a bit more indicative of a broader range of DB setups*, is used by a range of large companies**, and is more neutral in connotations. Drupal and Django made similar updates (even replacing the word "master"): * https://www.drupal.org/node/2275877 * https://github.com/django/django/pull/2692/files & https://github.com/django/django/commit/beec05686ccc3bee8461f9a5a02c607a023… I don't plan on doing anything to DB_MASTER, since it seems fine by itself, like "master copy", "master tape" or "master key". This is analogous to a master RDBMs database. Even multi-master RDBMs systems tend to have a stronger consistency than classic RDBMs slave servers, and present themselves as one logical "master" or "authoritative" copy. Even in it's personified form, a "master" database can readily be thought of as analogous to "controller", "governer", "ruler", lead "officer", or such.** * clusters using two-phase commit, galera using certification-based replication, multi-master circular replication, ect... ** https://en.wikipedia.org/wiki/Master/slave_(technology)#Appropriateness_of_… *** http://www.merriam-webster.com/dictionary/master?utm_campaign=sd&utm_medium… -- -Aaron

4 3

Several primary database masters switchovers scheduled (read-only required)
by Manuel Arostegui 26 Sep '19

26 Sep '19

Hello, The following primary database masters will be switched over during the next few weeks (more details at https://phabricator.wikimedia.org/T230788): Impact: *Writes will be blocked* *Reads will remain unaffected* These are the concrete days, hours and affected wikis: * s8: 10th Sept from 05:00-05:30 UTC. The affected wiki is: wikidatawiki - tracking task: T230762 * s2: 17th Sept from 05:00-05:30 UTC. The list of affected wikis is at: https://raw.githubusercontent.com/wikimedia/operations-mediawiki-config/mas… - tracking task: T230785 * s3: 24th Sept from 05:00-05:30 UTC. The list of affected wikis is at: https://raw.githubusercontent.com/wikimedia/operations-mediawiki-config/mas… - tracking task: T230783 * s4: 26th Sept from 05:00-05:30 UTC. The affected wiki is: commonswiki - tracking task: T230784 If everything goes well, we do not expect to use those 30 minutes of read-only and rather just a few minutes. We will email send an email the day of each failover before and after it is done. Sorry for any inconvenience this might cause.

2 9

m1 (etherpad, bacula, librenms, rt, racktables) db primary master switchover: 10th Sept 16:00 UTC
by Manuel Arostegui 10 Sep '19

10 Sep '19

Hello, The current primary master for m1 (db1063), which is mostly for internal services + etherpad isn't in a great healthy status: it is an old host, which needs to be decommissioned and which is having disks failing pretty much every week (plus disks on predictive failure). We have decided to fail it over one of the newer hosts, db1135: https://phabricator.wikimedia.org/T231403 We have scheduled this switchover for: Tuesday 10th September at 16:00 UTC This failover should be rather quick and would only take a few seconds (while we re-load the haproxy) - during those few seconds, the following services will be on read-only: bacula etherpadlite librenms puppet racktables rt Communication will happen at #wikimedia-operations If you are around at that time and want to help with the monitoring, please join us! Thanks

2 7

[Tech Talks] Sept 4, 2019, 6 PM UTC, Documenting Wikimedia technical projects
by Sarah R 03 Sep '19

03 Sep '19

Hi Everyone, It's time for Wikimedia Tech Talks 2019 Episode 7! This talk will take place *Sept 4, 2019 at 6PM UTC*. *Topic:* Documenting Wikimedia technical projects *Speaker:* Sarah R. Rodlund, Technical Writer -- Developer Advocacy *Summary: * This talk will discuss what technical writers do and why they are critical members of the Wikimedia technical community. You will learn more about the skills needed to be a technical writer and how to build these skills by participating on Wikimedia and other open source projects. The talk will also cover some ongoing initiatives to improve technical documentation for Wikimedia projects. *YouTube stream for viewers:* https://www.youtube.com/watch?v=GKvyk4VAPb0 During the live talk, you are invited to join the discussion on IRC at #wikimedia-office You can watch past Tech Talks here: https://www.mediawiki.org/wiki/Tech_talks If you are interested in giving your own tech talk, you can learn more here: https://www.mediawiki.org/wiki/Project:Calendar/How_to_schedule_an_event#Te… Many kindnesses, Sarah R. Rodlund Technical Writer, Developer Advocacy <https://meta.wikimedia.org/wiki/Developer_Advocacy> srodlund(a)wikimedia.org

1 1

Upcoming Search Platform Office Hours—September 4th
by Trey Jones 03 Sep '19

03 Sep '19

The Search Platform Team <https://www.mediawiki.org/wiki/Wikimedia_Search_Platform> usually holds office hours the first Wednesday of each month. Come talk to us about anything related to Wikimedia search! Feel free to add your items to the Etherpad Agenda for the next meeting. Details for our next meeting: Date: Wednesday, Sep 4th, 2019 Time: 15:00-16:00 GMT / 08:00-9:00 PDT / 11:00-12:00 EDT / 17:00-18:00 CEST Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link: https://meet.google.com/vyc-jvgq-dww Hope to talk to you in a week! Trey Jones Sr. Software Engineer, Search Platform Wikimedia Foundation UTC-4 / EDT

1 1

m5 (Wikitech) primary database master switchover
by Manuel Arostegui 03 Sep '19

03 Sep '19

Hello, We are going to switchover m5 (wikitech) primary database master on *Tuesday 3rd September at 13:00 UTC *https://phabricator.wikimedia.org/T229657 This switchover will also affect some Cloud services that will be handled on a different announcement. The current master (db1073) is an old and out of warranty hosts with several disks on predictive failure, it needs to be decommissioned. We will be failing over db1073 to db1133, a newer and more powerful host. We expect this to take around 15 minutes (or just a few seconds if everything goes fine). *Impact: * *- Writes will be blocked during the switchover* *- Reads will remain unaffected.* Communication will happen on #wikimedia-operations IRC channel. Thanks Manuel.

1 2

[Train] 1.34.0-wmf.20 status update
by Željko Filipin 02 Sep '19

02 Sep '19

The 1.34.0-wmf.20 version of MediaWiki is blocked. https://phabricator.wikimedia.org/T220745 The new version is not deployed anywhere yet. It will not be deployed until these issues are resolved: * ServiceContainer.php: Circular dependency when creating MobileFrontend service "AMC.UserMode > AMC.Manager > FeaturesManager > UserModes > AMC.UserMode" https://phabricator.wikimedia.org/T231014 * DefaultPreferencesFactory.php: Global default '' is invalid for field incubatortestwiki-code https://phabricator.wikimedia.org/T231029 * /w/api.php... ErrorException from line 0 of : PHP Notice: Unable to unserialize ... Size of serialized string ... exceeds max https://phabricator.wikimedia.org/T231071 * L10n cache is completely broken https://phabricator.wikimedia.org/T231183 Once these issues are resolved train can start. Additionally, I've found this errors in logs, but since they are not new, they are not blocking the train. * Assert.php: Bad value for parameter $responses: must have as many responses as requests https://phabricator.wikimedia.org/T231023 * Assert.php: Bad value for parameter $oldContent: must be a TextContent|null https://phabricator.wikimedia.org/T231084 * WikibaseClient.php: PHP Notice: Undefined index: https://phabricator.wikimedia.org/T231089 Thank you for your help resolving these issues! -- Your humble train toiler, Željko

2 4

Declaring methods final in classes
by Aryeh Gregor 01 Sep '19

01 Sep '19

I see that in some classes, like WANObjectCache, most methods are declared final. Why is this? Is it an attempt to optimize? The problem is that PHPUnit mocks can't touch final methods. Any ->method() calls that try to do anything to them silently do nothing. This makes writing tests harder. If we really want these methods to be marked final for some reason, the workaround for PHP is to make an interface that has all the desired methods, have the class implement the interface, and make type hints all refer to the interface instead of the class. But if there's no good reason to declare the methods final to begin with, it's simplest to just drop it.

9 23

Limits to configuration changes Meta page
by Martin Urbanec 01 Sep '19

01 Sep '19

Hello all, Meta Wiki has page called Limits to configuration changes < https://meta.wikimedia.org/wiki/Limits_to_configuration_changes> that is currently merely a list of declined requests. I believe it should be explaining why declining happens rather than document what happened, since the later is already documented in Phabricator. As such, I've decided to update it, since I'm active in that field. Since it is more inspired by the current version and not based on it (it's a complete rewrite). I would like to invite everyone to read the update at https://meta.wikimedia.org/wiki/Limits_to_configuration_changes/Update_2019, and join the discussion at https://meta.wikimedia.org/wiki/Talk:Limits_to_configuration_changes, or also to edit the update itself. Once we agree it's good to go, the text can be changed. Martin Urbanec

1 0

Timeless grant final report
by Isarra Yos 31 Aug '19

31 Aug '19

Hey all, the Timeless grant has wrapped up and I've submitted the final report for the project. Please check it out if you're interested: https://meta.wikimedia.org/wiki/Grants:Project/Timeless/Post-deployment_sup… A couple of things to highlight: * The associated learning pattern is likely going to be relevant to a lot of us: https://meta.wikimedia.org/wiki/Learning_patterns/Developing_intercompatibl… * Part of the purpose of the grant was to investigate into what we would actually need from a new skin on Wikimedia projects, should anyone decide to go that route. Here's what we found: > All we really need from a new skin is for everything that went wrong > with Minerva and Timeless, as far as readers and editors are concerned, > to not happen this time: > > * We need gadgets to work. > * We need content to display properly, and tools to be available. > * We need to properly reuse core interfaces so things remain consistent. > * We need to not distract users with unnecessary colours and affordances > that aren't relevant to whatever they're doing, and provide options > that suit their differing situations and needs. > * We need to not do dumb things, like reinvent random wheels or ignore > standard hooks and DOM expectations. I don't know if that's all that likely to help anyone, but... yeah. Anyway, I'll still be working on maintenance and some feature development in a volunteer capacity, mind you, so as always if you run into issues or whatever, please file them on phabricator or come yell at me! I just make absolutely no promises whatsoever as to when I (or whoever) will actually deal with it, but it's still really helpful to at least have them tracked. https://phabricator.wikimedia.org/tag/timeless/ (See, it's even sort of organised! I even renamed the 'Aaaaaaa' column!) -I

1 0

Oauth non working properly
by valdelli＠gmail.com 31 Aug '19

31 Aug '19

Hi all, I am doing an edit-a-thon and I have a strange behaviour with an user. He allows the use of Oauth for the Dashboard but he does the login and he is not redirected to the Dashboard. The page remains empty even if he is logged. Do you have an idea? Kind regards Ilario Valdelli

2 3

Code of Conduct committee meeting at Wikimania
by Amir Sarabadani 31 Aug '19

31 Aug '19

Hello, The CoC committee had a meeting with four members out of five (Lucie, Tony, Tonina, and Amir were at Wikimania and MusikAnimal wasn't present). These notes are the public outcomes of the meeting. * The committee chose Amir as the chair (Amir requested to be the table but it got rejected) * The committee is currently composing the anonymized report of its cases and will be published soon. * The committee soon will decide on the two amendments currently under discussion. Feel free to comment on these two: ** Public logging of bans: https://www.mediawiki.org/wiki/Topic:Uikcu1emvmw6e4z8 ** Modifications on the appeal process: https://www.mediawiki.org/wiki/Topic:Uay0vz6no8ayac3m * The committee is willing to introduce a new internal role which for lack of better term, we call it secretary for now. The role's main responsibility is to help the committee on organizational bookkeeping. Like reports, documentation of cases, etc. * At the end, the committee wants to remind people that its role is reactionary and we won't interfere when there's no report. If you see a behaviour you find unacceptable, stand up and send us a report and don't wait for the committee to jump in. Regards, Amir on behalf of the committee

3 2

Wikimedia production excellence (July 2019)
by Krinkle 30 Aug '19

30 Aug '19

📘 Read on Phabricator at https://phabricator.wikimedia.org/phame/post/view/164 ------- How’re we doing on that strive for operational excellence? Read this first anniversary edition to find out! #### 📊 Month in numbers - 5 documented incidents. [1] - 53 new Wikimedia-prod-error reports. [2] - 44 closed Wikimedia-prod-error reports. [3] - 218 currently open Wikimedia-prod-error reports in total. [4] The number of recorded incidents over the past month, at five, is equal to the median number of incidents per month (2016-2019). – Explore the data at https://codepen.io/Krinkle/full/wbYMZK To read more about these incidents, their investigations, and pending actionables; check https://wikitech.wikimedia.org/wiki/Incident_documentation#2019 ------- #### *️⃣One year of Excellent adventures! Exactly one year ago [5] this periodical started to provide regular insights on production stability. The idea was to shorten the feedback cycle between deployment of code that leads to fatal errors and the discovery of those errors. This allows more people to find reports earlier, which (hopefully) prevents them from sneaking into a growing pile of “normal” errors. 576 reports were created between 15 July 2018 and 31 July 2019 (tagged Wikimedia-prod-error). 425 reports got closed over that same time period. ------- #### 📉 Outstanding reports Take a look at the workboard and look for tasks that might need your help. The workboard lists error reports, grouped by the month in which they were first observed. → https://phabricator.wikimedia.org/tag/wikimedia-production-error/ Or help someone who already started with their patch: → https://phabricator.wikimedia.org/maniphest/query/pzVPXPeMfRIz/#R Breakdown of recent months (past two weeks not included): - November: 1 report left (unchanged). ⚠️ - December: 3 reports left (unchanged). ⚠️ - January: 1 report left (unchanged). - February: 2 reports left (unchanged). - March: 4 reports left (unchanged). - April: 10 of 14 reports left (unchanged). - May: 2 reports got fixed! (4 of 10 reports left). ✨ - June: 2 reports got fixed! (9 of 11 reports left). ✨️ - July: 18 new reports from last month remain unsolved. ------- #### 🎉 Thanks! Thank you to Aaron Schulz, Ariel Glenn, Brad Jorsch, CScott, Daimona, David Barratt, David Causse, EBernhardson, Gergő Tisza, James Forrester, Jeena Huneidi, MarcoAurelio, Roan Kattouw, Stephane Bisson, Tchanders, Tim Starling, Urbanecm; and everyone else who helped by finding, investigating, or resolving error reports in Wikimedia production. Thanks! Until next time, – Timo Tijhof ------- Footnotes: [1] Incidents. – https://wikitech.wikimedia.org/wiki/Special:PrefixIndex?prefix=Incident+doc… [2] Tasks created. – https://phabricator.wikimedia.org/maniphest/query/9m.NNvBAvuHF/#R [3] Tasks closed. – https://phabricator.wikimedia.org/maniphest/query/6sekLiUhHCmq/#R [4] Open tasks. https://phabricator.wikimedia.org/maniphest/query/47MGY8BUDvRD/#R [5] The initial e-mail. – https://lists.wikimedia.org/pipermail/wikitech-l/2018-July/090363.html

1 0

Scrum of scrums/2019-08-28
by Željko Filipin 29 Aug '19

29 Aug '19

Hi, for HTML version go to https://www.mediawiki.org/wiki/Scrum_of_scrums/2019-08-28 Željko -- = 2019-08-28 = == Callouts == * Release Engineering ** REMINDER: We're at 1.34.0-wmf.20 this week. The last branch for this release will be wmf.25 on 30 September. Teams who want to ship things for MW 1.34 should land them now. ** Search Platform blocked (by Core Platform?): RecentChange support for SDC: https://phabricator.wikimedia.org/T230862 == Product == === Readers === ==== iOS native app ==== * Updates: ** Development on 6.4 release https://phabricator.wikimedia.org/tag/ios-app-v6.4-seamonkey-on-a-hoverboar… ***iOS13/iPadOS updates ***New loading screen ***Bug fixes ***History & Diffs pushed off to the next release ==== Android native app ==== * Updates: ** Minor release now in production: optimized page loading speed and efficiency. ** Starting work on Suggested Edits v3 (design updates to user contributions screen). ** Continuing to profile and test integration with mobile-html endpoint. ==== Readers Web ==== * Updates: ** Summary: continuing advanced mode. ** Responsive website (MinervaNeue / MobileFrontend / Portals): *** Desktop refresh: **** Light technical planning. *** [ https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions Advanced mode]: **** [https://phabricator.wikimedia.org/T231358 <nowiki>File '/srv/mediawiki/php-1.34.0-wmf.20/skins/MinervaNeue/resources/resources/skins.minerva.personalMenu.icons/userSandbox.svg' does not exist</nowiki>] **** [https://phabricator.wikimedia.org/T231014 <nowiki>ServiceContainer.php: Circular dependency when creating MobileFrontend service "AMC.UserMode > AMC.Manager > FeaturesManager > UserModes > AMC.UserMode"</nowiki>] **** [https://phabricator.wikimedia.org/T231363 <nowiki>Contributions icon is missing from main menu in non-AMC mode for logged in users</nowiki>] **** [https://phabricator.wikimedia.org/T230033 <nowiki>Touch targets not large enough</nowiki>] **** [https://phabricator.wikimedia.org/T231057 <nowiki>Browser tests failing now AMC Outreach drawer is enabled</nowiki>] **** [https://phabricator.wikimedia.org/T230927 <nowiki>AMC Outreach Drawer encouraging me to turn on advanced mode sends me from Special:Homepage to my user page</nowiki>] **** [https://phabricator.wikimedia.org/T229295 <nowiki>Cleanup AMC feature flagging</nowiki>] **** [https://phabricator.wikimedia.org/T220016 <nowiki>Create, and deploy working MobileWebUIActionsTracking schema</nowiki>] **** [https://phabricator.wikimedia.org/T229447 <nowiki>Reduce spacing of icons in mobile header</nowiki>] **** [https://phabricator.wikimedia.org/T229440 <nowiki>Fix icon sizing by updating all 24x24 icons to 20x20 icons</nowiki>] *** Miscellaneous: **** [https://phabricator.wikimedia.org/T230274 <nowiki>Excessive paragraph padding in MF editor save dialog</nowiki>] **** [https://phabricator.wikimedia.org/T217826 <nowiki>Mobile VE doesn't use DB title for RESTBase load, resulting in a 301 redirect</nowiki>] **** [https://phabricator.wikimedia.org/T220849 <nowiki>Separate View code from SearchOverlay</nowiki>] **** [https://phabricator.wikimedia.org/T206265 <nowiki>Bug: Section edit links should be hidden for blocked users</nowiki>] ==== Product Infrastructure ==== * Updates: ** Pagelib *** CSS now minimized. Deployed yesterday *** Experimenting with pagelib to improve mobile-html pageload https://phabricator.wikimedia.org/T231201 ** Mobile-html *** Fixed srcset in media-list endpoint *** WIP: updating footer style ** Maps *** Investigated OSM DB bug, but results inconclusive ** Machine vision middleware *** Could be (optimistically) code-complete by EOW for DBA, Performance pre-deployment reviews *** Planning JS frontend integration ** Data Engineering *** Specified identifiers for event platform clients *** Specified sampling functions for event platform clients ==== Structured Data ==== * Updates: ** working on getting structured data in campaigns ready for WLM ==== Parsing ==== * Updates: ** Parsoid/PHP testing: *** Parser tests almost 100% green now (barring some insignificant key-value ordering differences in JSON blobs which we need to normalize to reduce this false positives). *** We have got our first runs of Parsoid/PHP round trip tests running scandium. We still have some crashers to fix and more bugs to fix from that initial test run. ** Performance-wise, after Tim's initial work tweaking some things in Parsoid, here is a rough performance update for test runs on scandium. We need real perf. benchmarking to get a more robust sense of where we are. *** Parsoid/PHP w2html: **** On large pages (like en:Barack_Obama), 1.5x slower than Parsoid/JS. But, most of that seems to be GC issues (both PHP7.2 and Parsoid/PHP). See https://phabricator.wikimedia.org/T230861 for details **** On medium and smaller pages, as fast as or upto 2x faster than Parsoid/JS in some cases which is a pretty good result. *** Parsoid/PHP html2wt (impacts VE and CX): **** On large DOMs (like en:Barack_Obama), >2x slower than Parsoid/JS. But, on slightly smaller DOMs (like en:Berlin), < 10% slower. So, we seem to be getting non-linear slowdown as the pages get larger. **** Needs additional testing on medium and smaller sized pages. **** Note that in production usage, we will be using a more efficient version of html -> wt. But, this slowdown is still a reliable indicator of where we need to focus our performance work. ** Other Parsoid/PHP updates: *** Working with core platform team on RESTBase - Parsoid/PHP integration testing. == Technology == === Analytics === * Blocking: ** Search Platform - https://phabricator.wikimedia.org/T229882 === Fundraising Tech === * Updates: ** Large-scale testing of India payments form ** Prepping new CiviCRM point upgrade with more of our local hacks upstreamed or turned into proper extensions: https://phabricator.wikimedia.org/T228826 ** Importing more donor data into CiviCRM: https://phabricator.wikimedia.org/T228715 ** Fixes for one-time->recurring conversion ask: https://phabricator.wikimedia.org/T228162 ** Deploying new Eventlogging-based pipeline for landing page and CentralNotice data *** Fixing issues with sending events from both sources: https://phabricator.wikimedia.org/T231406, https://phabricator.wikimedia.org/T231404 ** Payments form layout and text changes: https://phabricator.wikimedia.org/T169795, https://phabricator.wikimedia.org/T224063 ** Almost ready to deploy python3 upgrade to fundraising python tools: https://phabricator.wikimedia.org/T197805 === Core Platform === * Blocked by: ** Site Reliability Engineering: Investigate Kask request latency https://phabricator.wikimedia.org/T229697 * Blocking: ** Search Platform: RecentChange support for SDC: https://phabricator.wikimedia.org/T230862 ** Release Engineering: MediaWiki installer silently ignores invalid extensions https://phabricator.wikimedia.org/T225512 ** Performance: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/394430/ https://phabricator.wikimedia.org/T230800 (triaged by Daniel and Brad a week ago?) * Updates: ** Kask performance issue ** Parsoid/PHP last routes ** Core REST API routes in MediaWiki ** Action API integration testing === Engineering Productivity === ==== Performance ==== * Blocked by: ** WMDE/Wikidata: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/497537/ ** Core Platform: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/394430/ https://phabricator.wikimedia.org/T230800 (triaged by Daniel and Brad a week ago?) * Updates: ** resourceloader: New Grafana dashboard providing insight into JS startup cost for extensions. – https://grafana.wikimedia.org/d/BvWJlaDWk/startup-module-size ==== Release Engineering ==== * Blocked by: ** Core Platform Team: MediaWiki installer silently ignores invalid extensions https://phabricator.wikimedia.org/T225512 * Updates: ** REMINDER: We're at 1.34.0-wmf.20 this week. The last branch for this release will be wmf.25 on 30 September. Teams who want to ship things for MW 1.34 should land them now. ** Train Health *** Last week: 1.34.0-wmf.19 - https://phabricator.wikimedia.org/T220744 - some problems, but deployed on time *** This week: 1.34.0-wmf.20 - https://phabricator.wikimedia.org/T220745 *** Next week: 1.34.0-wmf.21 - https://phabricator.wikimedia.org/T220746 === Search Platform === * Blocked by: ** Core Platform: RecentChange support for SDC: https://phabricator.wikimedia.org/T230862 ** Analytics: https://phabricator.wikimedia.org/T229882 * Updates: ** Support for Wikidata Query Service will be limited for the foreseeable future; hiring is in progress ** Cloudelastic import still going on, causing some queues to grow quicker than expected https://phabricator.wikimedia.org/T231194 ** Still reindexing some wikis to enable the new create_timestamp sort criteria https://phabricator.wikimedia.org/T230990 ** Adapted search/analytics data pipelines to use swift to deploy data (did you mean suggestions and pageview ranking signal) to production https://phabricator.wikimedia.org/T227364 ** We evaluated druid+turnilo+superset for search metrics going forward and found it to be reasonable; built a superset dashboard for search satisfaction 'did you mean' metrics https://phabricator.wikimedia.org/T229268 ** Worked on documenting the limitations of the current search syntax https://phabricator.wikimedia.org/T228925 ** Added support to haswbstatement:* syntax: https://phabricator.wikimedia.org/T230175 ** Fixed issue with case sensitivity of deepcat searches: https://phabricator.wikimedia.org/T228633 ** SDC query test server: http://sdcquery.wmflabs.org/ - not updated yet! ** wdqs1009 restored to its role as auto-deploy test server: https://phabricator.wikimedia.org/T230244 ** WDQS Updater performance improvements code complete, starting testing: https://phabricator.wikimedia.org/T212826 ** Working on fixing categories delete issue: https://phabricator.wikimedia.org/T228348 === Security === * Updates: ** Backlog *** Security Review for MediaWiki REST API, assigned to Sam, https://phabricator.wikimedia.org/T230140 *** John to explore funding of 3rd party audits this quarter, stalled - https://phabricator.wikimedia.org/T155537, https://phabricator.wikimedia.org/T156960, https://phabricator.wikimedia.org/T148246, https://phabricator.wikimedia.org/T187846 ** Active *** Security review of Ex:DoubleWiki, in-progress - https://phabricator.wikimedia.org/T131199 *** Parsoid-PHP, moved to active, additional review by Sam, in-progress - https://phabricator.wikimedia.org/T227209 *** Security review of preact 8.4.2, Scott did a very simple assessment, unassigned, stalled https://phabricator.wikimedia.org/T227726 *** Page Content Service route /page/mobile-html, starting - https://phabricator.wikimedia.org/T227114 *** Security review of WebAuthn library dependencies, in-progress - https://phabricator.wikimedia.org/T227244 *** Planet wikimedia - assigned to Michal Anna, stalled - https://phabricator.wikimedia.org/T207246 ** Waiting *** Labs db/sanitarium and maintain-views.yaml audits, assigned to James F, stalled - https://phabricator.wikimedia.org/T169097, https://phabricator.wikimedia.org/T103011 ** Frozen (delayed indefinitely) *** Audiences growth team emails concept review, stalled - Jen contacted to close - https://phabricator.wikimedia.org/T220242 *** Banner preview, stalled - https://phabricator.wikimedia.org/T230176 ** Closing Soon / Closed *** Doublewiki (old), Scott to resolve this week, in-progress - can Jen contact to close? - https://phabricator.wikimedia.org/T131199 === Site Reliability Engineering === * Blocking: ** Core Platform Team, Investigate Kask request latency https://phabricator.wikimedia.org/T229697 == Wikimedia DE == === Wikidata === * Blocking: ** Performance - WMDE/Wikidata: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/497537/ == SoS Meeting Bookkeeping == * Updates: ** Don't forget to copy "blocked" section from your team to the team that's blocking you "blocking" section

1 0

Call for contributors, A survey of wiki sites
by Houcemeddine A. Turki 29 Aug '19

29 Aug '19

Dear all, I thank you for your efforts. I am managing to begin writing a survey about wiki sites and publish it in an appropriate journal... This survey will involve the software used (Mediawiki or other), the size, reference support, topic... I am looking for contributors to this project. Anyone who has published two papers about wikis in a research journal is invited to join the initiative. Yours Sincerely, Houcemeddine Turki (he/him) Medical Student, Faculty of Medicine of Sfax, University of Sfax, Tunisia Undergraduate Researcher, UR12SP36 GLAM and Education Coordinator, Wikimedia TN User Group Member, WikiResearch Tunisia Member, Wiki Project Med Member, WikiIndaba Steering Committee Member, Wikimedia and Library User Group Steering Committee Co-Founder, WikiLingua Maghreb Founder, TunSci _____________________ +21629499418

1 0

Discovery Weekly Update for the week starting 2019-08-05
by Chris Koerner 27 Aug '19

27 Aug '19

Hello, This is the weekly update from the Search Platform team for the week starting 2019-08-05. As always, feedback and questions are welcome. == Discussions == === Search === * Erik discovered that Prometheus was not collecting cloudelastic metrics...so, he defined cloudelastic as a cluster in hieradata [0] * The team worked on fixing documentation of boolean operators [1] and you can read about it. [2] * We needed to help get cloudelastic working with the request streams that we need, so, we fixed LVS IPv6 addresses [3] * There was a spike of errors on eventlogging events that needed to be addressed [4] * We evaluated druid+turnilo+superset for search metrics going forward and found it to be reasonable; so, now we'll build a superset dashboard for search satisfaction 'did you mean' metrics [5] * Erik finished yup the work to initialize CirrusSearch on cloudelastic [6] * *borked* — Icinga check defined from LVS configuration for cloudelastic and now fixed- cloudelastic checks for all ports have been generated on icinga. However, only IPv4 checks were generated and this is Ok for now. If there's need to generate IPv6 checks, please reopen this task [7] * David found an issue where labweb mw hosts were unable to contact any of the elasticsearch clusters and fixed it [8] === Wikidata Query Service === * Stas added support for SDC MediaInfo URIs in WDQS [9] * Stas moved DCAT-AP namespace to https://dcatap.wmflabs.org/ [10] [0] https://phabricator.wikimedia.org/T229937 [1] https://phabricator.wikimedia.org/T228925 [2] https://www.mediawiki.org/wiki/Help:CirrusSearch/Logical_operators [3] https://phabricator.wikimedia.org/T224324 [4] https://phabricator.wikimedia.org/T229614 [5] https://phabricator.wikimedia.org/T229268 [6] https://phabricator.wikimedia.org/T220625 [7] https://phabricator.wikimedia.org/T229621 [8] https://phabricator.wikimedia.org/T230994 [9] https://phabricator.wikimedia.org/T229608 [10] https://phabricator.wikimedia.org/T228297 ---- Subscribe to receive on-wiki (or opt-in email) notifications of the Discovery weekly update. https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner (he/him) Community Relations Specialist Wikimedia Foundation

1 0

Phabricator monthly statistics - 2019-07
by communitymetrics＠wikimedia.org 27 Aug '19

27 Aug '19

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2019-07): 290 Active Maniphest users (any activity) in (2019-07): 967 Task authors in (2019-07): 521 Users who have closed tasks in (2019-07): 293 Projects which had at least one task moved from one column to another on their workboard in (2019-07): 312 Tasks created in (2019-07): 2573 Tasks closed in (2019-07): 2403 Open and stalled tasks in total: 42433 Median age in days of open tasks by priority: Unbreak now: 2 Needs Triage: 522 High: 944 Normal: 1139 Low: 1552 Lowest: 1531 (How long tasks have been open, not how long they have had that priority) Active Differential users (any activity) in (2019-07): 12 To see the names of the most active task authors: * Go to https://wikimedia.biterg.io/ * Choose "Phabricator > Overview" from the top bar * Adjust the time frame in the upper right corner to your needs * See the author names in the "Submitters" panel TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on phab1003 at Tue Aug 27 14:32:03 UTC 2019)

3 2

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l August 2019