May 2023 - Wikitech-l - lists.wikimedia.org

Database dumps
by Byrial Jensen 17 Apr '25

17 Apr '25

Until some weeks ago http://dumps.wikimedia.org/backup-index.html used to show 4 dumps in progress at the same time. That meant that new database dumps normally was available within about 3 weeks for all databases except for enwiki and maybe dewiki where the dump process due to size took longer time. However the 4 dumps processes at one time become 3 some weeks ago. And after massive failures at June 4, only one dump has been in progress at the same time. So at the current speed it will take several months to come thru all dumps. Is it possible to speed up the process again using several dump processes at the same time? Thank you, Byrial

3 2

User-Agent:
by Domas Mituzas 17 Apr '25

17 Apr '25

Hi! from now on specific per-bot/per-software/per-client User-Agent header is mandatory for contacting Wikimedia sites. Domas

19 61

Acquiring list of templates including external links
by Takashi OTA 17 Apr '25

17 Apr '25

Hoi, This is an inquiry from my friend in academia, researching about Wikipedia. He would like to know whether there's a way to acquire a list of templates including external links. Here are some examples including external links. https://ja.wikipedia.org/wiki/Template:JOI/doc https://ja.wikipedia.org/wiki/Template:Twitter/doc Such links are stored in externallinks.sql.gz, in an expanded form. When you want to check increase/decrease of linked domains in chronological order through edit history, you have to check pages-meta-history1.xml etc. In a such case, traditional links and links by templates are mixed, Therefore, the latter ones (links by templates) should be expanded to traditional link forms. Sorry if what I am saying does not make sense. Thanks in advance, --Takashi Ota [[U:Takot]]

13 24

EBNF grammar project status?
by Steve Bennett 01 Apr '25

01 Apr '25

What's the status of the project to create a grammar for Wikitext in EBNF? There are two pages: http://meta.wikimedia.org/wiki/Wikitext_Metasyntax http://www.mediawiki.org/wiki/Markup_spec Nothing seems to have happened since January this year. Also the comments on the latter page seem to indicate a lack of clear goal: is this just a fun project, is it to improve the existing parser, or is it to facilititate a new parser? It's obviously a lot of work, so it needs to be of clear benefit. Brion requested the grammar IIRC (and there's a comment to that effect at http://bugzilla.wikimedia.org/show_bug.cgi?id=7 ), so I'm wondering what became of it. Is there still a goal of replacing the parser? Or is there some alternative plan? Steve

26 217

Phabricator upstream shutdown
by Brian Wolff 21 Mar '25

21 Mar '25

It sounds like phabricator upstream is going away: https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_op… Just curious, are we planning to continue using it long term or move to something else? -- Brian

10 12

Missing Section Headings
by Marc Riddell 13 Sep '24

13 Sep '24

Hello, I have been a WP editor since 2006. I hope you can help me. For some reason I no longer have Section Heading titles showing in the Articles. This is true of all Headings including the one that carries the Article subject's name. When there is a Table of Contents, it appears fine and, when I click on a particular Section, it goes to that Section, but all that is there is a straight line separating the Sections. There is also no button to edit a Section. If I edit the page and remove the "== ==" markers from the Section Titles, the Title then shows up, but not as a Section Heading. Also, I don't have any Date separators on my Want List. This started 2 days ago. Any thoughts? Thanks, Marc Riddell [[User:Michael David]]

10 11

How to make commits from mirrored Wikimedia repos show up on your GitHub profile
by Bartosz Dziewoński 27 Jun '24

27 Jun '24

I know it has been annoying a couple of people other than me, so now that I've learned how to make it work I'll share the knowledge here. tl;dr: Star the repositories. No, seriously. (And yes, you need to star each extension repo separately.) (Is there a place on mw.org to put this tidbit on?) ------- Forwarded message ------- From: "Brian Levine" <support(a)github.com> (GitHub Staff) To: matma.rex(a)gmail.com Cc: Subject: Re: Commits in mirrored repositories not showing up on my profile Date: Tue, 09 Jul 2013 06:47:19 +0200 Hi Bartosz In order to link your commits to your GitHub account, you need to have some association with the repository other than authoring the commit. Usually, having push access gives you that connection. In this case, you don't have push permission, so we don't link you to the commit. The easy solution here is for you to star the repository. If you star it - along with the other repositories that are giving you this problem - we'll see that you're connected to the repository and you'll get contribution credit for those commits. Cheers Brian -- Matma Rex

3 3

Research FAQ gets a facelift
by Dario Taraborelli 25 Jun '24

25 Jun '24

We just released a new version of Research:FAQ on Meta [1], significantly expanded and updated, to make our processes at WMF more transparent and to meet an explicit FDC request to clarify the role and responsibilities of individual teams involved in research across the organization. The previous version – written from the perspective of the (now inactive) Research:Committee, and mostly obsolete since the release of WMF's open access policy [2] – can still be found here [3]. Comments and bold edits to the new version of the document are welcome. For any question or concern, you can drop me a line or ping my username on-wiki. Thanks, Dario [1] https://meta.wikimedia.org/wiki/Research:FAQ [2] https://wikimediafoundation.org/wiki/Open_access_policy [3] https://meta.wikimedia.org/w/index.php?title=Research:FAQ&oldid=15176953 *Dario Taraborelli *Head of Research, Wikimedia Foundation wikimediafoundation.org • nitens.org • @readermeter <http://twitter.com/readermeter>

2 1

Announcing: Path review board
by Brian Wolff 04 Oct '23

04 Oct '23

Yesterday there was a conversation about code review on irc and among other things, how sometimes patches can get "stuck". I had an idea for a way to improve things. I'm not sure if it is a good idea, but there's only one way to find out. So without further ado, announcing the Code Review Patch Board: https://www.mediawiki.org/wiki/Code_review/patch_board In short - each person is allowed to list one of their patches on the board that they would really like to see reviewed. You can only list one patch at a time, and it should be a patch that you have been unable to get review for for at least a week through normal means. See the page for the full list of guidelines. I encourage people to give it a try. Add a patch you wrote that you cannot get a review for. Or if you have +2 rights, try giving some love to these underloved patches. I would also love to hear feedback on the general idea as well as the current guidelines. To repeat, the url is: https://www.mediawiki.org/wiki/Code_review/patch_board Thanks, bawolff

4 5

Service Decommission Notice: Mobile Content Service - July 2023
by Adam Baso 06 Jul '23

06 Jul '23

TL;DR: The legacy Mobile Content Service is going away in July 2023. Please switch to Parsoid or another API before then to ensure service continuity. Hello World, I'm writing about a service decommission we hope to complete mid-July 2023. The service to be decommissioned is the legacy Mobile Content Service ("MCS"), which is maintained by the Wikimedia Foundation's Content Transform Team. We will be marking this service as deprecated soon. We hope that with this notice, people will have ample time to update their systems for use of other endpoints such as Parsoid [1] (n.b., MCS uses Parsoid HTML). The MCS endpoints are the ones with the relative URL path pattern /page/mobile-sections* on the Wikipedias. For examples of the URLs see the "Mobile" section on the online Swagger (OpenAPI) specification documentation with matching URLs here: https://en.wikipedia.org/api/rest_v1/#/Mobile == History == The Mobile Content Service ("MCS") is the historical aggregate service that originally provided support for the article reading experience on the Wikipedia for Android native app, as well as some other experiences. We have noticed that there are other users of the service. We are not able to determine all of the users, as it's hard to tell with confidence from the web logs. The Wikimedia Foundation had already transitioned the Wikipedia for Android and iOS apps to the newer Page Content Service ("PCS") several years ago. PCS has some similarities with MCS in terms of its mobility focus, but it also has different request-response signatures in practice. PCS, as with MCS, is intended to primarily satisfy Wikimedia Foundation-maintained user experiences only, and so this is classified with the "unstable" moniker. == Looking ahead == Generally, as noted in the lead, we recommend that folks who use MCS (or PCS, for that matter) switch over to Parsoid for accessing Wikipedia article content programmatically for the most predictable service. The HTML produced by Parsoid has a versioned specification [2] and because Parsoid is accessed regularly by a number of components across the globe tends to have fairly well cached responses. However, please note that Parsoid may be subject to stricter rate limits that can apply under certain traffic patterns. At this point, I do also want to note that in order to keep up with contemporary HTML standards, particularly those favoring accessibility and machine readability enhancements, Parsoid HTML will undergo change as we further converge parsing stacks [3]. Generally, you should expect iteration on the Parsoid HTML spec, and of course as you may have come to appreciate that the shape of HTML in practice can vary nontrivially wiki-by-wiki as practices across wikis vary. You may also want to consider Wikimedia Enterprise API options, which range from no cost to higher volume access paid options. https://meta.wikimedia.org/wiki/Wikimedia_Enterprise#Access == Forking okay, but not recommended == Because MCS acts as a service aggregate and makes multiple backend API calls, caveats can apply for those subresources - possibility of API changes, deprecation, and the like. We do not recommend a plain fork of MCS code because of the subresource fetch behavior. This said, of course you are welcome to fork in a way compatible with MCS's license. == Help spread the word == Although we are aware of the top two remaining consumers of MCS, we also are not sure who else is accessing MCS and anticipate that some downstream tech may break when MCS is turned off. As we are cross-posting this message, we hope most people who have come to rely upon MCS will see this message. Please feel free to forward this message to contacts if you know they are using MCS. == Help == Although we intend to decommission MCS in July 2023, we would like to share resources if you need some help. We plan to hold office hours in case you would like to meet with us to discuss this or other Content Transform Team matters. We will host these events on Google Meet. We will provide notice of these office hours on the wikitech-l mailing list in the coming weeks and months. Additionally, if you would like to discuss your MCS transition plans, please visit the Content Transform Team talk page: https://www.mediawiki.org/wiki/Talk:Content_Transform_Team Finally, some Content Transform Team members will also be at the Wikimedia Hackathon [4] if you would like some in-person support. Thank you. Adam Baso (he/him/his/Adam), on behalf of the Content Transform Team Director of Engineering Wikimedia Foundation [1] https://www.mediawiki.org/wiki/Parsoid [2] https://www.mediawiki.org/wiki/Specs/HTML [3] https://www.mediawiki.org/wiki/Parsoid/Parser_Unification/Updates [4] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2023

3 7

Proposal: namespace Archived for mediawiki.org
by Derk-Jan Hartman 01 Jun '23

01 Jun '23

I'm proposing we introduce a new namespace on mediawiki.org. The namespace name will be Archived (numerical id to be determined) and its purpose will be to hold pages like "Subversion" that have the template "historical" applied to it. These pages would move info that namespace and you would get Archived:Subversion, Archived:Manual:Small padlock icon, Archived:Help:Extension:WebFonts etc. This will give us a place outside of the main content namespaces to keep information about configurations, manuals, extensions and skins that we want to keep, but where it will no longer pollute our set of currently relevant information. The namespace will not be part of $wgContentNamespaces and $wgNamespacesToBeSearchedDefault namespaces. Hopefully this will allow for a more searchable and better functioning mediawiki.org when it comes to documentation for new users, while preserving history in the spirit of wikis. I've discussed this with various people at various points in time who all seemed to think this was a good idea, but we've never really had an open discussion about it that could result into action.

6 6

Wikimedia developer satisfaction survey 2023 🧑‍💻📊
by Tyler Cipriani 01 Jun '23

01 Jun '23

Hello! Please take our annual* *Developer Satisfaction Survey*! Link: https://wikimediafoundation.limesurvey.net/484133 The survey is open until Fri, 17 Feb 2023—two weeks from today. ____ This survey is for members of the *Wikimedia Developer Community* and covers the following topics: - Code review tooling and process - Code quality - Phabricator - Continuous Integration - MediaWiki development environments - Beta cluster / Staging Please take the survey if you’ve used the above tools as part of your role developing software for the Wikimedia community. We’re soliciting your feedback to: - Measure developer satisfaction, and - determine where to invest resources in the future We will anonymize, explore, and report the data we gather on mediawiki.org. View previous years' survey results: https://www.mediawiki.org/wiki/Developer_Satisfaction_Survey Privacy statement: This survey will be conducted via a third-party service, which may subject it to additional terms. For more information on privacy and data-handling, see the survey privacy statement <https://foundation.wikimedia.org/wiki/Legal:2023_Developer_Satisfaction_Sur…> . Thank you! Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation ____ *: “annual,” except we missed 2022 🙁

2 3

Replag
by Martin Domdey 01 Jun '23

01 Jun '23

Hi, it looks like there is a very big replication lag in dewiki. Is a debugging already in progress? Kind regards, Martin ...

3 3

[Breaking Change] Two changes regarding searching for proto-relative URLs in external links
by Amir Sarabadani 30 May '23

30 May '23

Hello, As part of work to make storage of external links in MediaWiki continue to scale without risking site stability (T312666 <https://phabricator.wikimedia.org/T312666>), we are deprecating most of the special functionalities around proto-relative URLs (URLs that start with // instead of https:// or http://). Proto-relative URLs were beneficial a decade ago, when Wikimedia projects were being served in both encrypted and unencrypted traffic (http and https). However, since 2015, all of our traffic has been served encrypted only, and this functionality doesn’t provide much user benefit any more for Wikimedia wikis. With HTTP/2, a similar circumstance applies to all MediaWiki users. As well as being low-value, our external links storage (the externallinks table) has grown to be one of the biggest tables for each production wiki. This is due to many duplications of URL information, added to serve different use cases. With the changes, we are removing these duplications, and some of the functionality. You can read more about the work in T312666 <https://phabricator.wikimedia.org/T312666>. Storage of proto-relative URLs has changed to only store HTTPS URLs Previously, if a proto-relative was added in an edit, MediaWiki internally treated it as two links one with http:// and one with https://. From this week forward, for all Wikimedia wikis, the storage will change to store only https:// URLs. Once those wikis are switched to read the new database schema, the links will be presented as https only in Special:LinkSearch and their API counter-parts. This means effectively a proto-relative external link will be treated like a HTTPS one. This change will also apply to non-Wikimedia wikis using MediaWiki 1.41+. expandurl option is deprecated and ignored in the exturlusage and extlinks MediaWiki action API modules This means “expandurl” argument in exturlusage and extlinks <https://www.mediawiki.org/wiki/API:Extlinks> API modules will be ignored and proto-relative URLs will be always expanded to HTTPS. This will happen any time a wiki is switched to read from the new externallinks fields. (You can track the progress in T335343 <https://phabricator.wikimedia.org/T335343>) This change will also apply to non-Wikimedia wikis using MediaWiki 1.41+. If your wiki heavily uses proto-relative URLs in articles' wikitext, we recommend changing them to https instead which also improves storage as every proto-relative URLs takes up two rows. Thank you, -- Amir Sarabadani, Staff Database Architect James Forrester, Staff Software Engineer Timo Tijhof, Principal Performance Engineer

1 0

big spaces at Commons File pages
by Martin Domdey 30 May '23

30 May '23

Hi, there's a big space on Commons file pages. Have only I this big space? See link and attachment. Thank you, Martin ... https://upload.wikimedia.org/wikipedia/commons/archive/6/6a/20230530165927%…

3 2

Maps as article image?
by Strainu 26 May '23

26 May '23

Hey folks, The maps and aticle image have been good additions to the multimedia capabilities in Wikipedia in the last decade, widely used on my home wiki. For now, in Wikipedia the maps are also rendered as images and become interactive only when clicked on. This makes them potential candidates for being displayed as article images in article without a photo. I would like to find out how complicated it would technically be to make the article image extension also capable of using map images? I know this is probably nowhere on the roadmap, I'm only interested in the technical part of the idea. Thanks, Strainu

2 1

Outdated codesearch results since 2023-05-16
by Kunal Mehta 24 May '23

24 May '23

Hi, For about a week MediaWiki Codesearch was serving outdated and incomplete results (and later partially down outright) because of a full disk. This has been fixed now, you may wish to re-run searches as necessary. The full disk was accidentally caused by switching between gerrit and gerrit-replica (as previously discussed on this list)[1][2], so I deleted all the old gerrit-replica repository copies and also added ~20G more disk space to the instance. Monitoring did correctly pick up that there was some issue once the disk was full, it just wasn't properly examined because of hackathon travel / activities. [1] https://phabricator.wikimedia.org/T336710#8877525 [2] https://phabricator.wikimedia.org/T337263 -- Kunal / Legoktm

1 0

svg translation on Commons
by Igal Khitron 24 May '23

24 May '23

Hello. I tried to start an svg translation. Added just a couple of switch statements, just to see how it works before the rest job. But it doesn't. Could you tell me, please, what am I doing wrong? Thank you. Igal (User:IKhitron) https://commons.wikimedia.org/w/index.php?title=File%3AMoscow_metro_map_mul…

2 2

Gerrit replica downtime (up to 2 hours) on Thu, 25 May
by Daniel Zahn 22 May '23

22 May '23

Hello The read-only Gerrit replica[0] will be down for up to 2 hours in 2 days (Thu, 25 May 2023) between 15:00–17:00 UTC[1] due to a scheduled OS upgrade [2]. During this window, git reads from the replica will not work. To my knowledge, this affects bots which rely on the replica for git read operations. Codesearch is currently using gerrit.wikimedia.org and not gerrit-replica.wikimedia.org so should not be affected. If this affects you yoy may consider switching to gerrit.wikimedia.org instead of the replica. Apologies for any inconvenience. Daniel [0]: <https://wikitech.wikimedia.org/wiki/Gerrit/Replica> [1]: <https://zonestamp.toolforge.org/1685026803> [2]: <https://phabricator.wikimedia.org/T334521> -- Daniel Zahn <dzahn(a)wikimedia.org> Site Reliability Engineer

1 0

Change to default value of install.php --scriptpath
by Gergő Tisza 21 May '23

21 May '23

Hi! The default value of the installer's script path parameter was changed in gerrit 133222 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/133222> to make the installation process more intuitive. If you use the installer in your scripts, you might want to make sure that script path is set explicitly. (To keep the old behavior, add "--scriptpath wiki".)

1 0

Code of Conduct Committee -- call for new members
by Martin Urbanec 20 May '23

20 May '23

Hello all, The Code of Conduct Committee is a team of five trusted individuals (plus five auxiliary members) with diverse affiliations responsible for general enforcement of the Code of conduct for Wikimedia technical spaces. Committee members are in charge of processing complaints, discussing with the parties affected, agreeing on resolutions, and following up on their enforcement. For more on their duties and roles, see https://www.mediawiki.org/wiki/Code_of_Conduct/Committee. This is a call for community members interested in volunteering for appointment to this committee. Volunteers serving in this role should be experienced Wikimedians or have had experience serving in a similar position before. The current committee is doing the selection and will research and discuss candidates. Six weeks before the beginning of the next Committee term, meaning July 16, 2023, they will publish their candidate slate (a list of candidates) on-wiki. The community can provide feedback on these candidates, via private email to the group choosing the next Committee. The feedback period will be two weeks. The current Committee will then either finalize the slate, or update the candidate slate in response to concerns raised. If the candidate slate changes, there will be another two week feedback period covering the newly proposed members. After the selections are finalized, there will be a training period, after which the new Committee is appointed. The current Committee continues to serve until the feedback, selection, and training process is complete. If you are interested in serving on this committee or like to nominate a candidate, please write an email to techconductcandidates AT wikimedia.org with details of your experience on the projects, your thoughts on the code of conduct and the committee and what you hope to bring to the role and whether you have a preference in being auxiliary or main member of the committee. The committee consists of five main members plus five auxiliary members and they will serve for a year; all applications are appreciated and will be carefully considered. The deadline for applications is *the end of day on May 31 2023*. Please feel free to pass this invitation along to any users who you think may be qualified and interested. Best, Martin Urbanec, on behalf of the Code of Conduct Committee

1 0

[Train] 1.41.0-wmf.9 status update: blocked at group1 on T336962, T336964
by Brennen Bearnes 19 May '23

19 May '23

The 1.41.0-wmf.9 version of MediaWiki is blocked[0]. The new version is currently deployed to group1, but can proceed no further until these issues are resolved: * T336962 - UnexpectedValueException: Unknown image suggestions API kind: istype-depicts - https://phabricator.wikimedia.org/T336962 * T336964 - InvalidArgumentException: Data for lt_namespace and lt_title must be non-empty - https://phabricator.wikimedia.org/T336964 Once these issues are resolved train can resume. Thank you for any help resolving these issues! -- Your disgruntled train drivers [0]. https://phabricator.wikimedia.org/T330215 [1]. https://versions.toolforge.org/

2 2

+2 for Lucas Werkmeister [volunteer] in mediawiki/
by Lucas Werkmeister 19 May '23

19 May '23

Hi all, at the ongoing Hackathon, Legoktm made me nominate myself for +2 in mediawiki/* repositories ^^ see <https://phabricator.wikimedia.org/T337014>. Cheers, Lucas

3 2

Re-introducing MediaWiki CLI profiler for maintenance scripts
by Krinkle 17 May '23

17 May '23

TLDR: Those with shell access at WMF can now run maintenance scripts on mwdebug hosts, and use the *--profiler=text* option to produce a report detailing how long the script spent in each MediaWiki component, class, and function. == *What* == The MediaWiki platform at WMF consists of broadly four different deployments: app servers, api_app servers, job runners, and maint servers (diagram <https://wikitech.wikimedia.org/wiki/MediaWiki_at_WMF#/media/File:MediaWiki_…>). Each can be thought of as its own service for a specific purpose, composed of a subset of components from the MediaWiki codebase with fast local access to each. The largest of these is the appservers cluster, which is served by 150 servers of dedicated hardware across the two application data centers <https://wikitech.wikimedia.org/wiki/Data_centers>, and is responsible for responding to index.php (i.e. page views) and load.php (CSS, JS, and localisation via ResourceLoader). Today, we focus on the smallest of these: mwmaint servers <https://wikitech.wikimedia.org/wiki/Maintenance_server>. This is backed by two heavy-duty servers, one in each data center, that autonomously run essential tasks at a predefined schedule (i.e. not in direct response to a user action). Each of these ~50 different tasks is implemented as a MediaWiki maintenance script. Important examples include: sending email notifications (Echo extension), timely pruning of sensitive PII (CheckUser extension), computing mentee and link recommendation data (GrowthExperiments), and reclaiming disk space for expired caches (core/ParserCache). == *Why ==* We have detailed debug performance profiling in production for web requests via the WikimediaDebug extension, and we have detailed profiling in local development for both web requests and maintenance scripts (Docker recipe <https://www.mediawiki.org/wiki/MediaWiki-Docker/Configuration_recipes/Profi…>). What was missing is a way to profile maintenance scripts in production. This is important as maintenance scripts tend take many minutes or hours to process vast amounts of production data. While generally easy to debug locally for functional analysis, the performance bottlenecks individual teams care about are likely specific to the size of the data and the performance of other production components. Thanks also to Ahmon Dancy (RelEng), Giuseppe Lavagetto (SRE), and Aaron Schulz (Performance Team) for making this work possible, and Niklas Laxström (LangEng) for coming up with the idea. == *What's New* == Documentation: https://wikitech.wikimedia.org/wiki/WikimediaDebug#Plaintext_CLI_profile To profile a Maintenance script, run the script from the shell with *mwscript* as you normally would, but instead of connecting your terminal to mwmaint1002.eqiad.wmnet, connect to one of the *mwdebug* hosts (such as mwdebug1001.eqiad.wmnet). Then pass the *--profiler=text* option to generate a report with the performance analysis, which will be printed after the task is finished. Like so: > $ mwscript showSiteStats.php --wiki=nlwiki --profiler=text > Number of articles: 2122688 > Number of users : 1276507 > >  == *A peak behind the curtain* == Read on if you'd like to learn what hurdles we had to overcome for this to "simply" work in production, like it did for local development. The journey started when Niklas (WMF LangEng) proposed the idea at https://phabricator.wikimedia.org/T253547. *Firstly*, the profiler engine. In 2019 (blogpost <https://techblog.wikimedia.org/2019/12/16/wikimediadebug-v2-is-here/>), after we migrated from HHVM to PHP 7, we had to look for a new profiler engine for backend performance. We adopted the open source php-tideways package, and this has powered our browser-facing profiler since. Naturally, this was already installed on the mwdebug servers for that purpose. However, the package, and the accompanying *rdtsc* setting, were only set for php-fpm (web server), it was not yet enabled for php-cli (command-line). *Secondly*, the Profiler component in MediaWiki core had gotten out of sync with the needs of the Skin and Maintenance components. Over years of refactoring and more parts of each component gaining an active owner, the parts that lacked an owner eroded and stopped working, including the integration between these components. We decided to take active ownership over the remaining parts of MediaWiki-Core-Profiler and fix the disconnect. The meta work for that included identifying and re-triaging open issues under a new #mediawiki-core-profiler <https://phabricator.wikimedia.org/tag/mediawiki-core-profiler/> tag, automating discovery to our team inbox via Phabricator Herald rule, enlisting on Maintainers <https://www.mediawiki.org/wiki/Developers/Maintainers>, and automatic discovery of changesets to our code review dashboard. <https://gerrit.wikimedia.org/r/p/wikimedia/+/dashboard/teams:performance> The "output" step of the profiler was originally the responsibility of a wfLogProfilingData function that both webpages (Skin) and CLI (Maintenance) called upon toward the end of the response. After this function became deprecated, and further refactoring in the Skin and Maintenance component took on (some of) its responsibilities directly, the "output" step got lost. Adding this back in was non-trivial because by now, the code in question had gained an unintentional dependency on the WebRequest object, which is not valid in a CLI context. The code in question was simplified and decoupled in change 725152 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/725152/> and change 725440 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/725440>, when then made it possible to add back the "output" step in change 838884 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/838884/>. We also removed various options <https://gerrit.wikimedia.org/r/q/message:profiler+project:mediawiki/core+br…> (such as CMDLINE_CALLBACK <https://phabricator.wikimedia.org/T305422>) that we choose to not support and maintain going forward (undocumented, unused, broken, or without known use case after research). *Thirdly*, in parallel with our work above, another team refactored MediaWiki's SettingsLoader and MaintenanceRunner components. The Profiler dates back more than a decade and still relied on the fact that settings could be changed by CLI script options such as *--profiler=text*. The new SettingsLoader and the refactored MaintenanceRunner streamlined the order of operations during process startup, which had the side effect of initialising the profiler slightly earlier than it used to. This meant it would initialise before the --profiler=text option was applied, and thus the profiler that was initialised was unconditionally the "null" profiler. This did not produce an error since that is a valid configuration, the configuration of effectively all traffic besides ad-hoc debugging. I remedied this <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/875396/> by recognising the CLI option as its own setting (separate from the main one), and passing it down via dependency injection. Thus not relying on the subtle order of operations. The Profiler component is now significantly leaner than it was before, and its requirements are either explicitly coded through dependency injection requirements, or simplified/refactored such that they do not place requirements on other components. *Fourthly*, the wmf-config repository, where we control how and when MediaWiki Profiler can be enabled, had been changed by us the year before, to feature a new sampling profiler to produce detailed flame graphs over production traffic (blog post <https://techblog.wikimedia.org/2021/03/03/profiling-php-in-production-at-sc…>). I realised that this configuration assumed a web context. Another tweak over there <https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/874419/> made this safe to enabled in CLI/Maintenance script contexts. *Fifthly*, in order to actually enable the wmf-config/Profiler code in CLI, we needed to set the same *php.ini* configuration for php-cli as we already did years earlier for php-fpm (web server). In doing so, we ran into the problem that one server in particular (deploy1002, from where the MediaWiki train runs via Scap) had two mutually exclusive copies of the MediaWiki deployment (/srv/mediawiki and /srv/mediawiki-staging). This meant that preloading a profiler for either of them from php-cli would break the other. I identified this long-standing technical debt to Ahmon Dancy (RelEng), who then went on a whole journey of his own to radically simplify our deployment servers to not have these two separate installations (T329857 <https://phabricator.wikimedia.org/T329857>). *Finally*, with a one-line change in Puppet <https://gerrit.wikimedia.org/r/c/operations/puppet/+/910882/> config, the CLI profiler is now enabled in production (Thanks Giuseppe Lavagetto, from SRE). All in all, this made the different parts of codebase, and different parts of our platform less divergent and more unified than before. Thanks again to Aaron Schulz, Ahmon Dancy, Giuseppe Lavagetto, and Niklas Laxström for their help! -- Timo Tijhof, Principal Engineer, Performance Team, Wikimedia Foundation.

1 0

Permission model for /repos/mediawiki on GitLab
by Brennen Bearnes 16 May '23

16 May '23

Hey all, RelEng are working through some of the nitty-gritty details of migrating MediaWiki repositories to GitLab, and we could use some input on a permission model for the planned /repos/mediawiki namespace. Draft permission model policy: * https://www.mediawiki.org/wiki/GitLab/Policy#MediaWiki_permission_model Task: * T336807 - Define a permissions model for the /repos/mediawiki/ namespace on GitLab - https://phabricator.wikimedia.org/T336807 Thanks! -- Brennen Bearnes Release Engineering Wikimedia Foundation

1 0

send mail in python
by Tim Moody 16 May '23

16 May '23

I'd like to send an email from a python3 process on a wmcs VPS to report errors. I looked at https://wikitech.wikimedia.org/wiki/Help:Email_in_Cloud_VPS but could use some help. sudo echo "Subject: sendmail test2" | /usr/sbin/sendmail -v <myemail> works. When I try to send the equivalent from python smtplib I get a 221 error message. Thanks, Tim

2 3

Word embeddings / vector search
by Lars Aronsson 16 May '23

16 May '23

A partial outcome of the research in natural language processing in the last decade is the representation of language as numeric vectors, called word embeddings. These are used in large language models such as Bert, Elmo, and (Chat)GPT. A peculiar aspect of these numeric vectors is that they cluster semantically, so that words for similar concepts (dog, puppy, pet) group together even though their spelling is very different. This can be used for "semantic" search. If a search query (dog) is converted to a vector, it can search terms found in documents (e.g. wiki articles) that have similar vectors and find those of similar content even though the text doesn't match. https://en.wikipedia.org/wiki/Word_embedding Here are just two of very many videos that explain the concept: https://www.youtube.com/watch?v=xzHhZh7F25I https://www.youtube.com/watch?v=MUve9LiEAeI Is there any ongoing work at WMF or around the Mediawiki software to apply this new technique to search in Wikipedia? -- Lars Aronsson (lars(a)aronsson.se, user:LA2) Linköping, Sweden

6 8

Gerrit replica downtime (30 minutes) tomorrow Tue, 16 May 13:00-15:00 UTC
by Tyler Cipriani 16 May '23

16 May '23

Hello The read-only Gerrit replica[0] will be down for 30 minutes tomorrow (Tue, 16 May 2023) between 13:00–15:00 UTC[1] due to network switch upgrades in codfw row D[2]. During this window, git reads from the replica will not work. To my knowledge, this affects bots which rely on the replica for git read operations. Apologies for any inconvenience. Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation [0]: <https://wikitech.wikimedia.org/wiki/Gerrit/Replica> [1]: <https://zonestamp.toolforge.org/1684242019> [2]: <https://phabricator.wikimedia.org/T335042>

4 7

Gerrit downtime maintenance Wed, 10 May 2023 14:00–15:00 UTC
by Tyler Cipriani 11 May '23

11 May '23

Hi All We're moving the primary Gerrit host to new hardware on *Wed, 10 May 2023 at 14:00 UTC*[0] There will be downtime during the move. I expect downtime to be less than the full hour, but please plan for a full hour. Apologies for the inconvenience :( Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation [0]: <https://phabricator.wikimedia.org/T326368>

2 1

[Train] 1.41.0-wmf.8 status update: blocked at group1 on T336504 (Vector 2022 force-deploying on arbitrary pages)
by Brennen Bearnes 11 May '23

11 May '23

The 1.41.0-wmf.7 version of MediaWiki is blocked[0]. The new version is currently deployed to group1, but can proceed no further until this issue is resolved: * T336504 - Vector 2022 force-deploying on arbitrary pages - https://phabricator.wikimedia.org/T336504 Once these issues are resolved train can resume. Thank you for any help resolving these issues! -- Your humble train toilers [0]. https://phabricator.wikimedia.org/T330214 [1]. https://versions.toolforge.org/

1 0

Python requests broken by urllib3 version 2.x
by Antoine Musso 11 May '23

11 May '23

Hello, This is for python projects. Today, May 4th, urllib3 <https://pypi.org/project/urllib3/#history> has released a new major version 2.0.2 which breaks the extremely popular requests <https://pypi.org/project/requests/> library. The fix is to pin urllib3<2 to prevent the new major version from being installed (example <https://gerrit.wikimedia.org/r/c/integration/config/+/915736/1/tox.ini>). https://phabricator.wikimedia.org/T335977 Upstream issue: https://github.com/psf/requests/issues/6432 Antoine "hashar" Musso Wikimedia Release Engineering

8 12

[Technical Documentation] Metrics dashboard for informed docs decisions
by Kamil Bach 11 May '23

11 May '23

Hello everyone, Summary WMF’s technical writers are currently working on a dashboard that will display documentation metrics to help us make more data-driven docs decisions. We are currently in the design stage. If you maintain technical documentation for your team or project, we are very interested in hearing your thoughts! Please take a look at the requirements [1] and initial design ideas [2] and share your feedback either on talk pages or in Phabricator [3]. Long version Technical writers at WMF have started a project to gather relevant documentation metrics in one place. The intended outcome of this project is a publicly-available dashboard. This dashboard will help anyone who is maintaining documentation collections (sets of pages that describe the same project, tool, or area) to reason about the state of their docs, and plan documentation work. The dashboard will be immediately useful to technical writers, but we also want to design it in a way that will serve the broader community, specifically including the perspective of folks who only work with documentation occasionally. To this end, we’ve decided to reach out to you early to gather feedback about the requirements [1] and our initial design assumptions [2]. If you actively maintain any technical documentation, please take a look at the linked resources and share your thoughts and comments on talk pages or in Phabricator [3]. Thank you! On behalf of WMF’s technical writers, Kamil Bach [1] https://www.mediawiki.org/wiki/Documentation/Tools/Documentation_metrics_da… [2] https://www.mediawiki.org/wiki/Documentation/Tools/Documentation_metrics_da… [3] https://phabricator.wikimedia.org/T334839 -- Kamil Bach (they/them) Technical Writer Wikimedia Foundation

1 0

Congratulations to accepted candidates for Google Summer of Code 2023 & Outreachy Round 26
by Vasanth Gopa 10 May '23

10 May '23

Hello folks, Please join us in welcoming Wikimedia's accepted candidates for Google Summer of Code 2023 [1] and Outreachy Round 26 [2]! **Google Summer of Code 2023** 1. Ahmed Amine Hassou, Morocco, Wiki Education Dashboard: Refactoring and upgrading React <https://phabricator.wikimedia.org/T332976> Mentors: Sage Ross 2. Chenhao Liu, United States of America, Wiki farm support for Canasta < https://phabricator.wikimedia.org/T333773> Mentors: Yaron Koren, Jeffrey Wang 3. Nikhil Mahajan, India, End-to-end test coverage for Abstract Wikipedia's Wikifunctions <https://phabricator.wikimedia.org/T333498> Mentors: Stef Dunlap, James Forrester, Cory Massaro, Denny Vrandečić 4. Zexi Gong, China, Wikidocumentaries to import images from the web to Structured Data on Commons <https://phabricator.wikimedia.org/T332671> Mentors: TuukkaH, Susannaanas 5. Shashwat Khanna, India, Improve Programs & Events Dashboard UX for Article Scoped Programs <https://phabricator.wikimedia.org/T332932> Mentors: Sage Ross 6. Saurabh Jamadagni, India, Adding a menu to Scribe-iOS application and expanding keyboard languages <https://phabricator.wikimedia.org/T333156> Mentors: Andrew McAllister 7. Ritika Pahwa, India, Commons Android app: Make upload more reliable < https://phabricator.wikimedia.org/T332126> Mentors: Nicolas Raoul, Kaartic Sivaraam 8. Punith Nayak, India, Improve the functionality of VideoCutTool - New features <https://phabricator.wikimedia.org/T330465> Mentors: Gopa Vasanth, Sohom Datta 9. Varun Shrivastava, India, Improve the functionality of VideoCutTool - Code Quality/Code Health <https://phabricator.wikimedia.org/T333466> Mentors: Gopa Vasanth, Sohom Datta **Outreachy Round 26** 1. Nathaly Toledo, Venezuela, Research imbalances in translation between languages on Wikipedia <https://phabricator.wikimedia.org/T333792> Mentors: Adam Wight, Kavitha A, Simulo 2. Abhishek Bhardwaj, India, Research imbalances in translation between languages on Wikipedia <https://phabricator.wikimedia.org/T333097> Mentors: Adam Wight, Kavitha A, Simulo 3. Sulagna Saha, Bangladesh, Write a Ruby gem for analyzing Wikidata edits <https://phabricator.wikimedia.org/T333790> Mentors: Sage Ross, Will Kent We would like to encourage accepted candidates to introduce themselves on this thread and give a brief overview of the project they will be working on. We’re so proud of the contributions they have made so far to our community, and we look forward to having a wonderful time working with them over the summer. Also, a huge shout-out to the project mentors for their enthusiasm and commitment! [1] https://www.mediawiki.org/wiki/Google_Summer_of_Code/2023 [2] https://www.mediawiki.org/wiki/Outreachy/Round_26 Regards, GSoC / Outreachy Wikimedia organization administrators (Gopa, Sohom, Srishti)

2 1

Re: Congratulations to accepted candidates for Google Summer of Code 2023 & Outreachy Round 26
by Shashwat Khanna 09 May '23

09 May '23

Hi everyone 👋 I am Shashwat and I'm currently pursuing an undergrad degree in Computer Science. You may remember me from last year - I had participated in GSoC 2022 as well and my project was about improving the JavaScript tooling of WikiEduDashboard. As you might've guessed, that project was mostly relevant to developers and users of the Dashboard probably won't have noticed any changes. This year, I'm still working on the Dashboard but on a more user side project. My project is about improving/designing the UI for adding and editing the various scoping methods supported by the Dashboard to make more instructors aware of these features. It'll involve a bit of designing, some user research, and a lot of prototyping. I had a lot of fun last year and can't wait to meet everyone again. Thanks and congratulations to the rest of the interns. Shashwat On Tue, May 9, 2023, 7:55 PM Saurabh Jamadagni < saurabhjamadagni2002(a)gmail.com> wrote: > Hey all! > > My name is Saurabh Jamadagni and I am a Junior in college from India. I'll > be contributing to the Scribe-iOS project under the mentorship of Andrew. > > My tasks for the duration of my contributing period will be to create an > application menu based on the design files of Scribe. Currently, the mobile > app only consists of two pages with the core functionality. The Scribe > codebase has received a lot of changes in the past few months. These have > brought in new exciting features and a comprehensive menu will allow the > users to select their preferences in a more user friendly manner. The > second part of my project will be to add new languages to the Scribe > keyboard. As Scribe is a keyboard application for language learners, the > scope to expand the offerings is huge. By working on this project I will be > able to contribute in bringing more languages to Scribe using the abundant > Wikidata data stores. > > To talk about my likings outside programming, I play badminton and am part > of my college team. I also love tinkering with water colours and painting! > I am so excited to be a part of the community (even just being a part of > this email thread is thrilling haha). I would like to thank all the Org > Admins and my mentor for a smooth application process. You have been > immensely helpful! > > Looking forward to a great summer and beyond with you all :) > > Warm Regards, > Saurabh > > On Tue, May 9, 2023 at 7:35 PM Sohom Datta <dattasohom1(a)gmail.com> wrote: > >> Congratulations and welcome, looking forward to seeing you guys around in >> Wikimedia :) >> Regards, >> Sohom Datta >> --- >> Open-source contributor @Wikimedia >> >> >> On Tue, May 9, 2023 at 9:54 AM Nikhil <nik.xyz.in(a)gmail.com> wrote: >> >>> Greetings, >>> I would like to introduce myself as Nikhil Mahajan from India, currently >>> a sophomore at IIT Roorkee <https://www.iitr.ac.in/>. I am enthusiastic >>> about software development and have been actively involved as a software >>> developer at the coding club <https://github.com/mdgspace> at my >>> college. Some of my noteworthy projects can be found on my Github >>> profile <https://github.com/nik-55>. >>> My journey into the world of open source began through GSoC, and I made >>> my first contribution to MediaWiki. I was thrilled to see my first patch >>> get merged and it gives me a boost to make more contributions. During my >>> exploration of MediaWiki, I came across an interesting project called >>> "Abstract Wikipedia". The GSoC project is about writing end-to-end tests >>> for different critical user journeys in Wikifunctions. These tests will run >>> against any patch review. The tests fail if the patch introduces changes >>> that break down any of the user journeys. This will help avoid merging the >>> patches that mistakenly introduce bugs into the project >>> Initially, I faced some difficulty in understanding the concept of >>> "zobject" in the Abstract Wikipedia project. However, my mentor, @stef, >>> helped me to gain a clearer understanding of the project. >>> Although my university exams have kept me away from making contributions >>> for a while, I am now keen to contribute to the project. >>> >>> Thank you for your time and consideration. >>> >>> Sincerely, >>> Nikhil Mahajan >>> >>> On Tue, 9 May 2023 at 16:53, Vasanth Gopa <gopavasanth1999(a)gmail.com> >>> wrote: >>> >>>> Hello folks, >>>> >>>> Please join us in welcoming Wikimedia's accepted candidates for Google >>>> Summer of Code 2023 [1] and Outreachy Round 26 [2]! >>>> >>>> **Google Summer of Code 2023** >>>> >>>> 1. Ahmed Amine Hassou, Morocco, Wiki Education Dashboard: >>>> Refactoring and upgrading React < >>>> https://phabricator.wikimedia.org/T332976> >>>> Mentors: Sage Ross >>>> >>>> 2. Chenhao Liu, United States of America, Wiki farm support for >>>> Canasta <https://phabricator.wikimedia.org/T333773> >>>> Mentors: Yaron Koren, Jeffrey Wang >>>> >>>> 3. Nikhil Mahajan, India, End-to-end test coverage for Abstract >>>> Wikipedia's Wikifunctions <https://phabricator.wikimedia.org/T333498> >>>> Mentors: Stef Dunlap, James Forrester, Cory Massaro, Denny >>>> Vrandečić >>>> >>>> 4. Zexi Gong, China, Wikidocumentaries to import images from the >>>> web to Structured Data on Commons < >>>> https://phabricator.wikimedia.org/T332671> >>>> Mentors: TuukkaH, Susannaanas >>>> >>>> 5. Shashwat Khanna, India, Improve Programs & Events Dashboard UX >>>> for Article Scoped Programs <https://phabricator.wikimedia.org/T332932> >>>> Mentors: Sage Ross >>>> >>>> 6. Saurabh Jamadagni, India, Adding a menu to Scribe-iOS >>>> application and expanding keyboard languages < >>>> https://phabricator.wikimedia.org/T333156> >>>> Mentors: Andrew McAllister >>>> >>>> 7. Ritika Pahwa, India, Commons Android app: Make upload more >>>> reliable <https://phabricator.wikimedia.org/T332126> >>>> Mentors: Nicolas Raoul, Kaartic Sivaraam >>>> >>>> 8. Punith Nayak, India, Improve the functionality of VideoCutTool - >>>> New features <https://phabricator.wikimedia.org/T330465> >>>> Mentors: Gopa Vasanth, Sohom Datta >>>> >>>> 9. Varun Shrivastava, India, Improve the functionality of >>>> VideoCutTool - Code Quality/Code Health < >>>> https://phabricator.wikimedia.org/T333466> >>>> Mentors: Gopa Vasanth, Sohom Datta >>>> >>>> **Outreachy Round 26** >>>> >>>> 1. Nathaly Toledo, Venezuela, Research imbalances in translation >>>> between languages on Wikipedia < >>>> https://phabricator.wikimedia.org/T333792> >>>> Mentors: Adam Wight, Kavitha A, Simulo >>>> >>>> 2. Abhishek Bhardwaj, India, Research imbalances in translation >>>> between languages on Wikipedia < >>>> https://phabricator.wikimedia.org/T333097> >>>> Mentors: Adam Wight, Kavitha A, Simulo >>>> >>>> 3. Sulagna Saha, Bangladesh, Write a Ruby gem for analyzing >>>> Wikidata edits <https://phabricator.wikimedia.org/T333790> >>>> Mentors: Sage Ross, Will Kent >>>> >>>> We would like to encourage accepted candidates to introduce themselves >>>> on this thread and give a brief overview of the project they will be >>>> working on. >>>> >>>> We’re so proud of the contributions they have made so far to our >>>> community, and we look forward to having a wonderful time working with them >>>> over the >>>> summer. Also, a huge shout-out to the project mentors for their >>>> enthusiasm and commitment! >>>> >>>> [1] https://www.mediawiki.org/wiki/Google_Summer_of_Code/2023 >>>> [2] https://www.mediawiki.org/wiki/Outreachy/Round_26 >>>> >>>> Regards, >>>> GSoC / Outreachy Wikimedia organization administrators (Gopa, Sohom, >>>> Srishti) >>>> >>>

1 0

Re: Congratulations to accepted candidates for Google Summer of Code 2023 & Outreachy Round 26
by Sohom Datta 09 May '23

09 May '23

Congratulations and welcome, looking forward to seeing you guys around in Wikimedia :) Regards, Sohom Datta --- Open-source contributor @Wikimedia On Tue, May 9, 2023 at 9:54 AM Nikhil <nik.xyz.in(a)gmail.com> wrote: > Greetings, > I would like to introduce myself as Nikhil Mahajan from India, currently a > sophomore at IIT Roorkee <https://www.iitr.ac.in/>. I am enthusiastic > about software development and have been actively involved as a software > developer at the coding club <https://github.com/mdgspace> at my college. > Some of my noteworthy projects can be found on my Github profile > <https://github.com/nik-55>. > My journey into the world of open source began through GSoC, and I made my > first contribution to MediaWiki. I was thrilled to see my first patch get > merged and it gives me a boost to make more contributions. During my > exploration of MediaWiki, I came across an interesting project called > "Abstract Wikipedia". The GSoC project is about writing end-to-end tests > for different critical user journeys in Wikifunctions. These tests will run > against any patch review. The tests fail if the patch introduces changes > that break down any of the user journeys. This will help avoid merging the > patches that mistakenly introduce bugs into the project > Initially, I faced some difficulty in understanding the concept of > "zobject" in the Abstract Wikipedia project. However, my mentor, @stef, > helped me to gain a clearer understanding of the project. > Although my university exams have kept me away from making contributions > for a while, I am now keen to contribute to the project. > > Thank you for your time and consideration. > > Sincerely, > Nikhil Mahajan > > On Tue, 9 May 2023 at 16:53, Vasanth Gopa <gopavasanth1999(a)gmail.com> > wrote: > >> Hello folks, >> >> Please join us in welcoming Wikimedia's accepted candidates for Google >> Summer of Code 2023 [1] and Outreachy Round 26 [2]! >> >> **Google Summer of Code 2023** >> >> 1. Ahmed Amine Hassou, Morocco, Wiki Education Dashboard: Refactoring >> and upgrading React <https://phabricator.wikimedia.org/T332976> >> Mentors: Sage Ross >> >> 2. Chenhao Liu, United States of America, Wiki farm support for >> Canasta <https://phabricator.wikimedia.org/T333773> >> Mentors: Yaron Koren, Jeffrey Wang >> >> 3. Nikhil Mahajan, India, End-to-end test coverage for Abstract >> Wikipedia's Wikifunctions <https://phabricator.wikimedia.org/T333498> >> Mentors: Stef Dunlap, James Forrester, Cory Massaro, Denny >> Vrandečić >> >> 4. Zexi Gong, China, Wikidocumentaries to import images from the web >> to Structured Data on Commons <https://phabricator.wikimedia.org/T332671> >> Mentors: TuukkaH, Susannaanas >> >> 5. Shashwat Khanna, India, Improve Programs & Events Dashboard UX for >> Article Scoped Programs <https://phabricator.wikimedia.org/T332932> >> Mentors: Sage Ross >> >> 6. Saurabh Jamadagni, India, Adding a menu to Scribe-iOS application >> and expanding keyboard languages < >> https://phabricator.wikimedia.org/T333156> >> Mentors: Andrew McAllister >> >> 7. Ritika Pahwa, India, Commons Android app: Make upload more >> reliable <https://phabricator.wikimedia.org/T332126> >> Mentors: Nicolas Raoul, Kaartic Sivaraam >> >> 8. Punith Nayak, India, Improve the functionality of VideoCutTool - >> New features <https://phabricator.wikimedia.org/T330465> >> Mentors: Gopa Vasanth, Sohom Datta >> >> 9. Varun Shrivastava, India, Improve the functionality of >> VideoCutTool - Code Quality/Code Health < >> https://phabricator.wikimedia.org/T333466> >> Mentors: Gopa Vasanth, Sohom Datta >> >> **Outreachy Round 26** >> >> 1. Nathaly Toledo, Venezuela, Research imbalances in translation >> between languages on Wikipedia <https://phabricator.wikimedia.org/T333792 >> > >> Mentors: Adam Wight, Kavitha A, Simulo >> >> 2. Abhishek Bhardwaj, India, Research imbalances in translation >> between languages on Wikipedia <https://phabricator.wikimedia.org/T333097 >> > >> Mentors: Adam Wight, Kavitha A, Simulo >> >> 3. Sulagna Saha, Bangladesh, Write a Ruby gem for analyzing Wikidata >> edits <https://phabricator.wikimedia.org/T333790> >> Mentors: Sage Ross, Will Kent >> >> We would like to encourage accepted candidates to introduce themselves on >> this thread and give a brief overview of the project they will be working >> on. >> >> We’re so proud of the contributions they have made so far to our >> community, and we look forward to having a wonderful time working with them >> over the >> summer. Also, a huge shout-out to the project mentors for their >> enthusiasm and commitment! >> >> [1] https://www.mediawiki.org/wiki/Google_Summer_of_Code/2023 >> [2] https://www.mediawiki.org/wiki/Outreachy/Round_26 >> >> Regards, >> GSoC / Outreachy Wikimedia organization administrators (Gopa, Sohom, >> Srishti) >> >

1 0

🌱 Fresh 23.05.1 released!
by Krinkle 08 May '23

08 May '23

TLDR: fresh-node now supports a one-off "command" invocation mode. Learn more or install: https://gerrit.wikimedia.org/g/fresh Changelog: https://gerrit.wikimedia.org/g/fresh/+/23.05.1/CHANGELOG.md Each of the fresh-node scripts now supports a positional "command" argument, to run a single command without launching a shell first. For example: fresh-node -- npm install. Thanks *Gergő Tisza* and *Kosta Harlan* for their contributions! fresh-node16 has been upgraded to include Firefox 102.10.0esr and Chromium 112. The same container has been in use in WMF CI for npm tests in most repos since 12 April 2023. The welcome text saw a make-over this release, featuring a new mimalistic look. I hope this will make the environment feel even snappier. By consensing this baseline, timely warnings about enabled mount points and environment exposure should stand out more. *Before* <https://phabricator.wikimedia.org/F36989286> / *After* <https://phabricator.wikimedia.org/F36989285> Fresh is a fast way to launch isolated environments from your terminal. These can be used to work more responsibly with 'npm' developer tools such as ESLint, QUnit, Grunt, Selenium, and more. Example guide: https://www.mediawiki.org/wiki/Manual:JavaScript_unit_testing. To report issues or browse past and current tasks, check Phabricator at https://phabricator.wikimedia.org/tag/fresh/. -- Timo Tijhof, Principal Engineer, Wikimedia Foundation.

1 0

[Train] 1.41.0-wmf.7 status update: blocked at group1 on T336008, T336022
by Brennen Bearnes 05 May '23

05 May '23

The 1.41.0-wmf.7 version of MediaWiki is blocked[0]. The new version is currently deployed to group1, but can proceed no further until these issues are resolved: * T336022 - 1.41.0-wmf.7 increases Memcached call rate by +100% - https://phabricator.wikimedia.org/T336022 * T336008 - MWException: Internal error in ApiQueryRevisionsBase::getRevisionRecords: RevisionStore does not return record for [n] - https://phabricator.wikimedia.org/T336008 Once these issues are resolved train can resume. If they are not resolved before the end of working day today, the train can resume on Monday, May 8th. Thank you for any help resolving these issues! -- Your humble train toilers [0]. https://phabricator.wikimedia.org/T330213 [1]. https://versions.toolforge.org/

1 0

[CoC] Proposed amendment: Ban on soliciting tech support for hate groups
by Amir Sarabadani 05 May '23

05 May '23

Hello, In accordance with CoC's amendment policy <https://www.mediawiki.org/wiki/Code_of_Conduct/Amendments>. The committee proposes an amendment on behalf of one of the community members to be added in unacceptable behavior section <https://www.mediawiki.org/wiki/Code_of_Conduct#Unacceptable_behavior>: Soliciting help, support or technical assistance for websites whose predominant activity or content is behaviour that explicitly and significantly violates provisions of the code of conduct is forbidden in Wikimedia technical spaces. For example, websites advocating violence or hate speech are not welcome to ask for support on Wikimedia support forums. This is similar to w:en:Wikipedia:No Nazis <https://en.wikipedia.org/wiki/en:Wikipedia:No%20Nazis> This will be open for three months for community feedback in https://www.mediawiki.org/wiki/Topic:Xgg6dvg2athdb1h7 and if the community reaches consensus and the committee won't oppose it, It will be added to the policy. Best Amir (he/him), On behalf of the CoC committee

2 1

GitLab downtime Tue, 02 May 09:00 - 12:00 UTC
by Jelto Wodstrcil 04 May '23

04 May '23

Hello *GitLab will be down this coming Tuesday, 02 May, from 09:00 - 12:00 UTC* for the datacenter switchover[0]. Much of this switchover window will be downtime, during which you will be unable to access GitLab through ssh or the web interface. The process will involve DNS changes and a full backup of the GitLab server in the codfw datacenter, which will then be restored on the GitLab server in the eqiad datacenter. You can follow along on IRC in the libera.chat #wikimedia-gitlab channel[1]. [0]: <https://phabricator.wikimedia.org/T335504> [1]: <ircs://irc.libera.chat:6697/wikimedia-gitlab>

2 2

Talk to the Search Platform / Query Service Team—May 3, 2023
by Guillaume Lederrey 03 May '23

03 May '23

Hello all! The Search Platform Team usually holds an open meeting on the first Wednesday of each month. Come talk to us about anything related to Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query Service (WCQS), etc.! Feel free to add your items to the Etherpad Agenda for the next meeting. Details for our next meeting: Date: Wednesday, May 3, 2023 Time: 16:00-17:00 UTC / 08:00 PDT / 11:00 EDT / 17:00 CET Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours Google Meet link: https://meet.google.com/vgj-bbeb-uyi Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927 Have fun and see you soon! Guillaume -- *Guillaume Lederrey* (he/him) Engineering Manager Wikimedia Foundation <https://wikimediafoundation.org/>

3 4

Continuing development of mediawiki Extension AutoCreatePage
by Oliver Tautz 03 May '23

03 May '23

Hello, I use the extension AutoCreatePage (https://www.mediawiki.org/wiki/Extension:AutoCreatePage) in a project at my University. But it does not work in current mediawiki versions as it was last updated . So I try to adapt it to work in mediawiki 1.39.3. It almost works now and I'm considering releasing it but don't know how to proceed. I forked the github project (https://github.com/mkroetzsch/AutoCreatePage/pull/16) but got no response for now. There are also older pull requests not answered. Should I somehow replace the existing extension with my new one or would it be better to make my fork a new extension with a different name? It does not feel right, because I use a lot of the code from the old extension. I hope this is the correct mailing list for this topic. Otherwise I would be thankful if you could direct me to the correct place to ask about this. Kind Regards, Oliver Tautz

2 1

Two breaking changes related to Databases
by Amir Sarabadani 02 May '23

02 May '23

Hello, We have two breaking changes to announce. Breaking change to Rdbms Database subclasses There are no Database subclasses currently known in Codesearch (besides those built-in to core). But, since the class is documented as "stable to extend" in API, an announcement is in order. First, an explanation on what is happening. Currently, queries flow in the Rdbms library as follows: When Database::select() or similar is called (directly, or via the query builders), the Rdbms library internally builds the SQL text and then calls the general-purpose function Database::query(). Subsequently, ::query() then runs several regexes over the SQL text to figure out information about this query (which tables are affected, whether it’s a write query, what the query “verb” is, and so on). This is taxing and slow. The appservers at WMF, for example, spend 0.34% of CPU on web requests just to extract information from already-formatted SQL queries for logging purposes. This logic drastically reduces readability and maintainability of Rdbms internals. To fix this, we've internally introduced a new class called Query, to represent the SQL text in a more structured way in the first place. This avoids having to first format structure data into SQL text only to reverse-engineer it right afterwards. This change requires changing the protected Database::executeQuery() method to no longer accept a string argument and instead require a Query object. Additionally IDatabase::getTempTableWrites() has become a private method. See the patch <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/910756> or the ticket <https://phabricator.wikimedia.org/T326181> for more information. The public Database::query() method continues to support a string argument, but now also supports a Query object. When given a string, it falls back to the old regexes to extract and create a Query object. With the newly introduced UnionQueryBuilder, we believe there are no longer cases where MediaWiki extensions have to call Database::query, and can instead adopt query builders for all their queries. Direct calls to Database::query outside of MediaWiki core are now highly discouraged. Breaking change to typehints of IDatabase Many parts of core typehint to IDatabase, but most times these always return a replica database. For example, ApiBase::getDB(). This means the interface technically allows calling insert(), and this is indeed valid, but would also immediately throw a fatal error once the method is entered. Going forward, most of these will be typehinted to IReadableDatabase instead, which is a narrower interface (34 public methods instead of 83). This is not a breaking change in practice, because calls to those methods would already produce an error, but the typehint makes it official now. This <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/911269> is the patch changing ApiBase::getDB(), and we expect more such changes to other parts of MediaWiki over the coming months. Those will not be individually announced. Best -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

New query builders and IConnectionProvider
by Amir Sarabadani 02 May '23

02 May '23

Hello, As part of ongoing refactors and improvements to the Rdbms component in MediaWiki, we have made some changes that might affect your work. Read the February announcement <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…> for previous changes in this area. New DeleteQueryBuilder and UnionQueryBuilder We introduced the chainable SelectQueryBuilder <https://www.mediawiki.org/wiki/Manual:Database_access#SelectQueryBuilder> in 2020 to create SELECT queries. This "fluent" API style allows each segment of the query to be set through a discoverable method name with rich documentation and parameter types. Last February we introduced UpdateQueryBuilder in the same vein. The new DeleteQueryBuilder similarly supersedes use of IDatabase::delete() for building DELETE operations. Check change 913646 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/913646/1/includes/user/Us…> for how this is used in practice. Union queries are fairly rare in our codebases. Previously, one would have to bypass the query builders (by calling the underlying SQL text formatters like Database::selectSqlText) and pass several raw SQL strings through Database::unionQueries and then to Database::query. As you can see, this is not optimal. The new UnionQueryBuilder enables you to combine several SelectQueryBuilder objects and then use the familiar fetchResultSet() method on the resulting UnionQueryBuilder object. You can see an example of this in change 906101 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/906101/5/maintenance/find…> . IConnectionProvider typehint In February, we introduced the LBFactory->getPrimaryConnection() and LBFactory->getReplicaConnection() methods to improve ergonomics around getting database connections, and the IReadableDatabase typehint for replica connections. We're now introducing the IConnectionProvider typehint as a stable interface holding the above methods without the rest of LBFactory. Meaning, if you all you need is a connection, you can typehint to IConnectionProvider. This is an extremely narrow interface (four public methods!) compared to LBFactory (42 public methods). This will reduce coupling and should ease testing as well. We already adopted IConnectionProvider in several MediaWiki core components, which made a notable dent in MediaWiki's PhpMetrics complexity score <https://doc.wikimedia.org/mediawiki-core/master/phpmetrics/complexity.html> . This backwards-compatible change can be adopted through only a change in typehint. MediaWiki’s LBFactory service implements IConnectionProvider. Dependency injection remains unchanged. You can see an example in change 913649 <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DiscussionTools/+/913…>. We recommend changing variable names like $lbFactory to $dbProvider for consistency in usage. Future changes We will continue transforming MediaWiki core to adopt query builders, IConnectionProvider, and IReadableDatabase. And, we'll continue helping various core components and extensions to phase out their direct calls to LoadBalancer::getConnection(). This should improve code health, readability, testability, and overall coupling scores in MediaWiki. Our next announcement will introduce InsertQueryBuilder, and we're considering a query builder for "UPSERT" queries as well. Lastly, we are planning to discourage remaining use of raw SQL in "where" conditions by introducing an expression builder (T210206 <https://phabricator.wikimedia.org/T210206>). You can help Nearly every feature of MediaWiki requires retrieving or storing information from a database at one point or another. Reworking these is a long tail of effort. Any help in adopting query builders, and updating code away from ILoadBalancer and ILBFactory would be really appreciated. Together we can improve readability and code health (now with less footguns!). Until the next update, -- Amir Sarabadani (he/him), Staff Database Architect Timo Tijhof, Principal Performance Engineer

1 0

2023-01 to 2023-04: daily user pageview statistic
by Dušan Kreheľ 01 May '23

01 May '23

Hi everyone. The daily user pageview statistic, between 2023-01 to 2023-04 and grouped by local wikis, are public: https://archive.org/details/2023-daily_user_pageviews The one bug is reported: https://phabricator.wikimedia.org/T335685 Dušan Kreheľ

1 0

Phabricator monthly statistics - 2023-04
by aklapper＠wikimedia.org 30 Apr '23

30 Apr '23

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2023-04): 296 Active Maniphest users (any activity) in (2023-04): 1109 Task authors in (2023-04): 562 Users who have closed tasks in (2023-04): 306 Projects which had at least one task moved from one column to another on their workboard in (2023-04): 283 Tasks created in (2023-04): 1921 Tasks closed in (2023-04): 1999 Open and stalled tasks in total: 52957 * Only open tasks in total: 51955 * Only stalled tasks in total: 1002 Median age in days of open tasks by priority: Unbreak now: 13 Needs Triage: 844 High: 1117 Normal: 1845 Low: 2519 Lowest: 2700 (How long tasks have been open, not how long they have had that priority) Active Differential users (any activity) in (2023-04): 5 To see the names of the most active task authors: * Go to https://wikimedia.biterg.io/ * Choose "Phabricator > Overview" from the top bar * Adjust the time frame in the upper right corner to your needs * See the author names in the "Submitters" panel TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on phab1004 at Mon 01 May 2023 12:00:22 AM UTC)

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2023