(If you don’t work with links tables such as templatelinks, pagelinks and
so on, feel free to ignore this message)
TLDR: The schema of links tables (starting with templatelinks) will change
to have numeric id pointing to linktarget table instead of repeating
namespace and title.
The current schema and storage of most links tables are: page id (the
source), namespace id of the target link and title of the target. For
example, if a page with id of 1 uses Template:Foo, the row in the database
would be 1, 6, and Foo (Template namespace has id of 6)
Repeating the target’s title is not sustainable, for example more than half
of Wikimedia Commons database is just three links tables. The sheer size of
these tables makes a considerable portion of all queries slower, backups
and dumps taking longer and taking much more space than needed due to
unnecessary duplication. In Wikimedia Commons, on average a title is
duplicated around 100 times for templatelinks and around 20 times for
pagelinks. The numbers for other wikis depend on the usage patterns.
Moving forward, these tables will be normalized, meaning a typical row will
hold mapping of page id to linktarget id instead. Linktarget is a new table
deployed in production and contains immutable records of namespace id and
string. The major differences between page and linktarget tables are: 1-
linktarget values won’t change (unlike page records that change with page
move) 2- linktarget values can point to non-existent pages (=red links).
The first table being done is templatelinks, then pagelinks, imagelinks and
categorylinks will follow. During the migration phase both values will be
accessible but we will turn off writing to the old columns once the values
are backfilled and switched to be read from the new schema. We will
announce any major changes beforehand but this is to let you know these
changes are coming.
While the normalization of all links tables will take several years to
finish, templatelinks will finish in the next few months and is the most
So if you:
… rely on the schema of these tables in cloud replicas, you will need to
change your tools.
… rely on dumps of these tables, you will need to change your scripts.
Currently, templatelinks writes to both data schemes for new rows in most
wikis. This week we will start backfilling the data with the new schema but
it will take months to finish in large wikis.
You can keep track of the general long-term work in
https://phabricator.wikimedia.org/T300222 and the specific work for
templatelinks in https://phabricator.wikimedia.org/T299417. You can also
read more on the reasoning in https://phabricator.wikimedia.org/T222224.
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
[If you don't write gadgets, user scripts or work on MediaWiki code feel
free to ignore this message]
Given the hackathon this weekend, now seemed like a good idea to talk about
us having a policy for code we write for gadget and user scripts developers
and as gadget and user script developers. TDLR: I am proposing a policy to
guide developers and authors of these kinds of scripts. I would like people
to read through my first draft
 and give me feedback on the talk page. Please feel free to share on
*What is being proposed*
A policy page that would be editable on wiki and linked to from the editing
interfaces on gadgets, to guide both parties on how to write code that's
sustainable and less prone to breakage.
*Why is this needed?*
Despite gadgets and user scripts (which will be referred from now on as
wiki-based code) being a key component of MediaWiki projects, up until now
frontend APIs (e.g. how wiki-based code should interact with source control
provided code) have been ill-defined leading to misunderstandings between
engineers and wiki-based code developers when wiki-based code break. This
also leads to code rot, where developers do not feel empowered to make
changes as it's unclear how their changes will impact wiki-based code
developers. On top of this, when wiki-based code breaks it's not clear who
can and will fix them.
To solve this a policy I have been pushing for some time to make the
contract between MediaWiki developers and wiki-based code developers
explicit and less confusing.
I hope on the long run a policy would restore trust and good faith between
the two parties.
*How can you help?*
To contribute to the policy please use the discussion page to raise
concerns, suggestions, removals or additions.
*What's the deadline?*
I'd like to have all feedback gathered by 30th May 2022
Hi, this is a friendly reminder that we would love to hear from you
about your experience at last weekend's Hackathon.
Please fill in the survey until Sunday, May 29th and help us improve.
See the links below.
Thanks a lot in advance!
-------- Quoted Message --------
From: Haley Lepp
Date: Sun, 22 May 2022 11:13:39 -0700
On behalf of the 2022 Wikimedia Hackathon Committee, we would like to
thank you for coming to the Wikimedia Hackathon!
Please consider giving us feedback on the Hackathon and your
suggestions for improvement.
There are two ways to give feedback:
1. Fill out the Wikimedia Hackathon Survey
more information on privacy and data-handling, see the survey privacy
>. The survey will remain open until May 29, 2022.
2. If you would like to share feedback but do not wish to take the
Qualtrics survey, you can leave feedback on
the Etherpad <https://etherpad.wikimedia.org/p/Wikimedia_Hackathon_2022_Feedback
Finally, check out
the badges <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2022/How_to#Joining_a_se…
> the committee made. You can put them on your userpages to show your
Thank you again for joining us! It was so much fun to meet everyone and
See you at the Wikimania Hackathon in August!
Haley, on behalf of the
2022 Wikimedia Hackathon Team
Andre Klapper (he/him) | Bugwrangler / Developer Advocate
Wikitech-ambassadors mailing list -- wikitech-ambassadors(a)lists.wikimedia.org
To unsubscribe send an email to wikitech-ambassadors-leave(a)lists.wikimedia.org
This email is a summary of the Wikimedia production deployment of
- Conductor: Ahmon Dancy (That's my name, don't wear it out)
- Backup Conductor: Jaime Nuche
- Blocker Task: T305219 <https://phabricator.wikimedia.org/T305219>
- Status: Rolled out to all wikis on May 26, 2022
📈 According to our calculations
Sparklines comparing the last 5 trains.
- 299 Patches ▁▃▂█▆
- 0 Rollbacks ▁▁█▁▁
- 0 Days of delay ▁▁█▁▁
- 4 Blockers ▁▁█▃▂
🎉 Trainbow Love ✨Thanks to folks who reported or resolved blockers:
- Bartosz Dziewoński
- Abijeet Patro
- James D. Forrester
- Jon Robson
At the Wikimedia Hackathon 2022 that ended yesterday I have showed a program in the Showcase that can convert blocks from visual-programming-language Snap! to source code. This is the link to the folder where the program is located in. https://public.paws.wmcloud.org/User:Hog%C3%BC-456/BlocktoCode/ The program reads an XML-File with the definition of a program and gives the source code as an output. The platform for creating the blocks I used is called Snap!. This is an further development of Scratch. Scratch is an visual programming language based on blocks, that can be combined to create a program. A block is a small sentence with gaps for the variables. In Snap! it is possible to create own blocks and it includes an feature to directly convert blocks to code. I dont know how far this is developed and after I havent understand how to export the result with the code I have written a own program to do that.
What do you think are potential use cases for low code platforms like Snap! within the Wikimedia Projects. From my point of view such platforms offer a chance to make programming accessible to more people. It is from my point of view easier with such a platform to write small programs as without such an support. I am interested in use cases where the built-in codification feature or my program can be used to generate code that will be then useful within the Wikimedia projects.
Have a nice day and I am interested in your thoughts.
Just a brief notice that we're planning to take gitlab.wikimedia.org
down for around 2 hours this Thursday, June 2nd, at 15:00 UTC for a
migration to new physical hardware.
You can follow this work in Phabricator:
We're also available in #wikimedia-gitlab on libera.chat for any questions.
I'm happy to share that the first Web Perf Hero award of 2022 goes to Amir Sarabadani!
This award is in recognition of Amir's work (@Ladsgroup) over the past six months, in which he demonstrated deep expertise of the MediaWiki platform and significantly sped up the process of saving edits in MediaWiki. This improved both the potential of MediaWiki core, and as experienced concretely on WMF wikis, especially on Wikidata.org.
Refer to the below medawiki.org page to read about what it took to *cut latencies by half*:
This award is given on a quarterly basis, and manifests as a Phabricator badge:
-- Timo Tijhof, on behalf of WMF Performance Team.