Hello,
We have several more changes to the rdbms library’s interface to announce.
We previously made such announcements in May 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
and February 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
.
Query builders for all common query types
We now have query builders for all common query types:
-
SELECT
-
INSERT
-
UPDATE
-
DELETE
-
UPSERT
-
REPLACE
We encourage you to migrate calls using the old methods (such as
Database::insert) to use the query builders instead. Starting from 1.41,
all the old methods are considered internal and might change without prior
notice.
See T335377 <https://phabricator.wikimedia.org/T335377> and previous
announcements such as the one in May 2023
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
for more information.
Script for migrating to SelectQueryBuilder
Given that there are many calls to Database::select in extensions we wrote
a simple script to help with the migration to SelectQueryBuilder. It is not
perfect and doesn’t cover many aspects such as joins but it takes away a
lot of tedious work.
Using that script we made hundreds of migrations in the MediaWiki core. As
of now, we have more than 800 calls to newSelectQueryBuilder() and only
roughly 100 direct calls to Database::select() left in core.
For more information, check out
https://gitlab.wikimedia.org/ladsgroup/migrateselect
Access to external clusters simplified
If you ever had to make database queries against external cluster databases
(such as extension1, otherwise known as x1), you will like this change.
Many of our extensions store their data in x1, for example: Echo,
UrlShortener, Cognate, Translate, ContentTranslation, GrowthExperiments,
Campaigns and many more. And it requires a lot of complexity with many
caveats (see T330590 <https://phabricator.wikimedia.org/T330590> for more
information).
Now, an extension can introduce one or more virtual domains in their
extension.json and then if you set the configuration such as:
$wgVirtualDomainsMapping['urlshortener'] = [ 'cluster' => 'extension1',
'db' => 'wikishared' ];
then $lbf->getPrimaryDatabase( 'urlshortener' ) will make the right
connection. If it’s not set in the mapping of virtual domains
configuration, it simply makes the connection to the local database. You
don’t need to do any special coding anymore which simplifies the logic a
lot and removes the need for extra configuration variables. This also helps
in facilitating proper support of external clusters in the database updater
and improvements in testing in the future.
You can take a look at the example of url shortener (gerrit:963293
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UrlShortener/+/963293>)
for how adoption is done.
Thank you,
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi friends,
GitLab will be down this coming Thursday, 5th October from 09:00-13:00
UTC for the datacenter switchover[0].
Much of this switchover process will result in downtime, during which
you will be unable to access GitLab through ssh or the web interface.
We hope to be completed in less time than the window, but the service
should not be relied upon during the time stated above.
If this presents a problem for you or your team, please let us know.
You can follow along on IRC in the libera.chat #wikimedia-gitlab channel[1].
--Eoghan
[0]: <https://phabricator.wikimedia.org/T345531>
[1]: <ircs://irc.libera.chat:6697/wikimedia-gitlab>
Deployment train for 1.41.0-wmf.29 of MediaWiki is blocked[0].
The new version is deployed to group0 wikis[1]. But it can not continue
until these issues are resolved:
- OutputPageParserOutput hook to mutate TOC deprecated in MediaWiki 1.41
– <https://phabricator.wikimedia.org/T348134>
- Note: deprecation warnings do not usually block deploys, but this
caused 10s of thousands of warnings, flooding the logs
- TypeError: CirrusShowSearchHitHandler::newFromGlobalState() – <
https://phabricator.wikimedia.org/T348181>
- New error in 1.29.0-wmf.29, creating a high error rate on commons and
wikidata
Once these issues are resolved, the train can resume. If these issues are
resolved on a Friday the train will resume Monday.
Thank you for your help resolving these issues!
Tyler Cipriani (he/him)
Engineering Manager, Release Engineering
Wikimedia Foundation
[0]: <https://phabricator.wikimedia.org/T347080>
[1]: <https://versions.toolforge.org/>
Yesterday there was a conversation about code review on irc and among other
things, how sometimes patches can get "stuck".
I had an idea for a way to improve things. I'm not sure if it is a good
idea, but there's only one way to find out.
So without further ado, announcing the Code Review Patch Board:
https://www.mediawiki.org/wiki/Code_review/patch_board
In short - each person is allowed to list one of their patches on the board
that they would really like to see reviewed. You can only list one patch at
a time, and it should be a patch that you have been unable to get review
for for at least a week through normal means. See the page for the full
list of guidelines.
I encourage people to give it a try. Add a patch you wrote that you cannot
get a review for. Or if you have +2 rights, try giving some love to these
underloved patches.
I would also love to hear feedback on the general idea as well as the
current guidelines.
To repeat, the url is:
https://www.mediawiki.org/wiki/Code_review/patch_board
Thanks,
bawolff
Hello all!
The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, October 4, 2023
Time: 15:00-16:00 UTC / 08:00 PT / 11:00 EDT / 17:00 CET
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
Have fun and see you soon!
Guillaume
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>
Hey Hey Everyone,
We're thrilled to share an update on Outreachy Round 27: The project and mentor selection phase has successfully concluded, and we're excited to introduce 7 projects guided by a talented team of 16 mentors. If you know someone who’s eligible for the Outreachy contribution phase, encourage them to explore the Wikimedia projects here: https://www.outreachy.org/apply/project-selection/#wikimedia
If you're interested in these projects, you can stay updated by subscribing to the Phabricator tickets or share your ideas by commenting.
Learn more here: www.mediawiki.org/wiki/Outreachy/Round_27
Regards,
Onyinye & Sheila
Hello,
There is now a newsletter for the Wikimedia apps you can subscribe to:
https://meta.wikimedia.org/wiki/Wikimedia_Apps/Newsletter
Now, you can stay updated on all the latest news, updates, and details of
our ongoing projects.
Respectfully,
*Amal Ramadan* (She\Her)
Sr. Community Relations Specialist
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi all! This is an announcement for a new developer feature in MediaWiki.
If you don’t develop MediaWiki core, extensions or skins, you can stop
reading :)
MediaWiki interface messages are generally “safe” to edit: when they
contain markup, it is either parsed (as wikitext), sanitized, or fully
HTML-escaped. For this reason, administrators are allowed to edit normal
messages on-wiki in the MediaWiki: namespace, while editing JS code (which
is more dangerous) is restricted to interface administrators. (A few
exceptions, messages that are not escaped and which can only be edited by
interface administrators, are configured in $wgRawHtmlMessages
<https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:$wgRawHtmlMessages>.)
Occasionally, a bug in the software means that a message isn’t properly
escaped, which can in theory be abused by administrators to effectively
gain interface administrator powers (by editing a MediaWiki: page for a
message to contain <script> tags, or onclick="" attributes, or whatever).
Such bugs are usually considered low-severity security issues; some of them
are tracked in T2212 <https://phabricator.wikimedia.org/T2212>. (The
general issue is known as cross-site scripting
<https://www.mediawiki.org/wiki/Special:MyLanguage/Cross-site_scripting>
and can be much more severe when it’s not limited to interface messages.)
Previously, checking for these issues as a developer was tedious: if you
suspected that a message was vulnerable to HTML injection, you had to
create a page for it in the MediaWiki: namespace, or edit the corresponding
en.json file on disk (and potentially rebuild the localisation cache). The
recently merged “xss language code” feature simplifies this process. If the
developer setting $wgUseXssLanguage is set to true, then an “x-xss”
language code becomes available and can be selected with *?uselang=x-xss*
in the URL. When using this language code, all messages become “malicious”:
every message is replaced by a snippet of HTML that tries to run alert('
*message-key*'). If everything is implemented correctly, all of those HTML
snippets should be escaped, and no alerts should fire, although the wiki
will look quite strange:
If you see any alert, then that means that a message has not been escaped
correctly; use the message key shown in the alert to hunt down the buggy
code (or add the message key to $wgRawHtmlMessages). This feature is
intended to be especially useful during code review: check out the change,
load a page with ?uselang=x-xss, and see if any alerts come up.
Miscellaneous notes:
- This is a developer-only feature. I strongly recommend against
setting $wgUseXssLanguage
= true; in any production setting. (It will be added to
DevelopmentSettings.php soon.)
- Above, I focused on the possibility to abuse unescaped messages via
the MediaWiki: namespace. You might also be thinking about the potential
for translatewiki.net contributors to inject malicious HTML into message
translations; however, the translation exports from translatewiki.net to
the JSON files automatically check for any HTML in translations, and flag
suspicious cases for human review. Therefore, it’s much harder to exploit
an unescaped message via translatewiki.net than via the MediaWiki:
namespace.
- Finally, I should mention that we already found several
vulnerabilities using this feature, which will be fixed with the upcoming
security release
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>.
If you try out this feature now, and find a vulnerable message, I suggest
you wait until then, and check whether it’s still vulnerable, before
reporting it.
Cheers,
Lucas
--
Lucas Werkmeister (he/er)
Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
Steuernummer 27/029/42207.
Hi Community Metrics team,
This is your automatic monthly Phabricator statistics mail.
Accounts created in (2023-09): 226
Active Maniphest users (any activity) in (2023-09): 997
Task authors in (2023-09): 551
Users who have closed tasks in (2023-09): 321
Projects which had at least one task moved from one column to another on
their workboard in (2023-09): 295
Tasks created in (2023-09): 2399
Tasks closed in (2023-09): 2126
Open and stalled tasks in total: 53676
* Only open tasks in total: 52659
* Only stalled tasks in total: 1017
Median age in days of open tasks by priority:
Unbreak now: 13
Needs Triage: 893
High: 1235
Normal: 1940
Low: 2570
Lowest: 2866
(How long tasks have been open, not how long they have had that priority)
Differential users who created or updated a patchset in (2023-09): 0
To see the names of the most active task authors:
* Go to https://wikimedia.biterg.io/
* Choose "Phabricator > Overview" from the top bar
* Adjust the time frame in the upper right corner to your needs
* See the author names in the "Submitters" panel
TODO: Numbers which refer to closed tasks might not be correct, as
described in https://phabricator.wikimedia.org/T1003 .
Yours sincerely,
Fab Rick Aytor
(via community_metrics.sh on phab1004 at Sun 01 Oct 2023 12:00:19 AM UTC)