[Wikitech-l] Three new rdbms changes

Hello,

We have several more changes to the rdbms library’s interface to announce. We previously made such announcements in May 2023 https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GATRFC63FPQJ6Z43WZDIKSHYZQJCI62H/ and February 2023 https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/YNLVJVTYTK3IGQF4HY7ITGMIJ2W5Q7CG/ . Query builders for all common query types

We now have query builders for all common query types:

-

SELECT -

INSERT -

UPDATE -

DELETE -

UPSERT -

REPLACE

We encourage you to migrate calls using the old methods (such as Database::insert) to use the query builders instead. Starting from 1.41, all the old methods are considered internal and might change without prior notice.

See T335377 https://phabricator.wikimedia.org/T335377 and previous announcements such as the one in May 2023 https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/GATRFC63FPQJ6Z43WZDIKSHYZQJCI62H/ for more information. Script for migrating to SelectQueryBuilder

Given that there are many calls to Database::select in extensions we wrote a simple script to help with the migration to SelectQueryBuilder. It is not perfect and doesn’t cover many aspects such as joins but it takes away a lot of tedious work.

Using that script we made hundreds of migrations in the MediaWiki core. As of now, we have more than 800 calls to newSelectQueryBuilder() and only roughly 100 direct calls to Database::select() left in core.

For more information, check out https://gitlab.wikimedia.org/ladsgroup/migrateselect Access to external clusters simplified

If you ever had to make database queries against external cluster databases (such as extension1, otherwise known as x1), you will like this change. Many of our extensions store their data in x1, for example: Echo, UrlShortener, Cognate, Translate, ContentTranslation, GrowthExperiments, Campaigns and many more. And it requires a lot of complexity with many caveats (see T330590 https://phabricator.wikimedia.org/T330590 for more information).

Now, an extension can introduce one or more virtual domains in their extension.json and then if you set the configuration such as:

$wgVirtualDomainsMapping['urlshortener'] = [ 'cluster' => 'extension1', 'db' => 'wikishared' ];

then $lbf->getPrimaryDatabase( 'urlshortener' ) will make the right connection. If it’s not set in the mapping of virtual domains configuration, it simply makes the connection to the local database. You don’t need to do any special coding anymore which simplifies the logic a lot and removes the need for extra configuration variables. This also helps in facilitating proper support of external clusters in the database updater and improvements in testing in the future.

You can take a look at the example of url shortener (gerrit:963293 https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UrlShortener/+/963293) for how adoption is done.

Thank you,

-- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation https://wikimediafoundation.org/