Hello, in the next coming months, these changes will happen in databases
and the infrastructure. And it might affect you if you rely on them in your
tools or queries. This list is ordered based on how soon the change will
happen.
We understand that updating your tools and systems can be time consuming,
hence we are giving an advanced notice. I truly apologize for the
inconvenience but many of these changes are needed to keep the site running
smoothly.
Image table redesign
Around fourteen years after the creation of T28741
<https://phabricator.wikimedia.org/T28741>, we are implementing the changes
described therein. Currently, every current version of an image has a row
in the image table and if there are older versions of that file, those rows
could be found in the oldimage table. These two tables (image and oldimage)
will be dropped in around two months. The replacement will be two main
tables: file and filerevision. Every file will have a row in the file table
describing the name and the type. Every version of the file (current and
old) will have a row in filerevision describing the file-specific
information such as its size or the hash of the file, similar to the
existing distinction between pages and revisions. Another improvement is
that every file and file revision will get a unique auto increment id
simplifying many operations and queries. You can check T28741
<https://phabricator.wikimedia.org/T28741> for more information. The new
tables are already accessible in wikireplicas but the data hasn’t been
fully migrated yet.
Term store split out of wikidata’s database
Wikidata’s database has been growing too fast and we need to move the term
store (tables starting with wbt_) to a dedicated cluster to allow growth
and improve wikidata’s performance by utilizing cache locality. The new
section will be called x3 and you will be able to access it in wikireplicas
but this also means you won’t be able to join these tables with the rest of
wikidata’s database (such as page table) since they will be residing in two
physically separate servers that also means most of your queries to
wikidata’s database (and term store) will become faster. We are aiming for
the switch to happen in three months’ time. You can follow the work in
T351820 <https://phabricator.wikimedia.org/T351820>.
Additionally, wb_type table will be dropped and the mapping will be
hard-coded in the code instead. See gerrit:1110810
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110810>
for more details. This helped us simplify a lot of Wikibase code (example
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110720>).
Categorylinks normalization
Categorylinks is the next table in the series of links tables being
normalized via the linktarget table (parent ticket
<https://phabricator.wikimedia.org/T300222>, RFC
<https://phabricator.wikimedia.org/T222224>). Similar to templatelinks and
pagelinks tables, cl_to will be dropped and instead the new field
cl_target_id will point to lt_id in the linktarget table. We will also drop
the cl_collation field and replace it with cl_collation_id which will point
to the collation_id field on the new table we are introducing called
collation. We are aiming to get this fully done by the end of the next
quarter (end of June 2025) but it depends on how fast the migration script
can operate and that’s outside of our control. You can follow the work in
T299951 <https://phabricator.wikimedia.org/T299951>.It’s worth noting that
after this migration is done, we will start working on the imagelinks table.
Thank you
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi,
I am experiencing very slow https file transfer from the instance
mediawiki2latex in the project collection-alt-renderer to my local
computer in Germany. Transfers from upload.wikimedia.org to my local
computer still work quite fast. I already rebooted the instance but it
didn't help. Has anyone got an idea what is going wrong?
Yours Dirk
Hello,
there will be a network maintenance operation happening on Thursday 2025-03-27
12:30 UTC that may impact all Wikimedia Cloud Services, including:
* Cloud VPS
* Toolforge
* PAWS
* and any others
The planned operation window will last for 1h, from 12:30 UTC to 13:30 UTC.
We have prepared the required changes and we are expecting no outages. But the
operation is sensitive and we would like you to be informed of potential service
disruptions as a result of the operations.
If you want more technical details, we are using a Phabricator ticket:
https://phabricator.wikimedia.org/T389958
regards.
Next week I'll be performing a routine openstack upgrade, moving us to
version 2024.2 'dalmation'.
This is a minor change; apart from a few horizon fixes the change should
not be noticeable. There will be a short Horizon outage and a network
reset during the upgrade but users should be otherwise unaffected.
I'll start the upgrade at 14:00 UTC.
-Andrew
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello,
If you have a Cloud VPS project and use opentofu to manage virtual resources,
this message is for your -- otherwise feel free to ignore. In particular,
Toolforge users are free to ignore this message.
The main Cloud VPS virtual network resource name is changing:
* from: lan-flat-cloudinstances2b
* to: VLAN/legacy
If your opentofu / terraform code mentions this network name, you will need to
update it.
See also Phabricator ticket: https://phabricator.wikimedia.org/T389942
regards.
Hello,
Are Wikidata QRank scores located at https://qrank.wmcloud.org/ still being hosted and supported by the WikiMedia Cloud Services? Currently, the site throws 500 errors and is preventing downloading the most recent set of QRank data. Is this a known intermittent issue, or is that data being permanently removed? If being removed, are there are other sources hosting that data?
Thanks,
David
Hello WMCS users,
I just wanted to follow up on my earlier email from 04 March. The Cloudelastic migration from Elastic -> OpenSearch is now complete as of 13 March[0] . As previously stated, this change should be invisible from a user perspective. That being said, please let us know if you are seeing any issues by responding here or in the ticket[1].
Best,
Brian King
SRE, Data Platform/Search Platform
Wikimedia Foundation
IRC: inflatador
[0] https://phabricator.wikimedia.org/T387904#10634555
[1] https://phabricator.wikimedia.org/T387904