Hello,
if you don't query wbc_entity_usage
<https://www.mediawiki.org/wiki/Wikibase/Schema/wbc_entity_usage> table,
feel free to ignore this message.
We are moving this table from each wiki's core database to the x1 cluster
instead. This would free up considerable storage space in core databases
(for example, around 16% of Wikimedia Commons database) and also would
allow us to increase the tracking granularity which in turn would increase
the accuracy of recentchanges entries being added from Wikidata to other
wikis.
In our survey of usage of this table in WMCS, only one bot has been brought
up as using this table. Since it's still considered used, we are announcing
that these tables won't exist in core databases anymore and will be moved
to a different cluster that has more headroom for growth. This change will
happen in one month.
The x1 cluster currently is not replicated to the cloud but we are working
on making it happen which would allow you to not just query this table but
many other requested tables that were not possible before (including but
not limited to CX tables <https://phabricator.wikimedia.org/T196020>,
globaljsonlinks <https://phabricator.wikimedia.org/T387419>, url shortener
table, and many more).
There might be a gap between drop of this table in core databases and x1
being available in wikireplicas. We are working to make sure that doesn't
happen but given the extremely low usage, even if it happens, the
disruption would be minimal.
You can follow the work of wbc_entity_usage migration in
https://phabricator.wikimedia.org/T438496 and the work of adding x1 to
wikireplicas in https://phabricator.wikimedia.org/T395881
Thank you and our apologies for the inconvenience.
--
*Amir Sarabadani* (he/him)
Staff Software Engineer
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello,
Here is an update on the ongoing work to move the links tables of
Commons to a dedicated cluster (x4).
On Monday, September 7th, we will switch production reads to the new x4
cluster. At this point, the x4 hosts are just replicas of the s4 hosts,
so the data served is identical.
On Tuesday, September 8th, we will take the x4 hosts out of that
replication and we will switch the writes to x4 as well. From that point
on, the affected tables will no longer be written to s4. They will
remain present in the wiki replicas, but will no longer receive any
updates. A wiki replica for x4 will be set up afterwards; this can't
happen earlier, as we do not have enough disk space to run replicas for
these tables on both clusters at the same time.
The following tables are affected by this move:
* linktarget
* externallinks
* pagelinks
* templatelinks
* categorylinks
* collation
* imagelinks
* globalimagelinks
* iwlinks
* existencelinks
* langlinks
The page and redirect tables will be present in both clusters.
Progress is tracked at: https://phabricator.wikimedia.org/T437108 and
https://phabricator.wikimedia.org/T437111
Background/rationale: https://phabricator.wikimedia.org/T398709
Thank you,
Alexander Vorwerk — IRC: Zabe
Reminder: this upgrade is coming up in about an hour.
On 8/17/26 2:12 PM, Andrew Bogott wrote:
> Next Wednesday we will up upgrading the Toolforge Kubernetes install
> to version 1.33.
>
> This upgrade should not result in any feature changes or downtime but
> there may be brief network resets and pods may be restarted and/or
> rescheduled.
>
> This upgrade is tracked here: https://phabricator.wikimedia.org/T433132
>
> -Andrew
>
Hello,
This is a friendly reminder that the next Toolforge Monthly Meeting is
coming up on Tuesday, August 25.
The Toolforge monthly meeting is a one-hour engineering-focused meeting to
discuss the current state and the future plans of Toolforge, to unblock
some decisions or to get everyone on the same page. It's also an
opportunity for technical contributors to connect with the Tools Platform /
Infrastructure teams and contribute to the ongoing and future work to
improve Toolforge.
The meeting agenda and joining information are available on the meeting
wiki page[0].
The current agenda includes:
-
Kubernetes upgrade workgroup progress
-
Push-to-Deploy updates
If there are other topics you'd like to discuss, please feel free to add it
to the agenda on the wiki before the meeting. We'd be happy to include
community-driven discussion topics.
Whether you're a long-time Toolforge maintainer or simply interested in the
platform, you're welcome to join the discussion.
We look forward to seeing you there!
[0]:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Monthly_meeting
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
Hi all,
The Toolforge Elasticsearch cluster is being replaced with a cluster
based on OpenSearch v2[0]. If your tool uses the writable
Elastic/OpenSearch functionality you will need to manually migrate
your tool to the new cluster as described in the rest of this email.
(Note that this is different from the "Cloud Elastic" service which
provides read-only access to the search indexes of Wikimedia wikis.)
The Wikitech page[0] has been updated with the details of the new
cluster, which are almost identical to the previous cluster except the
hostname and software version used. The same set of credentials will
also work on the new cluster. There are, however, some breaking API
changes between the old version and the new one, which is why we are
not just migrating all indexes for all tools at once.
To help migrate data from the old cluster to the new one, we have
temporarily added a `toolforge migrate-index` command to the bastion
servers to copy over an index and its mappings, aliases and
replication settings. The tool takes a space-separated list of indexes
to migrate, and will use credentials saved as envvars by default (and
prompt you if those are not available). Migration is relatively quick,
in my tests an index with about 400k records and 1+1 replication takes
about a minute to migrate.
The old Elasticsearch cluster will be shut down on 24 August.
This work is tracked as T401818.[1]
[0]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/OpenSearch
[1]: https://phabricator.wikimedia.org/T401818
--
Taavi Väänänen (he/they)
Site Reliability Engineer, Tools Infrastructure
Wikimedia Foundation
Hello,
This is a reminder that Debian Bullseye will reach the end of its Long Term
Support (LTS) on August 31, 2026[0]
If your Cloud VPS project still has instances running Debian Bullseye, now
is the time to migrate them to **Debian Bookworm or Debian Trixie**.
We have published a migration guide to help with the process.[1]
As part of this migration, it may also be worth evaluating whether your
workload is a good fit for Toolforge.
If feasible, migrating to Toolforge can reduce future maintenance burden,
as operating system upgrades are much simpler.
Our goal is to complete these migrations before Bullseye reaches end of
support. After the upstream LTS deadline,
Bullseye instances may become subject to shutdown or cleanup as part of the
deprecation process.
If you are unable to complete your migration before August 31, or need
assistance, please let us know as soon as possible.
In addition to replying on this mailing list or contacting the Tools
Platform team[2], please file a Phabricator task
tagged **Cloud-VPS (Debian Bullseye Deprecation[3])** so we can track your
project and work with you on a migration plan.
[0]: https://phabricator.wikimedia.org/T401804
[1]:
https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_instance_operating_syste…
[2]:
https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS#Communication_and_support
[3]: https://phabricator.wikimedia.org/project/view/8107/
Thanks!
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
Next Wednesday we will up upgrading the Toolforge Kubernetes install to
version 1.33.
This upgrade should not result in any feature changes or downtime but
there may be brief network resets and pods may be restarted and/or
rescheduled.
This upgrade is tracked here: https://phabricator.wikimedia.org/T433132
-Andrew
(If your third-party tool does not read data from [wbc_entity_usage] via
Wiki Replicas, please ignore this email)
Hello everyone,
planning is underway to move the Wikibase [wbc_entity_usage] tables for all
wikis to a different “*x1 extension cluster*” to improve database space
usage [0]. We expect that few, (if any) third-party tools directly rely on
this table — but before making any breaking changes, we want to confirm
that.
If your tool uses this table, hearing from you will help us understand how
many tools are affected and what that usage looks like, so we can plan the
migration accordingly.
[wbc_entity_usage] tracks which Wikidata entities are used on a given
Wiki’s pages, and also powers Wikidata-triggered Recent Changes and
Watchlist injections to client Wikis.
*// What's affected //*
If you operate a tool that reads from the [wbc_entity_usage] table —
through Wiki Replicas or directly from SQL, this may affect you.
This is especially likely if your query JOINs [wbc_entity_usage] with other
MediaWiki tables (e.g. ‘page’, ‘revision’), since cross-cluster JOINs will
stop working when the migration is completed.
*// What to do //*
Please comment on [1] (even a simple “yes, I am affected” is helpful). If
you can share more, we'd like to know:
-
The Tool name and location (e.g. Toolforge, repo link).
-
If your query JOIN’s [wbc_entity_usage] with other tables, please tell
us which.
-
How often the queries run against [wbc_entity_usage], (e.g. real-time,
daily, weekly, manually).
Please be aware:
-
[wbc_entity_usage] will eventually be removed from Wiki Replicas and
direct SQL. Existing queries against this table will need to be replaced
with a different approach.
-
During the lengthy transition period, the tables will still operate, so
existing queries will continue to function until access is turned off when
the migration is complete [2].
-
Once the Wiki Replicas access ends, tool users should switch over to
using the MediaWiki API [3] or Recent Changes API [4] to read changes
instead.
Queries on the table will no longer work *after* the migration is
completed, so they will need to be adjusted.
*// Links //*
[0] - https://phabricator.wikimedia.org/T424133
[2] - https://phabricator.wikimedia.org/T424440
[3] -
https://en.wikipedia.org/w/api.php?action=help&modules=query%2Bwbentityusage
[4] - https://www.mediawiki.org/wiki/API:RecentChanges
Please report if your tool is affected to the relevant Phabricator ticket:
[1] - https://phabricator.wikimedia.org/T430289
Thank you,
Danny (on behalf of the Wikidata For Wikimedia Projects
<https://meta.wikimedia.org/wiki/Wikidata_For_Wikimedia_Projects> team)
--
*Danny Benjafield*
Community Communications Manager
Wikidata For Wikimedia Projects
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de
Keep up to date! Current news and exciting stories about Wikimedia,
Wikipedia and Free Knowledge in our newsletter (in German): Subscribe now.
<https://www.wikimedia.de/newsletter/>
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Charlottenburg, VR 23855 B.
Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin,
Steuernummer 27/029/42207. Geschäftsführende Vorstände: Franziska Heine
This email relates to the shared /data/scratch mount used by toolforge
and some cloud-vps projects. It is provided to the following projects:
- dumps
- language
- maps
- math
- mwoffliner
- paws
- statanalyser
- testlabs
- tools
- toolsbeta
- video
- wikidata-query
- wikidumpparse
- wikiqlever
As part of our push to eliminate the use of Debian Bullseye, we will be
replacing the server that hosts the nfs scratch volume on Monday,
sometime around 15:00 UTC. Testing suggests that this will only produce
a few minutes of downtime for the volume (likely reported as 'Permission
Denied'); if you find your system frozen please give it a few minutes
and then reboot if waiting doesn't help.
The data on the old server will be transferred to the new server, so no
data will be lost.[0]
Bonus points: If you are a cloud-vps admin and have read this far,
please consider whether you still need the scratch volume. This service
was initially provided before we had attachable block storage
available[1], so many users rely on scratch not for file transfer
between hosts but because they needed extra storage space. If you are
one of those cases, please open a quota ticket[2] and mention that you
are moving off of scratch. Life without NFS is better for everyone!
This work is tracked on phabricator here:
https://phabricator.wikimedia.org/T433426
- Andrew + the WMCS team
[0] ...but if you have data on scratch that you care about, it should be
somewhere else! We make no reliability guarantees for this storage.
[1]
https://wikitech.wikimedia.org/wiki/Help:Adding_disk_space_to_Cloud_VPS_ins…
[2] https://phabricator.wikimedia.org/project/view/2880/
Hello all,
If you don't access (xmldata)dumps over NFS you can stop reading now.
*tl;dr make sure your Cloud VPS and Toolforge code accesses NFS dumps via
canonical path /public/dumps*
*Paths with hostnames hardcoded
(/mnt/nfs/dumps-clouddumps100[12].wikimedia.org <http://wikimedia.org>)
are deprecated and will stop working after Monday 28th September 2026*
Over the next few days Cloud VPS hosts will start mounting xmldatadumps at
/mnt/nfs/dumps. This move is part of the work to simplify dumps-nfs
operations (https://phabricator.wikimedia.org/T411248) and deprecate paths
with hardcoded hostnames.
What does this mean for your code?
If your code accesses xmldatadumps via /public/dumps symlinks in Cloud VPS
there is no action required. This location will be maintained as the public
and recommended interface to access xmldatadumps over NFS as per
https://wikitech.wikimedia.org/wiki/Help:Shared_storage#/public/dumps
If your Cloud VPS/Toolforge code uses paths with hostnames hardcoded
(/mnt/nfs/dumps-clouddumps1001.wikimedia.org and /mnt/nfs/
dumps-clouddumps1002.wikimedia.org) then please change it to use
/public/dumps/public instead. The legacy locations with hostnames will be
maintained as symlinks during a transition phase up until Monday September
28th
best,
Filippo
--
*Filippo Giunchedi*
Staff Site Reliability Engineer
Wikimedia Foundation <https://wikimediafoundation.org/>