There are still dozens of VMs in cloud-vps running Debian Stretch. All
of these hosts will need to be deleted and replaced with VMs running
either Buster or Bullseye in the next few months. Beginning in May we
will begin to shut down Stretch instances.
Please check this page for your projects, and take whatever steps are
necessary to move off of Stretch:
Don't hesitate to reach out for help on IRC or mailing list if you need
help with this migration. You may find Cinder volumes
especially useful for transferring data between VMs. Note that we do NOT
recommend in-place OS upgrades of VMs; it is almost always better to
start with a fresh host and transfer workloads over.
Details about the what and why for this process can be found here:
Here is the remaining deprecation timeline:
May 1, 2022: All active Stretch VMs will be shut down (but not deleted)
by WMCS admins.
June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will
be deleted by WMCS admins
Next week (Tuesday, 2022-04-19 15:00 UTC) we will be upgrading the
operating system that hosts the shared Toolsdb servers. This upgrade may
take an hour or more, during which time the databases will not be available.
This outage will be VERY DISRUPTIVE to many toolforge tools, as all
database access will fail during the upgrade. Toolforge users may want
to disable tools before the outage and/or check in to verify proper
recovery after service is restored.
There is likely to be a similar (or longer) outage in subsequent weeks
as we also need to upgrade the database servers themselves. Toolsdb has
grown to an ungainly size and can't be easily handled using standard
rolling upgrade procedures; the WMCS team is in ongoing discussions
about long-term solutions for this issue. In the meantime you can help
us out by engaging in periodic cleanup of your database usage and
dumping or dropping data that's no longer of use.
-Andrew + the WMCS team
Today 2022-04-06 we're performing some network maintenance operations on
Cloud VPS that could affect all cloud egress/ingress traffic, including
Toolforge. The cuts, if noticeable, should last a few minutes at most.
Some operations were also conducted yesterday (without this email
notice), and some unexpected hiccups occurred. That's why the email today.
Arturo Borrero Gonzalez
Site Reliability Engineer
Wikimedia Cloud Services