Debian Stretch's security support ends in mid 2022, and the Foundation's
OS policy already discourages use of existing Stretch machines. That
means that it's time for all project admins to start rebuilding your VMs
with Bullseye (or, if you must, Buster.)
Any webservices running in Kubernetes created in the last year or two
are most likely using Buster images already, so there's no action needed
for those. Older kubernetes jobs should be refreshed to use more modern
images whenever possible.
If you are still using the grid engine for webservices, we strongly
encourage you to migrate your jobs to Kubernetes. For other grid uses,
watch this space for future announcements about grid engine migration;
we don't yet have a solution prepared for that.
Details about the what and why for this process can be found here:
https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation
Here is the deprecation timeline:
March 2021: Stretch VM creation disabled in most projects
July 6, 2021: Active support of Stretch ends, Stretch moves into LTS
<- You are Here ->
January 1st, 2022: Stretch VM creation disabled in all projects,
deprecation nagging begins in earnest. Stretch alternatives will be
available for tool migration in Toolforge
May 1, 2022: All active Stretch VMs will be shut down (but not deleted)
by WMCS admins. This includes Toolforge grid exec nodes.
June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will
be deleted by WMCS admins
Hello, all!
We are in the process of re-engineering and virtualizing[0] the NFS
service provided to Toolforge and VMs. The transition will be rocky and
involve some service interruption... I'm still running tests to
determine exactly host much disruption will be required.
The first volume that I'd like to replace is 'scratch,' typically
mounted as /mnt/nfs/secondary-scratch. I'm seeking feedback about how
vital scratch uptime is to your current workflow, and how disruptive it
would be to lose data there.
If you have a project or tool that uses scratch, please respond with
your thoughts! My preference would be to wipe out all existing data on
scratch and also subject users to several unannounced periods of
downtime, but I also don't want anyone to suffer. If you have
important/persistent data on that volume then the WMCS team will work
with you to migrate that data somewhere safer, and if you have an
important workflow that will break due to Scratch downtime then I'll
work harder on devising a low-impact roll-out.
Thank you!
-Andrew
[0] https://phabricator.wikimedia.org/T291405
Hello cloud-vps users,
There are still about 84 unclaimed projects at
https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2021_Purge
Please take a moment to look at that page and mark projects that you are
using.
Unclaimed projects will be in danger of shutdown on February 1st, 2022.
Thank you to those of you who have already acted on this.
Thank you!
- Komla
-------- Forwarded Message --------
Subject: Cloud VPS users, please claim your projects (and, introducing
Komla)
Date: Thu, 2 Dec 2021 14:42:08 -0600
From: Andrew Bogott <abogott(a)wikimedia.org> <abogott(a)wikimedia.org>
Reply-To: abogott(a)wikimedia.org
Organization: The Wikimedia Foundation
To: Cloud-announce(a)lists.wikimedia.org
CC: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>
<wikitech-l(a)lists.wikimedia.org>
Hello cloud-vps users!
It's time for our annual cleanup of unused projects and resources. Our new
developer advocate Komla Sapaty will be guiding this process; please
respond promptly to his emails and do your best to make him feel welcome!
Every year or so the Cloud Services team tries to identify and clean up
unused projects and VMs. We do this via an opt-in process: anyone can mark
a project as 'in use,' and that project will be preserved for another year.
I've created a wiki page that lists all existing projects, here:
https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2021_Purge
If you are a VPS user, please visit that page and mark any projects that
you use as {{Used}}. Note that it's not necessary for you to be a project
admin to mark something -- if you know that you're currently using a
resource and want to keep using it, go ahead and mark it accordingly. If
you /are/ a project admin, please take a moment to mark which VMs are or
aren't used in your projects.
When February arrives, I will shut down and begin the process of reclaiming
resources from unused projects.
If you think you use a VPS project but aren't sure which, I encourage you
to poke around on https://tools.wmflabs.org/openstack-browser/ to see what
looks familiar. Worst case, just email cloud(a)lists.wikimedia.org with a
description of your use case and we'll sort it out there.
Exclusive toolforge users are free to ignore this email and future related
things.
Thank you!
-Andrew and the WMCS team