Debian Stretch's security support ends in mid 2022, and the Foundation's
OS policy already discourages use of existing Stretch machines. That
means that it's time for all project admins to start rebuilding your VMs
with Bullseye (or, if you must, Buster.)
Any webservices running in Kubernetes created in the last year or two
are most likely using Buster images already, so there's no action needed
for those. Older kubernetes jobs should be refreshed to use more modern
images whenever possible.
If you are still using the grid engine for webservices, we strongly
encourage you to migrate your jobs to Kubernetes. For other grid uses,
watch this space for future announcements about grid engine migration;
we don't yet have a solution prepared for that.
Details about the what and why for this process can be found here:
https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation
Here is the deprecation timeline:
March 2021: Stretch VM creation disabled in most projects
July 6, 2021: Active support of Stretch ends, Stretch moves into LTS
<- You are Here ->
January 1st, 2022: Stretch VM creation disabled in all projects,
deprecation nagging begins in earnest. Stretch alternatives will be
available for tool migration in Toolforge
May 1, 2022: All active Stretch VMs will be shut down (but not deleted)
by WMCS admins. This includes Toolforge grid exec nodes.
June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will
be deleted by WMCS admins
Next Tuesday we'll be performing a routine upgrade of the OpenStack
software in cloud-vps.
During the upgrade, Horizon will be disabled and many OpenStack API
calls will fail. VMs and Tools will be largely unaffected, but midway
through the upgrade there will be a very brief network interruption
which will definitely reset all network connections and may also cause
timeouts.
The entire upgrade should take around two hours.
-Andrew + the WMCS team
Hello!
A friendly reminder that the feedback period ends on the 21st of August.
Please spare 5-10 minutes to leave feedback[0] on the Toolhub taxonomy[1].
Toolhub[2] is a catalog of 1500+ tools used by a wide range of Wikimedia
contributors: editors, developers, patrollers, researchers, admins and more.
We want to make finding and categorizing these tools as easy as possible.
The taxonomy is at the heart of how tool search works, and your feedback
would help improve it.
Whether you are a current user of Toolhub or hearing about it for the first
time doesn't matter – your input is valuable and much appreciated either
way!
=== How To Provide Feedback ===
Use the discussion page[3] of the feedback page to provide your responses
to the questions.
You will find more details on the feedback page.
=== Implementation ===
At the end of the feedback round, the team will evaluate and work on the
necessary improvements.
This is expected to be completed by the end of September 2022.
[0]: https://meta.wikimedia.org/wiki/Toolhub/Data_model/Feedback
[1]: https://meta.wikimedia.org/wiki/Toolhub/Data_model#Taxonomy_v2
[2]: https://toolhub.wikimedia.org/
[3]: https://meta.wikimedia.org/wiki/Talk:Toolhub/Data_model/Feedback
Thanks
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
Thanks to a profound disagreement between fingers and brain, I've just
accidentally rebooted several cloudvirts and they're associated VMs.
Everything will be back up shortly, and no action is needed from users
unless you have services that need manual restarting.
Sorry for the noise!
-Andrew
Hi everyone!
TL;DR;
Currently there's a degradation on the service for VMs and anything running on them (ex. toolforge, quarry, paws,
...), you might be able to use the services or they might become too slow, we are working on it and will update when
fixed.
Long story:
We were adding a new ceph node to the ceph cluster. This time the node was in a different subnet, but ceph is supposed
to be transparently able to work with many subnets. For some reason the new node was added to the cluster, but it's
missing to reply to any heartbeats sent from any other nodes in the cluster and that causes the cluster to keep
rebalancing data around, what creates a continuous IO slowness for any clients (like VMs).
We are trying to minimize the impact by limiting the amount of data that gets re-shuffled, that slows down the
intervention a bit, but should improve the client experience.
We are actively working on this, and will update with any changes.
Cheers!
--
David Caro
SRE - Cloud Services
Wikimedia Foundation <https://wikimediafoundation.org/>
PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3
"Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment."
Hello!
This is a friendly reminder for you to spare 5-10 minutes to leave
feedback[0] on the Toolhub taxonomy[1].
The period for providing feedback ends on August 21, 2022.
Toolhub[2] is a catalog of 1500+ tools used by a wide range of Wikimedia
contributors: editors, developers, patrollers, researchers, admins and more.
We want to make finding and categorizing these tools as easy as possible.
The taxonomy is at the heart of how tool search works, and your feedback
would help improve it.
Whether you are a current user of Toolhub or hearing about it for the first
time doesn't matter – your input is valuable and much appreciated either
way!
=== How To Provide Feedback ===
Use the discussion page[3] of the feedback page to provide your responses
to the questions.
You will find more details on the feedback page.
=== Implementation ===
At the end of the feedback round, the team will evaluate and work on the
necessary improvements.
This is expected to be completed by the end of September 2022.
[0]: https://meta.wikimedia.org/wiki/Toolhub/Data_model/Feedback
[1]: https://meta.wikimedia.org/wiki/Toolhub/Data_model#Taxonomy_v2
[2]: https://toolhub.wikimedia.org/
[3]: https://meta.wikimedia.org/wiki/Talk:Toolhub/Data_model/Feedback
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
Hi all,
We've added Node.js version 16 as an available image[0] to Toolforge
Kubernetes. Note that you might need to delete and re-create the
node_modules directory with the newer runtime for it to work properly.
[0]:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes#Container_ima…
-- Taavi Väänänen, on behalf of the entire Toolforge admin team
Hello!
The Toolhub project team wants your feedback[0] on the Toolhub taxonomy[1].
Toolhub[2] is a catalog of 1500+ tools used by a wide range of Wikimedia
contributors: editors, developers, patrollers, researchers, admins and more.
We want to make finding and categorizing these tools as easy as possible.
The taxonomy is at the heart of how tool search works, and your feedback
would help improve it.
Whether you are a current user of Toolhub or hearing about it for the first
time doesn't matter – your input is valuable and much appreciated either
way!
Please take 5-10 minutes to leave feedback.
=== How To Provide Feedback ===
Use the discussion page[3] of the feedback page to provide your responses
to the questions.
You will find more details on the feedback page.
The period for providing feedback ends on August 21, 2022.
=== Implementation ===
At the end of the feedback round, the team will evaluate and work on the
necessary improvements.
This is expected to be completed and announced by the end of September 2022.
[0]: https://meta.wikimedia.org/wiki/Toolhub/Data_model/Feedback
[1]: https://meta.wikimedia.org/wiki/Toolhub/Data_model#Taxonomy_v2
[2]: https://toolhub.wikimedia.org/
[3]: https://meta.wikimedia.org/wiki/Talk:Toolhub/Data_model/Feedback
Thanks
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services