Cloud-announce January 2023

cloud-announce@lists.wikimedia.org

3 participants
7 discussions

Upcoming shutdown of osmdb.eqiad.wmnet
by Andrew Bogott 01 Feb '23

01 Feb '23

The hardware that hosts osmdb.eqiad.wmnet is long past its end of life and will be shut down on February 12th. WMF staff do not plan to support that database after that date, and the domain will be shut down. I am pretty sure that we have already made arrangements with all current users of the service, but I'm sending this email out of an abundance of caution. If you think you are using it, please chime in on the associated phabricator ticket[0] so that we know you exist! There is a volunteer-maintained replacement that you should be able to switch to with a minimum of effort. Thanks for reading! -Andrew + the WMCS team [0] https://phabricator.wikimedia.org/T323159

1 0

PAWS k8s upgrade 2023-01-31
by Vivian Rook 28 Jan '23

28 Jan '23

PAWS will be upgrading to k8s 1.22 on 2023-01-31 If you were running a workload at that time it will need to be restarted. -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

New toolforge-jobs features
by Arturo Borrero Gonzalez 24 Jan '23

24 Jan '23

Hi there, The Toolforge jobs framework just got upgraded with a few new features: * support for custom logs * support for job failure retry policy * new behavior with job image listing * some initial validation of YAML files The documentation should be mostly up-to-date in wikitech: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework You can stop reading here unless you want more details :-) The custom log files feature will allow you do things like: * using a custom directory to store log files * merging stdout/stderr logs together into a single file * ignoring one of the two log streams The job retry policy allows to instruct the computing engine to restart jobs that failed, up to 5 times. Job images are now listed in a different format, and deprecated images are hidden by default, to encourage usage of newer ones. Regarding the YAML validation, the toolforge-jobs utility will now emit a warning if some key is unknown. We plan to make this more robust in the future, also providing a schema file. We don't usually announce upgrades, but this one in particular contained much awaited features. This is the result of hard work by several folks, in particular Taavi (community member) and Raymond (WMF contractor). Happy `toolforging`. Regards. -- Arturo Borrero Gonzalez Senior SRE / Wikimedia Cloud Services Wikimedia Foundation

1 0

openstack + Horizon upgrades Monday, 2022-11-14
by Andrew Bogott 18 Jan '23

18 Jan '23

I will be upgrading the cloud-vps openstack install on Monday afternoon my time (beginning around 18:00 UTC). Here's what to expect: - Intermittent Horizon and API downtime (maybe an hour or two total) - Inability to schedule new VMs (also for an hour or two) - Some mild Horizon dashboard changes as I'll also be upgrading the dashboards to version 'Zen'. Toolforge users will be unaffected by this outage. Existing, running services and VMs on cloud-vps should also be unaffected. -Andrew + the WMCS team

1 4

Cloud VPS users, please claim your projects
by Andrew Bogott 12 Jan '23

12 Jan '23

Hello cloud-vps users! It's time for our annual cleanup of unused projects and resources. Every year or so the Cloud Services team tries to identify and clean up unused projects and VMs. We do this via an opt-in process: anyone can mark a project as 'in use,' and that project will be preserved for another year. I've created a wiki page that lists all existing projects, here: https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2022_Purge If you are a VPS user, please visit that page and mark any projects that you use as {{Used}}. Note that it's not necessary for you to be a project admin to mark something -- if you know that you're currently using a resource and want to keep using it, go ahead and mark it accordingly. If you /are/ a project admin, please take a moment to mark which VMs are or aren't used in your projects. When February arrives, I will shut down and begin the process of reclaiming resources from unused projects. If you think you use a VPS project but aren't sure which, I encourage you to poke around on https://tools.wmflabs.org/openstack-browser/ to see what looks familiar. Worst case, just email cloud(a)lists.wikimedia.org with a description of your use case and we'll sort it out there. Exclusive toolforge users are free to ignore this email. Thank you! -Andrew and the WMCS team

1 2

Toolforge jobs: briefly maintenance today 2023-01-10 @ 11:30 UTC
by Arturo Borrero Gonzalez 10 Jan '23

10 Jan '23

Hi there, the Toolforge jobs service [0] (the one you would use via the `toolforge-jobs` command line interface) will have a brief maintenance today 2023-01-10 @ 11:30 UTC (in about 15 minutes). We need to restart the API service and it will be down for a couple of minutes (perhaps even less). During that time, using the toolforge-jobs command line interface will most likely fail. regards. [0] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework -- Arturo Borrero Gonzalez Senior SRE / Wikimedia Cloud Services Wikimedia Foundation

1 0

PAWS moving k8s clusters 2023-01-17
by Vivian Rook 09 Jan '23

09 Jan '23

On Tuesday 2023-01-17 PAWS will be moving k8s clusters. As a result any running workloads or active sessions will stop and need to be restarted. https://phabricator.wikimedia.org/T326554 -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

Cloud-announce January 2023