Hello,
there will be a network maintenance operation happening on Thursday 2025-03-27 12:30 UTC that may impact all Wikimedia Cloud Services, including:
* Cloud VPS * Toolforge * PAWS * and any others
The planned operation window will last for 1h, from 12:30 UTC to 13:30 UTC.
We have prepared the required changes and we are expecting no outages. But the operation is sensitive and we would like you to be informed of potential service disruptions as a result of the operations.
If you want more technical details, we are using a Phabricator ticket: https://phabricator.wikimedia.org/T389958
regards.
On 3/26/25 10:24, Arturo Borrero Gonzalez wrote:
Hello,
there will be a network maintenance operation happening on Thursday 2025-03-27 12:30 UTC that may impact all Wikimedia Cloud Services, including:
- Cloud VPS
- Toolforge
- PAWS
- and any others
The planned operation window will last for 1h, from 12:30 UTC to 13:30 UTC.
We have prepared the required changes and we are expecting no outages. But the operation is sensitive and we would like you to be informed of potential service disruptions as a result of the operations.
If you want more technical details, we are using a Phabricator ticket: https://phabricator.wikimedia.org/T389958
Hello,
I'm sorry to inform we did have a full network outage during the course of this operation.
Further investigation will be needed to understand what is happening with this network maintenance before we can proceed.
regards.
On Thu, Mar 27, 2025 at 2:20 PM Arturo Borrero Gonzalez aborrero@wikimedia.org wrote:
I'm sorry to inform we did have a full network outage during the course of this operation.
Further investigation will be needed to understand what is happening with this network maintenance before we can proceed.
In the aftermath of the outage a few hours ago, we are still seeing some problems with NFS servers and clients.
We are going to reboot all the NFS servers and clients, which will cause another brief outage of Cloud VPS projects and Toolforge tools which are depending on NFS.
Thanks for your patience while we fix this issue.
cloud-admin@lists.wikimedia.org