Routine network upgrades are scheduled for Thursday which may result in
brief WMCS service interruptions. In particular, Wikitech and Horizon
may stop working, and instance creation/deletion/updating may briefly fail.
The network engineers have reserved a two-hour window beginning at 16:00
UTC (9 AM Pacific time). Their worst case scenario is 30 minutes of
downtime but if all goes well the interruption should not be noticeable,
just the time to unplug a cable from one switch and plug it into another.
We would like to share some information regarding Wikimedia Cloud
Services plans for deprecating Ubuntu, specially Trusty.
Ubuntu Trusty's end-of-life is April 2019 and the WMF decided to
consolidate in a single operating system, which is Debian.
In Cloud VPS, projects containing Ubuntu virtual machine instances have
been contacted by means of a Phabricator task. Toolforge users aren't
affected by this right now, because Toolforge itself is currently
running Trusty. But we are already working on the next, Debian-based,
All this information, more details (and timelines), can be found on
this Wikitech page:
Please, let us know any question or doubt you may have.
At 6pm UTC, Thursday 2018-09-27, we’ll be taking the primary tools/project NFS system offline for a brief moment to attempt some upgrades, which should cause load to climb a bit and then settle. This should be a relatively low-impact change and has been thoroughly tested in a separate environment, but if anything goes wrong with the process, corrective action may cause a few minutes of problems.
Wikimedia Cloud Services
At 6pm UTC, we will be briefly resetting the main NFS server for tools and projects to upgrade the tooling around NFS there. This should cause a brief climb in processes waiting for IO and then that will settle out.
If things don’t operate as expected, rolling back the change could take a few minutes, but that seems unlikely. Please let us know if there are prolonged issues in the #wikipedia-cloud channel.
Wikimedia Cloud Services
There are currently 57 unclaimed projects on
https://wikitech.wikimedia.org/wiki/Cloud_VPS_2018_Purge. I will start
shutting down unclaimed projects at the beginning of next month, and
those projects will be left behind in the future network migration
and, eventually, deleted.
If you see anything in this list that you care about, please indicate
that the project is in use on the Wikitech page. Here are the 57
projects that are currently unmarked:
As you may have read about in other venues, the Wikimedia Foundation will
be conducting a switchover of datacenters on September 11 and 12, with the
switch back to occur on October 10 and 11. During the switchover and
switch-back events, the wikis will be in read-only mode.
Cloud VPS and Toolforge are not affected by these switchovers. However, if
you run bots or scripts that make edits to the wikis, they may be unable to
make edits during this time. This should appear to your bots as the
standard read-only mode which occurs unplanned on the production wikis from
time to time.
Please let me know if you have any questions.
Associate Product Manager