Hello!
The Wikimedia datacenter team will be performing some routine network
maintenance[1] next Wednesday. This will cause brief, rolling network
interruptions for essentially all tools, services, and virtual servers
-- each physical server will be briefly unplugged as its network cable
is moved to a new switch. Ideally these interruptions will only last a
few seconds and be unnoticeable, but downtime may be longer if there are
unexpected complications.
This process will begin at 14:00 UTC on Wednesday the 24th, which is 7
AM in California. All moves should be completed within an hour.
-Andrew
[1] https://phabricator.wikimedia.org/T183585
Region migration is going smoothly, and it's time to plan out the next
week of moves. For details about what's happening here, consult the
link below[1].
Here is the schedule for the next week of moves:
Monday, 2018-11-05: discovery-stats, globaleducation, hat-imagescalers,
language
Tuesday, 2018-11-06: matrix, mcr-dev, monitoring, mwfileimport, openocr
Wednesday, 2018-11-07: ores, puppet
Thursday, 2018-11-08: puppet-diffs, recommendation-api, sciencesource,
saerch, sentry
Friday, 2018-11-09: visualeditor, wikicitevis, wikidiff2-wmde-dev,
wikidiff2-wmde-dev, wikidocumentaries, wikilabels, wpx
Please let me know if you are involved in one those projects and need to
postpone the move, or schedule a to-the-minute migration window.
- Andrew + the WMCS team
[1] https://phabricator.wikimedia.org/phame/post/view/120/neutron_is_here/
Beginning next week, the cloud team will start migrating projects to
Neutron[1] in earnest. I will attempt to reach out individually to
affected project admins as well, but here is the upcoming migration
schedule:
Friday, 2018-10-19: analytics
Monday, 2018-10-22: antiharassment, catgraph, codereview, cvn
Tuesday, 2018-10-23: cyberbot, fastcci, general-k8s
Wednesday, 2018-10-24: huggle, logging, mobile, mwstake
Thursday, 2018-10-25: planet, pluggableauth, privpol-captcha, qna
Friday, 2018-10-26: reading-web-staging, suggestbot, test-twemproxy,
wikibase-registry, wikibrain, wikidata-primary-sources-tool
That seemingly-arbitrary selection of projects is chosen to free up some
hardware space and make room for additional migrations in the future.
Even if you don't see your project name in that list, please keep an eye
on your inbox :)
- Andrew + the WMCS team
[1] https://phabricator.wikimedia.org/phame/post/view/120/neutron_is_here/
Routine network upgrades are scheduled for Thursday which may result in
brief WMCS service interruptions. In particular, Wikitech and Horizon
may stop working, and instance creation/deletion/updating may briefly fail.
The network engineers have reserved a two-hour window beginning at 16:00
UTC (9 AM Pacific time). Their worst case scenario is 30 minutes of
downtime but if all goes well the interruption should not be noticeable,
just the time to unplug a cable from one switch and plug it into another.