Hi,
The next datacenter switchover, going from codfw (Texas) to eqiad (Virginia), will be the week of Sept. 13. The schedule is as follows:
Services: Monday, Sept 13th 14:00 UTC Traffic: Monday, Sept 13th 15:00 UTC MediaWiki: Tuesday, Sept 14th 14:00 UTC
If you're not familiar with this process, see https://wikitech.wikimedia.org/wiki/Switch_Datacenter. We also recently published a blog post recapping the June switchover: https://techblog.wikimedia.org/2021/07/23/june-2021-data-center-switchover/.
https://phabricator.wikimedia.org/T287539 is the Phabricator tracking task for this switchover.
Thanks, -- Kunal
Hi,
Just a reminder that this is happening next week.
On 8/4/21 1:27 PM, Kunal Mehta wrote:
Services: Monday, Sept 13th 14:00 UTC Traffic: Monday, Sept 13th 15:00 UTC MediaWiki: Tuesday, Sept 14th 14:00 UTC
The services/traffic switchovers should be invisible to users, while we expect around 2 minutes of read-only time during the MediaWiki switchover if all goes well. Coordination will take place on IRC in #wikimedia-operations, like usual.
We did a live test of the switchover process earlier today and it went well, the two issues that we noticed[1][2] have already been fixed.
[1] https://phabricator.wikimedia.org/T290677 [2] https://phabricator.wikimedia.org/T290676
-- Kunal
Hi,
This week's datacenter switchovers went very smoothly overall, in total we had 2 minutes and 42 seconds of read-only time when switching MediaWiki from the codfw to eqiad datacenter, which is pretty close to the 2 minutes we were aiming for.
The main follow-up task from today is that our cache warmup process needs to also warmup some Wikidata caches: https://phabricator.wikimedia.org/T290989.
Yesterday we also switched over various services, the main follow-ups are that we want our cookbook automation to support pooling services in both datacenters (active/active): https://phabricator.wikimedia.org/T290919, and that we have so many services that the list no longer fits on one IRC !log line: https://phabricator.wikimedia.org/T285709.
-- Kunal
wikitech-l@lists.wikimedia.org