On Wed, Sep 12, 2018 at 11:16 AM, Alexandros Kosiaris akosiaris@wikimedia.org wrote:
Hello all,
Today we've successfully migrated our wikis (MediaWiki and associated services) from our primary data center (eqiad) to our secondary (codfw), an exercise we've done for the 3rd year in a row. During the most critical part of the switch today, the wikis were in read-only mode for a duration of 7 and a half minutes - a significant improvement from last year.
Everyone involved worked hard to make this happen, but I'd like to give a special shout out to Giuseppe Lavagetto for taking the time to follow up on a VisualEditor problem that affected Wikitech (https://phabricator.wikimedia.org/T163438). We noticed during the April 2017 switchover that the client side code for VE was failing to communicate with the backend component while the wikis were being served from the Dallas datacenter. We guessed that this was a configuration error of some sort, but did not take the time to debug in depth. When the issue reoccurred during the current datacenter switch, Giuseppe took a deep dive into the code and configuration, identified the configuration difference that triggered the problem, and made a patch for the Parsoid backend that fixes Wikitech.
Wikitech is a low volume wiki for both edits and reads, and for various historical and technical reasons is different from all other wikis that we host. Keeping it available for reading is important to our technical teams because it hosts many of the troubleshooting playbooks that we use to diagnose and correct operational problems on the rest of the wikis. Taking the time to work on an editing bug that only impacted edits done using VisualEditor is awesome, but not the sort of thing I would normally expect to be worked on promptly. For me, Giuseppe's work on this bug is a sign that that he cares about the small details, and also that the rest of the switchover went well giving him the time to investigate lower impact edge cases like this.
Bryan