[Labs-announce] rolling outages and reboots on Wednesday, including tools-bastion-02

Andrew Bogott abogott at wikimedia.org
Wed Mar 22 13:29:30 UTC 2017


Reminder:  All of this will start happening in about 30 minutes.

On 3/20/17 6:19 PM, Andrew Bogott wrote:
> One of our virtualization hosts (labvirt1001) is exhibiting troubling 
> behavior.  Nothing hosted on that box is currently affected, but to be 
> on the safe side we're going to evacuate its VMs and run it through 
> some tests.
>
> The evacuation process is a bit slow and haphazard -- while being 
> transferred, any one VM may experience 30-60 minutes of downtime, and 
> will be fully rebooted at the end.  The VMs that will be affected by 
> this process are:
>
> tools-bastion-02 | tools
> citoidtest       | services
> clm-web-01       | community-labs-monitoring
> db               | cognate
> elastic1         | analytics
> etherpad01       | etherpad
> maps-tiles2      | maps
> osmit-tre        | osmit
> perf-testing     | mobile
> puppet-mailman   | puppet
> test-spm-1       | project-proxy
> wikidata-mobile  | wikidata-dev
> wmt-exec         | wmt
>
>
> Note that tools-bastion-02 is on that list!  I will be moving that 
> system first, at 14:00 UTC on Wednesday the 22nd (that's 7AM San 
> Francisco time.)
>
> The other instances will be evacuated over the course of that day, in 
> no particular order and at no particular time.  If you require a more 
> specific migration window, please respond and let me know. 
> Alternatively, if you totally don't care about downtime and are happy 
> to have your instance migrated tomorrow instead of Wednesday, that 
> would be good to know as well.
>
> Thank you, and sorry for the inconvenience.
>
> -Andrew + the Labs Team
>




More information about the Labs-announce mailing list