[Labs-l] (Tool labs impact) Reboot of virt11 Friday Sept 6 at 20:00 UTC

[Marc A. Pelletier]

On 09/04/2013 05:38 PM, Ryan Lane wrote:
> This will cause a reboot of about 45 instances.

Here is the expected impact on Tool Labs infrastructure:

- Bastion instances (tools-login and tools-dev) are going to be
unavailable during the outage, stopping interactive sessions.  Scheduled
(cron) jobs will be delayed until after the outage*.

- Mail to and from the tools.wmflabs.org domain will be delayed until
the outage is complete.

- The local (tools-db) database will be down during the outage, possibly
impacting tools that use it.

The latter is the only impact that will affect tools running on the grid
proper.  You may have to restart continuous jobs that rely on the local
database if they are not able to recover from lost database connections
on their own.

-- Marc

* Cron automatically fires "missed" jobs when it restarts, but note that
if a scheduled job would have fired more than one time during the
outage, it will only be fired /once/ when cron restarts.