The maintenance is finished. Cron jobs likely would have failed during
the maintenance and other issues. Please report new issues as they show
up.
Brooke Storm
SRE
Wikimedia Cloud Services
bstorm(a)wikimedia.org
IRC: bstorm_
On 2/26/20 10:47 AM, Brooke Storm wrote:
This is just a reminder of this maintenance and that
it is starting in
15 min. Interruptions will be likely for an hour or so.
Brooke Storm
SRE
Wikimedia Cloud Services
bstorm(a)wikimedia.org
IRC: bstorm_
On 2/20/20 3:14 PM, Brooke Storm wrote:
On Wed 26-Feb, we are making a large change to
how NFS is mounted in
Cloud Services
https://gerrit.wikimedia.org/r/c/operations/puppet/+/571821. This
will impact any Cloud VPS projects that mount NFS for home
directories, project directories and scratch, including Toolforge.
During this change, NFS will become unresponsive for a short time.
Some NFS clients will recover on their own with little impact. Where
needed, WMCS will reboot or remount NFS clients.
This change will improve future NFS management and hopefully reduce
future disruptions from maintenance.