Hi Brooke,
Did NFS just stop working on toollabs about 10 minutes ago? Is that related to this?
Maarten
On 26-02-20 22:17, Brooke Storm wrote:
The maintenance is finished. Cron jobs likely would have failed during the maintenance and other issues. Please report new issues as they show up. Brooke Storm SRE Wikimedia Cloud Services bstorm@wikimedia.org IRC: bstorm_ On 2/26/20 10:47 AM, Brooke Storm wrote:
This is just a reminder of this maintenance and that it is starting in 15 min. Interruptions will be likely for an hour or so. Brooke Storm SRE Wikimedia Cloud Services bstorm@wikimedia.org IRC: bstorm_ On 2/20/20 3:14 PM, Brooke Storm wrote:
On Wed 26-Feb, we are making a large change to how NFS is mounted in Cloud Services https://gerrit.wikimedia.org/r/c/operations/puppet/+/571821. This will impact any Cloud VPS projects that mount NFS for home directories, project directories and scratch, including Toolforge. During this change, NFS will become unresponsive for a short time. Some NFS clients will recover on their own with little impact. Where needed, WMCS will reboot or remount NFS clients.
This change will improve future NFS management and hopefully reduce future disruptions from maintenance.
Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud