Hi,
there was a outage today 2019-05-29 in CloudVPS/Toolforge involving keystone + NFS. All CloudVPS projects (including Toolforge) had troubles using the NFS-based storage due to an upgrade operation we were doing in cloudcontrol1003.wikimedia.org.
You can read more about the incident here: https://wikitech.wikimedia.org/wiki/Incident_documentation/20190529-NFS-keys...
The incident postmorten is not completed yet, but you can already read the main sections: * what happened * timeline * things to improve in the future
regards.
cloud-admin@lists.wikimedia.org