While troubleshooting an infrastructure issue I just now accidentally triggered a reboot of a few VMs, including the primary host of toolsdb. If you see a little storm of alert in your tool logs about timeouts and disconnections, that was what that was.
Everything should be back to normal now!
-Andrew
On Wed, Dec 7, 2022 at 11:19 PM Andrew Bogott abogott@wikimedia.org wrote:
Everything should be back to normal now!
Update: everything should be back to normal _now_ :)
Long story: after the reboot, toolsdb was available but set to read-only mode. TIL: Write mode needs to be enabled manually after a reboot.
Apologies for the inconvenience.
Francesco
cloud-announce@lists.wikimedia.org