Any update on this? I still get a "The MariaDB server is running with the --read-only option so it cannot execute this statement" message when I try to do anything!

  Arthur

On Tue, Nov 10, 2020 at 10:52 AM Brooke Storm <bstorm@wikimedia.org> wrote:
This will be happening in around 10 minutes. ToolsDB will be read-only until we can get a consistent dump to rebuild replication.

Brooke Storm
Staff SRE
Wikimedia Cloud Services
IRC: bstorm

On Nov 6, 2020, at 12:12 PM, Brooke Storm <bstorm@wikimedia.org> wrote:

The ToolsDB service suffered a breakage in replication on 2020-10-27. WMCS has tried to restore replication of data, but that has been unsuccessful so far including doing a dump to rebuild replication without downtime.
At this point, we have a new server waiting to become the replica, but to start the replication process, we need to set the database to read-only for a full dump. This could easily take more than an hour. During that entire time, the database will be read-only.

We will begin at 1600 UTC and finish when it is done. The database is quite large, but, with it in read-only mode, I hope the backup will not take terribly long.

Please see https://phabricator.wikimedia.org/T266587 for additional information. 

Brooke Storm
Staff SRE
Wikimedia Cloud Services
IRC: bstorm


_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
_______________________________________________
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud