The ToolsDB service suffered a breakage in replication on 2020-10-27. WMCS has tried to
restore replication of data, but that has been unsuccessful so far including doing a dump
to rebuild replication without downtime.
At this point, we have a new server waiting to become the replica, but to start the
replication process, we need to set the database to read-only for a full dump. This could
easily take more than an hour. During that entire time, the database will be read-only.
We will begin at 1600 UTC and finish when it is done. The database is quite large, but,
with it in read-only mode, I hope the backup will not take terribly long.
Please see
https://phabricator.wikimedia.org/T266587
<https://phabricator.wikimedia.org/T266587> for additional information.
Brooke Storm
Staff SRE
Wikimedia Cloud Services
bstorm(a)wikimedia.org <mailto:bstorm@wikimedia.org>
IRC: bstorm
_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce(a)lists.wikimedia.org (formerly labs-announce(a)lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce