[Labs-l] Where can I see the Tool Labs DB replag?

[Tim Landscheidt]

Asher Feldman <afeldman at wikimedia.org> wrote:

> Replication was halted on a lock timeout, likely caused by a query writing
> to a userdb while reading from a dewiki table.  I had intended to set
> innodb_locks_unsafe_for_binlog on all labsdb servers, which disables gap
> and next key locking, while also only guaranteeing read_committed
> isolation.  This is completely safe in labs since nothing but the slave sql
> thread writes to the replicated tables, and it should prevent this issue.
> I ended up setting it for everything except s4 and s5; probably related to
> the mad scrambling for the Amsterdam hackathon :)  S4 and s5 now need a
> mariadb restart for the setting to go live, which I'll work with Marc to
> schedule.

> I've also set slave_transaction_retries to a very large value on all of the
> labsdb's, a change which does not require a restart.  That should also
> prevent a recurrence in the interim before the the server restarts occur.
> And at least the s5 replica caught up very quickly.

> [...]

Don't know if and how this is related, but replication for
s1 seems to have stopped around 19:26Z.

Tim