Hello!
A few weeks ago, during some Wikidata overloads, we observed that the host with Buster and 10.4 performed better, CPU wise, than the ones running 10.1 and Stretch with InnoDB compression, having the same hardware. We haven't found 100% why, but we believe that 10.4 might have some optimizations.
Considering the fact that Quarry points to labsdb1011, and in order to discard a specific issue with that host we thought about replacing labsdb1011 with another host (
https://phabricator.wikimedia.org/T247978#5980018) but maybe we can just try to upgrade labsdb1011 to buster and 10.4
So far we have had normal 1 instance hosts upgraded, multi-instance (2 mysqld processes) upgraded, and we need to have a multisource (labsdb) host upgrade, to make sure 10.4 works fine or to know what might need work (mysqld-exporter
https://phabricator.wikimedia.org/T247290 or whatever), better to know in advance.
10.4 also fixes some bugs that are hitting labsdb hosts specifically:
- GTID works on multisource:
https://jira.mariadb.org/browse/MDEV-12012 this is one of the early bugs we filed with MariaDB almost 3 years ago and looks like it is now working even though - this requires some work on the master's side, but my last tests are looking good and if we can enable GTID on labsdb hosts that'd we be a BIG improvement towards avoiding corruption during a crash.
So, any objections to reimage labsdb1011 as Buster and 10.4 (/srv won't be formatted, so we don't have to rebuild that host).
Cheers
Manuel.