We also have one host down for maintenance https://phabricator.wikimedia.org/T222978 which puts more load on the other two.

El lun., 10 jun. 2019 16:35, Yetkin Sakal <superyetkin@yahoo.com> escribió:
All shards are affected. Please view the heartbeat report at https://tools.wmflabs.org/replag/

On Monday, June 10, 2019, 6:27:44 PM GMT+3, Manuel Arostegui <marostegui@wikimedia.org> wrote:


We currently have some issues going on.
One of the hosts had issues with s4 (commonswiki) https://phabricator.wikimedia.org/T225390

And then S3 replica's master (which holds like 900 wikis) had hardware issues we are still trying to figure out https://phabricator.wikimedia.org/T225391 so that means that those S3 wikis are currently not getting the new data.

We are trying to sort things out.
Sorry for the inconveniences

El lun., 10 jun. 2019 16:20, Yetkin Sakal via Cloud <cloud@lists.wikimedia.org> escribió:

What can be done to decrease the lagging on web.db.svc.eqiad.wmflabs? Tools working on this are severely affected ands up with outdated results.

Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org)