This last few days analytics-store replication has started to lag by some hours. Currently s1 (enwiki) and s5 (dewiki, wikidatawiki) are most affected. Eventlogging is not lagging, due to the nicely batched writes it does now-a-days :-)

There are many slow queries running from the research user on stat1003, referencing eventlogging tables like MobileWebDiffClickTracking*.

I'm not sure who belongs to them, or if they're new, or if they're safe to kill, so this is mainly a heads-up email. Let us know if ops should kill stuff to let the box catch up again.

BR
Sean

--

DBA @ WMF