Hi Sean,
Here's Marcel from Analytics.
I'd like to comment with you some strange behaviors that we've observed on
EventLogging database (m4-master.equiad.wmnet).
1) There are some time spans where there is no data in any table. Examples
follow:
- 2015-04-09 17:20 -> 18:35
- 2015-04-11 03:35 -> 05:20
- 2015-04-11 14:00 -> 15:20
- 2015-04-11 19:00 -> 20:00
- 2015-04-12 14:30 -> 15:40
- 2015-04-13 11:35 -> 12:30
- 2015-04-13 16:30 -> 17:35
This is happening a lot, as you can see, so we are really concerned about
it.
These outages are not explained by any of the EventLogging logs (processor
logs, consumer logs) which confirm that the events were actually sent
normally to the database (and written) for those time spans.
2) Maybe it has something to do with it, or not. But we observed that
sometimes the last events inserted in the tables date from 30mins -
1h30mins in the past. At other moments, the last inserts date from seconds
ago. This is maybe expected due to slave replication but we're not sure.
Do you have some opinion on what can be happening? Any thought would help a
lot.
Thanks!
Marcel