[I'm checking this with Nuria now.]

Amir Elisha Aharoni‏ ። אָמִיר אֱלִישָׁע אַהֲרוֹנִי
Language Engineering‏ ። הַנְדָּסָה לְשׁוֹנִית
Wikimedia Foundation‏ ። קֶרֶן וִיקִימֶדְיָה

2014-06-25 18:12 GMT+03:00 Nuria Ruiz <nuria@wikimedia.org>:

We had an spike on EL yesterday nite that was caught by our alarms. The spike can be seen here:


We look at the data for a little while and we can see the schema 'UniversalLanguageSelector-tofu' logging at a higher rate than it normally does. 

Can you guys look into what there might have been going on? Looks like the "higher than normal logging" might have been triggered by a Localization update that happened yesterday.

The event "02:50 logmsgbot: LocalisationUpdate ResourceLoader cache refresh completed at Wed Jun 25 02:48:53 UTC 2014 (duration 48m 52s)" from https://wikitech.wikimedia.org/wiki/Server_Admin_Log
happened about at the same time we saw the spike. 



---------- Forwarded message ----------
From: <icinga@neon.wikimedia.org>
Date: Wed, Jun 25, 2014 at 4:46 AM
Subject: ** PROBLEM alert - tungsten/Throughput of event logging events is CRITICAL **
To: nuria@wikimedia.org

❤❤❤❤❤ Icinga ❤❤❤❤❤

Notification Type: PROBLEM

Service: Throughput of event logging events
Host: tungsten

Date/Time: Wed Jun 25 02:46:22 UTC 2014

Additional Info:

CRITICAL: 7.14% of data exceeded the critical threshold [500.0]
Love, Icinga