Re: [Analytics] [EE] Notifications dashboard

A short post-mortem of the problem with missing notification data. tl;dr: the drop is not a real one and is being fixed. As some of you noticed, we had a sudden drop in Echo notification counts across all wikis for some categories of events as of November 7 [1]. The drop was only in the EventLogging data, not in the actual delivery of notifications on site or by mail. Here’s what happened: * a change was made to Echo’s instrumentation to log the revid associated with a notification [2] * when the change got merged, the schema ID was not updated in the config file * as a result, events started getting validated against a stale schema, which caused most types of notifications (anything except for page-linked and welcome) to fail validation These are the next steps we’re considering * Benny wrote a patch to bump the schema id [3] it hasn’t been pushed to production yet, but when it does we will have valid data in the MySQL log DB again (in a table called Echo_6081131) * We have a copy of the raw events stored as JSON on stat1, we may want to restore this data and import it into the DB * After the new log is in production, we will update the scripts generating the data dumps to query a union of tables (across multiple schema IDs). This is one of the potential limitations of using SQL as a store from which to extract EventLogging data to generate dashboards. * We will need to figure out whether passing additional fields that are not required and not specified in a schema should invalidate an event or we should relax the validation criteria. Dario [1] http://ee-dashboard.wmflabs.org/graphs/frwiki_echo_category [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=46045 [3] https://gerrit.wikimedia.org/r/#/c/96901/ On Nov 21, 2013, at 2:51 PM, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote: _______________________________________________ EE mailing list EE(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/ee