Howdy!

Just wanted to let y’all know about some updates to Discovery Dashboards[1]:

- All dashboards have been updated to use a new collection of datasets[2]
- The Wikipedia.org Portal dashboard[4] now tracks:
    - Clicks on sister project links (T152617)
    - Clicks on mobile app links (T154634)
- The WDQS dashboard[3] now tracks LDF endpoint usage (T153936)

So that’s the front-end news. On the back-end:

- We’ve refactored the entire metric-calculation codebase[5] to use Analytics’ Reportupdater infrastructure[6] (T150915)
- That codebase is now thoroughly documented[7]
- We’ve introduced additional rules for detecting automata to several scripts that calculate traffic and usage metrics
- We’ve erased and backfilled all the metrics from 2017-01-01 using all the new definitions and updated UDFs
    - So if you see drastic changes to a metric starting on 1st January 2017, that’s why :)
    - Accordingly, all the graphs have been annotated with this info
- We are no longer undercounting full-text and geo search API usage (see [8], hence the jump on 2017-01-01)
- Adding new metrics and backfilling data is now way easier
    - We’re already working on adding the ability to look at several search metrics by language and project (T150410)
    - We can even add predictive models of metrics AS metrics for the codebase to calculate (T112170, [9])

A huge thanks to Deb for her support and patience through this project, and huge thanks to Chelsy for her thorough code review of and help with [10].

Cheers,
Mikhail on behalf of Discovery’s Analysis team

[1]: https://discovery.wmflabs.org/
[2]: https://datasets.wikimedia.org/aggregate-datasets/discovery/
[3]: https://discovery.wmflabs.org/wdqs/
[4]: https://discovery.wmflabs.org/portal/
[5]: https://phabricator.wikimedia.org/diffusion/WDGO/
[6]: https://wikitech.wikimedia.org/wiki/Analytics/Reportupdater
[7]: https://github.com/wikimedia/wikimedia-discovery-golden/blob/master/README.md
[8]: https://gerrit.wikimedia.org/r/#/c/315503/
[9]: https://github.com/wikimedia/wikimedia-discovery-golden/blob/master/README.md#adding-new-forecasting-modules
[10]: https://gerrit.wikimedia.org/r/#/c/325870/

--
Mikhail Popov // Count Logula
Discovery, Wikimedia Foundation

PGP Public Key: https://people.wikimedia.org/~bearloga/public.asc
Fingerprint: B362 76D4 FEBE F715 5F40  0C9C 4BC8 A265 E573 3216


_______________________________________________
discovery mailing list
discovery@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/discovery