Thanks. I'll sift through this today/tomorrow.
--tomasz
On Tue, Apr 16, 2013 at 7:13 PM, David Schoonover <dsc(a)wikimedia.org> wrote:
Howdy Tomasz & co!
The backfill job for mobile web pageviews by device class + device OS[1]
finished this morning, and data is now available for all of March; April is
up to date, excepting yesterday[2]. Check it out:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/201…
The job runs daily at midnight, producing a rollup for the last 24 hours
...such as this one from my birthday:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/201…
The same job then recalculates the rollup for the current month, keeping the
aggregate fresh. Here's March (which won't change any further):
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/201…
The numbers for March should complete and correct, though there's certainly
room for improvement in both classification and data hygiene[3]. Tablets are
especially a pain-point for dClass, and we know that's important to y'all.
That said, all feedback and suggestions are very are welcome, especially if
you see anything fishy. Chat up Diederik or the list and we'll totally get
all Mingley with your ideas.
Cheers!
Team Analytics
--
David Schoonover
dsc(a)wikimedia.org
[1] Feature card:
https://mingle.corp.wikimedia.org/projects/analytics/cards/61
[2] A race when restarting Hadoop's ResourceManager and slave NodeManagers
after a config upgrade caused a silent failure, impacting imports for
approximately five hours. (Specifically, imports wadded up into in an ugly,
sticky, 158GB mess of duplicated records.) I've cleansed the data and
restored the import boundaries, but I believe the Device Props job triggered
before I was finished. Once all the data is in for today, I'll rerun both
4/15 and 4/16.
[3] Card tracking the hygiene issues:
https://mingle.corp.wikimedia.org/projects/analytics/cards/591