Howdy Tomasz & co!

The backfill job for mobile web pageviews by device class + device OS[1]  finished this morning, and data is now available for all of March; April is up to date, excepting yesterday[2]. Check it out:

http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013/

The job runs daily at midnight, producing a rollup for the last 24 hours ...such as this one from my birthday:

http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013/03/mobile_device_props-day-2013-03-24.tsv

The same job then recalculates the rollup for the current month, keeping the aggregate fresh. Here's March (which won't change any further):

http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013/03/mobile_device_props-month-2013-03.tsv

The numbers for March should complete and correct, though there's certainly room for improvement in both classification and data hygiene[3]. Tablets are especially a pain-point for dClass, and we know that's important to y'all. That said, all feedback and suggestions are very are welcome, especially if you see anything fishy. Chat up Diederik or the list and we'll totally get all Mingley with your ideas.

Cheers!

Team Analytics

--
David Schoonover
dsc@wikimedia.org


[1] Feature card: https://mingle.corp.wikimedia.org/projects/analytics/cards/61
[2] A race when restarting Hadoop's ResourceManager and slave NodeManagers after a config upgrade caused a silent failure, impacting imports for approximately five hours. (Specifically, imports wadded up into in an ugly, sticky, 158GB mess of duplicated records.) I've cleansed the data and restored the import boundaries, but I believe the Device Props job triggered before I was finished. Once all the data is in for today, I'll rerun both 4/15 and 4/16.
[3] Card tracking the hygiene issues: https://mingle.corp.wikimedia.org/projects/analytics/cards/591