Howdy Tomasz & co!
The backfill job for mobile web pageviews by device class + device OS[1] finished this morning, and data is now available for all of March; April is up to date, excepting yesterday[2]. Check it out:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
The job runs daily at midnight, producing a rollup for the last 24 hours ...such as this one from my birthday:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
The same job then recalculates the rollup for the current month, keeping the aggregate fresh. Here's March (which won't change any further):
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
The numbers for March should complete and correct, though there's certainly room for improvement in both classification and data hygiene[3]. Tablets are especially a pain-point for dClass, and we know that's important to y'all. That said, all feedback and suggestions are very are welcome, especially if you see anything fishy. Chat up Diederik or the list and we'll totally get all Mingley with your ideas.
Cheers!
Team Analytics
-- David Schoonover dsc@wikimedia.org
[1] Feature card: https://mingle.corp.wikimedia.org/projects/analytics/cards/61 [2] A race when restarting Hadoop's ResourceManager and slave NodeManagers after a config upgrade caused a silent failure, impacting imports for approximately five hours. (Specifically, imports wadded up into in an ugly, sticky, 158GB mess of duplicated records.) I've cleansed the data and restored the import boundaries, but I believe the Device Props job triggered before I was finished. Once all the data is in for today, I'll rerun both 4/15 and 4/16. [3] Card tracking the hygiene issues: https://mingle.corp.wikimedia.org/projects/analytics/cards/591
On 04/16/2013 07:13 PM, David Schoonover wrote:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
Is there a graph visualizing this data, or even a plain HTML page showing the numbers? I tried to find it at http://stats.wikimedia.org/
Thank you.
Hi Quim,
No there is no visualization of this data yet as that was outside the scope of the initial request. I can create a feature request if this is something that you really would like to see.
Best, Diederik
On Wed, Apr 17, 2013 at 10:35 AM, Quim Gil qgil@wikimedia.org wrote:
On 04/16/2013 07:13 PM, David Schoonover wrote:
http://stats.wikimedia.org/**kraken-public/webrequest/** mobile/device/props/2013/http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013/
Is there a graph visualizing this data, or even a plain HTML page showing the numbers? I tried to find it at http://stats.wikimedia.org/
Thank you.
-- Quim Gil Technical Contributor Coordinator @ Wikimedia Foundation http://www.mediawiki.org/wiki/**User:Qgilhttp://www.mediawiki.org/wiki/User:Qgil
______________________________**_________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/analyticshttps://lists.wikimedia.org/mailman/listinfo/analytics
Thanks. I'll sift through this today/tomorrow. --tomasz
On Tue, Apr 16, 2013 at 7:13 PM, David Schoonover dsc@wikimedia.org wrote:
Howdy Tomasz & co!
The backfill job for mobile web pageviews by device class + device OS[1] finished this morning, and data is now available for all of March; April is up to date, excepting yesterday[2]. Check it out:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
The job runs daily at midnight, producing a rollup for the last 24 hours ...such as this one from my birthday:
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
The same job then recalculates the rollup for the current month, keeping the aggregate fresh. Here's March (which won't change any further):
http://stats.wikimedia.org/kraken-public/webrequest/mobile/device/props/2013...
The numbers for March should complete and correct, though there's certainly room for improvement in both classification and data hygiene[3]. Tablets are especially a pain-point for dClass, and we know that's important to y'all. That said, all feedback and suggestions are very are welcome, especially if you see anything fishy. Chat up Diederik or the list and we'll totally get all Mingley with your ideas.
Cheers!
Team Analytics
-- David Schoonover dsc@wikimedia.org
[1] Feature card: https://mingle.corp.wikimedia.org/projects/analytics/cards/61 [2] A race when restarting Hadoop's ResourceManager and slave NodeManagers after a config upgrade caused a silent failure, impacting imports for approximately five hours. (Specifically, imports wadded up into in an ugly, sticky, 158GB mess of duplicated records.) I've cleansed the data and restored the import boundaries, but I believe the Device Props job triggered before I was finished. Once all the data is in for today, I'll rerun both 4/15 and 4/16. [3] Card tracking the hygiene issues: https://mingle.corp.wikimedia.org/projects/analytics/cards/591
Great work guys! Some comments though - that are aimed mainly at Tomasz, Maryana and the analytics team:
This concern might be premature (I must confess I'm not sure what the current road map looks like and what's coming next as I'm out of the analytics loop right now) but I query the usefulness of this data (for _me_ at least). Since the story doesn't say who this output is for and what problem it is is solving this might meet Maryana or Tomasz's goals but not mine. " so that I can get an idea of current types of devices being used to acces the mobile site" - (yes but why do you want to know that?)
The data format is very good in that I could easily whip up a web page to analyse it although I was left a bit confused as the fake data in the story card looks different to the output.
That said, I looked through a few data sets and couldn't find usage statistics for Kindle. Also what is the difference between BlackBerry and BlackBerry OS and Bada / Bada OS? I'm also assuming this should be a handheld ? (handheld RIM Tablet OS 807)
This is great in terms of allowing me to get vague ideas of our audience - which use Blackberry and which use Android but from my perspective it is not useful unless coupled with the browser they are using. For example which of our users on android are using Opera Mini, which of our users are using Opera Mobile?
This is why I don't trust this sort of data. It makes assumptions about how I consume the data and I worry that by grouping data in this way you lose a lot of the meaning of it.
Is it possible in future/now to access these numbers but instead of grouping showing the exact user agent? (I know this would be a longer list with lots of odd user agents - but for the problems I am trying to solve (getting ideas of importance of bugs) this would be much more useful.
Jon, I requested this data for our forward fiscal year planning and I'm happy to talk to you about why. Just grab me in the office.
--tomasz --tomasz
On Wed, Apr 17, 2013 at 11:01 AM, Jon Robson jrobson@wikimedia.org wrote:
Great work guys! Some comments though - that are aimed mainly at Tomasz, Maryana and the analytics team:
This concern might be premature (I must confess I'm not sure what the current road map looks like and what's coming next as I'm out of the analytics loop right now) but I query the usefulness of this data (for _me_ at least). Since the story doesn't say who this output is for and what problem it is is solving this might meet Maryana or Tomasz's goals but not mine. " so that I can get an idea of current types of devices being used to acces the mobile site" - (yes but why do you want to know that?)
The data format is very good in that I could easily whip up a web page to analyse it although I was left a bit confused as the fake data in the story card looks different to the output.
That said, I looked through a few data sets and couldn't find usage statistics for Kindle. Also what is the difference between BlackBerry and BlackBerry OS and Bada / Bada OS? I'm also assuming this should be a handheld ? (handheld RIM Tablet OS 807)
This is great in terms of allowing me to get vague ideas of our audience - which use Blackberry and which use Android but from my perspective it is not useful unless coupled with the browser they are using. For example which of our users on android are using Opera Mini, which of our users are using Opera Mobile?
This is why I don't trust this sort of data. It makes assumptions about how I consume the data and I worry that by grouping data in this way you lose a lot of the meaning of it.
Is it possible in future/now to access these numbers but instead of grouping showing the exact user agent? (I know this would be a longer list with lots of odd user agents - but for the problems I am trying to solve (getting ideas of importance of bugs) this would be much more useful.
On Wed, Apr 17, 2013 at 11:01 AM, Jon Robson jrobson@wikimedia.org wrote:
Also what is the difference between BlackBerry and BlackBerry OS and Bada / Bada OS?
Excellent question. :)
I'm also assuming this should be a handheld ? (handheld RIM Tablet OS 807)
That'll be the BlackBerry Playbook 7-inch tablet, which runs a precursor to BB10. For OS breakdown purposes, it should possibly be grouped with BB 10 and both should be separate from classic BlackBerry.
-- brion
Yes, there needs to be do some cleaning up of the Operating System labels: see "dClass is messy with Device Class and OS labels " ( https://mingle.corp.wikimedia.org/projects/analytics/cards/591)
Jon: your requests are captured in Mingle cards * "Provide raw useragent string datasets " https://mingle.corp.wikimedia.org/projects/analytics/cards/321 (@jon: would love to get your input on this card) *"Non-web based querying of mobile webrequest data" https://mingle.corp.wikimedia.org/projects/analytics/cards/383
Thanks for the comments! best, D
On Wed, Apr 17, 2013 at 3:19 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Wed, Apr 17, 2013 at 11:01 AM, Jon Robson jrobson@wikimedia.orgwrote:
Also what is the difference between BlackBerry and BlackBerry OS and Bada / Bada OS?
Excellent question. :)
I'm also assuming this should be a handheld ? (handheld RIM Tablet OS 807)
That'll be the BlackBerry Playbook 7-inch tablet, which runs a precursor to BB10. For OS breakdown purposes, it should possibly be grouped with BB 10 and both should be separate from classic BlackBerry.
-- brion
Thanks! I explained to Tomasz the story card David referenced had be somewhat confused as I was not seeing a 1-1 mapping of what it described to what I was seeing so my mind wandered. Hope my feedback was useful regardless.
On Wed, Apr 17, 2013 at 1:58 PM, Diederik van Liere dvanliere@wikimedia.org wrote:
Yes, there needs to be do some cleaning up of the Operating System labels: see "dClass is messy with Device Class and OS labels " (https://mingle.corp.wikimedia.org/projects/analytics/cards/591)
Jon: your requests are captured in Mingle cards
- "Provide raw useragent string datasets "
https://mingle.corp.wikimedia.org/projects/analytics/cards/321 (@jon: would love to get your input on this card) *"Non-web based querying of mobile webrequest data" https://mingle.corp.wikimedia.org/projects/analytics/cards/383
Thanks for the comments! best, D
On Wed, Apr 17, 2013 at 3:19 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Wed, Apr 17, 2013 at 11:01 AM, Jon Robson jrobson@wikimedia.org wrote:
Also what is the difference between BlackBerry and BlackBerry OS and Bada / Bada OS?
Excellent question. :)
I'm also assuming this should be a handheld ? (handheld RIM Tablet OS 807)
That'll be the BlackBerry Playbook 7-inch tablet, which runs a precursor to BB10. For OS breakdown purposes, it should possibly be grouped with BB 10 and both should be separate from classic BlackBerry.
-- brion
Cool, thank you!
And yeah, we're aware of the data hygiene issues ("BlackBerry" vs "BlackBerry OS"); I mentioned this card in my first mail: https://mingle.corp.wikimedia.org/projects/analytics/cards/591
-- David Schoonover dsc@wikimedia.org
On Wed, Apr 17, 2013 at 2:08 PM, Jon Robson jrobson@wikimedia.org wrote:
Thanks! I explained to Tomasz the story card David referenced had be somewhat confused as I was not seeing a 1-1 mapping of what it described to what I was seeing so my mind wandered. Hope my feedback was useful regardless.
On Wed, Apr 17, 2013 at 1:58 PM, Diederik van Liere dvanliere@wikimedia.org wrote:
Yes, there needs to be do some cleaning up of the Operating System
labels:
see "dClass is messy with Device Class and OS labels " (https://mingle.corp.wikimedia.org/projects/analytics/cards/591)
Jon: your requests are captured in Mingle cards
- "Provide raw useragent string datasets "
https://mingle.corp.wikimedia.org/projects/analytics/cards/321 (@jon:
would
love to get your input on this card) *"Non-web based querying of mobile webrequest data" https://mingle.corp.wikimedia.org/projects/analytics/cards/383
Thanks for the comments! best, D
On Wed, Apr 17, 2013 at 3:19 PM, Brion Vibber bvibber@wikimedia.org
wrote:
On Wed, Apr 17, 2013 at 11:01 AM, Jon Robson jrobson@wikimedia.org wrote:
Also what is the difference between BlackBerry and BlackBerry OS and Bada / Bada OS?
Excellent question. :)
I'm also assuming this should be a handheld ? (handheld RIM Tablet OS 807)
That'll be the BlackBerry Playbook 7-inch tablet, which runs a precursor to BB10. For OS breakdown purposes, it should possibly be grouped with
BB 10
and both should be separate from classic BlackBerry.
-- brion