I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
At some point I believe we hope to just, you know. Have a regularly
updated browser matrix somewhere. I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
Do we not have more recent data than May?
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
There are these legacy reports, but as you can see from the bug list above the table,
they have seen little maintenance for a while, so treat with caution:
I'm hoping Hadoop will blow this out of the water soonish.
http://stats.wikimedia.org/wikimedia/squids/SquidReportClients.htm
http://stats.wikimedia.org/wikimedia/squids/SquidReportCountryBrowser.htm
From: analytics-bounces@lists.wikimedia.org [mailto:analytics-bounces@lists.wikimedia.org] On Behalf Of Nuria Ruiz Sent: Friday, October 10, 2014 22:02 To: A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics. Cc: mobile-tech Subject: Re: [Analytics] Traffic device breakdown
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere.
I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
Do we not have more recent data than May?
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote:
At some point I believe we hope to just, you know. Have a regularly
updated browser matrix somewhere. I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
I agree it would be nice. It's one of those things that will either come as a side-effect of other stuff, OR require subsantially more work, and nothing in-between. Things we need for it:
1. A UDF for ua-parser or whatever we decide to use (this will possibly be necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for text/html requests overall (depends on query runtime), maybe don't.
The *ideal* implementation, obviously, is to pair this up with a site that automatically parses the results into HTML. That should be the end goal. but in terms of engineering support we can get most of the way there simply by ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if there are urgent needs.
Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <- as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = FALSE, quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop and that setup, we can totally generate interim data from the sampled logs using the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the sampling, and people can convince [[Dario|Our Great Leader]] to authorise me to spend 15 minutes of my time on it.
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
- A UDF for ua-parser or whatever we decide to use (this will possibly be
necessary for pageviews, but not necessarily - it depends on our
spider/automaton detection strategy)
We got this one ready today: https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote:
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote:
At some point I believe we hope to just, you know. Have a regularly
updated browser matrix somewhere. I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
I agree it would be nice. It's one of those things that will either come as a side-effect of other stuff, OR require subsantially more work, and nothing in-between. Things we need for it:
- A UDF for ua-parser or whatever we decide to use (this will possibly be
necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for text/html requests overall (depends on query runtime), maybe don't.
The *ideal* implementation, obviously, is to pair this up with a site that automatically parses the results into HTML. That should be the end goal. but in terms of engineering support we can get most of the way there simply by ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if there are urgent needs.
Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <- as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = FALSE, quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop and that setup, we can totally generate interim data from the sampled logs using the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the sampling, and people can convince [[Dario|Our Great Leader]] to authorise me to spend 15 minutes of my time on it.
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote:
- A UDF for ua-parser or whatever we decide to use (this will possibly
be necessary for pageviews, but not necessarily - it depends on our
spider/automaton detection strategy)
We got this one ready today: https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote:
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote:
At some point I believe we hope to just, you know. Have a regularly
updated browser matrix somewhere. I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
I agree it would be nice. It's one of those things that will either come as a side-effect of other stuff, OR require subsantially more work, and nothing in-between. Things we need for it:
- A UDF for ua-parser or whatever we decide to use (this will possibly
be necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for text/html requests overall (depends on query runtime), maybe don't.
The *ideal* implementation, obviously, is to pair this up with a site that automatically parses the results into HTML. That should be the end goal. but in terms of engineering support we can get most of the way there simply by ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if there are urgent needs.
Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <- as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = FALSE, quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop and that setup, we can totally generate interim data from the sampled logs using the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the sampling, and people can convince [[Dario|Our Great Leader]] to authorise me to spend 15 minutes of my time on it.
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
I was going through our backlog again today, and I noticed a bug about supporting editing on Windows Phones with IE9 [1]
Yet again, I wondered 'how many of our users are using IE9' as I wondered if because of this lack of support we are losing out on lots of potential editors.
What's the easiest way to get this information now? Is it available?
[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
Woah! Nice :D How are definitions updates handled?
Since we talked about this on IRC, restating here to keep the archives happy. We pull the ua parser jar from our archiva depot, an update will involve building a new jar, uploading it to archiva and updating our dependency file (pom.xml) to point to the newly updated version.
On Fri, Oct 10, 2014 at 9:59 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote:
- A UDF for ua-parser or whatever we decide to use (this will possibly
be necessary for pageviews, but not necessarily - it depends on our
spider/automaton detection strategy)
We got this one ready today: https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote:
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote:
At some point I believe we hope to just, you know. Have a regularly
updated browser matrix somewhere. I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
I agree it would be nice. It's one of those things that will either come as a side-effect of other stuff, OR require subsantially more work, and nothing in-between. Things we need for it:
- A UDF for ua-parser or whatever we decide to use (this will possibly
be necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for text/html requests overall (depends on query runtime), maybe don't.
The *ideal* implementation, obviously, is to pair this up with a site that automatically parses the results into HTML. That should be the end goal. but in terms of engineering support we can get most of the way there simply by ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if there are urgent needs.
Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <- as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = FALSE, quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop and that setup, we can totally generate interim data from the sampled logs using the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the sampling, and people can convince [[Dario|Our Great Leader]] to authorise me to spend 15 minutes of my time on it.
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote:
Hi Jon -- I'm sure other folks will have more information but here's a link to a slide with some data from May[1]. We don't see a lot of Windows phone traffic.
-Toby
[1] https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org wrote:
> I was going through our backlog again today, and I noticed a bug > about > supporting editing on Windows Phones with IE9 [1] > > Yet again, I wondered 'how many of our users are using IE9' as I > wondered if because of this lack of support we are losing out on lots > of potential editors. > > What's the easiest way to get this information now? Is it available? > > [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599 > > _______________________________________________ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics >
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
Anything would be useful. I just hit this situation again. I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
In the mean time while I don't have a fancy place to find out the answers to this how can I get these answers? Should I mail the analytics mailing list to ask these questions? Cc a point person on bugzilla with the question? Ping someone privately?
Jon
On Tue, Oct 14, 2014 at 10:30 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
Since we talked about this on IRC, restating here to keep the archives happy. We pull the ua parser jar from our archiva depot, an update will involve building a new jar, uploading it to archiva and updating our dependency file (pom.xml) to point to the newly updated version.
On Fri, Oct 10, 2014 at 9:59 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote:
- A UDF for ua-parser or whatever we decide to use (this will possibly
be necessary for pageviews, but not necessarily - it depends on our
spider/automaton detection strategy)
We got this one ready today: https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote:
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote:
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere.
I REALLY think this should make it into our goals, if it cannot be done this quarter it should for sure be done this quarter.
I agree it would be nice. It's one of those things that will either come as a side-effect of other stuff, OR require subsantially more work, and nothing in-between. Things we need for it:
- A UDF for ua-parser or whatever we decide to use (this will possibly
be necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for text/html requests overall (depends on query runtime), maybe don't.
The ideal implementation, obviously, is to pair this up with a site that automatically parses the results into HTML. That should be the end goal. but in terms of engineering support we can get most of the way there simply by ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if there are urgent needs.
Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <- as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = FALSE, quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop and that setup, we can totally generate interim data from the sampled logs using the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the sampling, and people can convince [[Dario|Our Great Leader]] to authorise me to spend 15 minutes of my time on it.
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Email Dario and I, if he prioritises it I'll run a check on more recent data.
At some point I believe we hope to just, you know. Have a regularly updated browser matrix somewhere. This comes some time after pageviews though.
On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org wrote: > > Hi Jon -- I'm sure other folks will have more information but here's > a link to a slide with some data from May[1]. We don't see a lot of Windows > phone traffic. > > -Toby > > [1] > https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca... > > On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson jrobson@wikimedia.org > wrote: >> >> I was going through our backlog again today, and I noticed a bug >> about >> supporting editing on Windows Phones with IE9 [1] >> >> Yet again, I wondered 'how many of our users are using IE9' as I >> wondered if because of this lack of support we are losing out on >> lots >> of potential editors. >> >> What's the easiest way to get this information now? Is it available? >> >> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599 >> >> _______________________________________________ >> Analytics mailing list >> Analytics@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics > >
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
On 15 October 2014 10:35, Jon Robson jrobson@wikimedia.org wrote:
I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
Huh, that's odd; we have a shim for JSON which provides this functionality for older browsers, as announced last month:
https://www.mail-archive.com/wikitech-l%40lists.wikimedia.org/msg78108.html
Is something failing to load it correctly?
J.
I don't know, I didn't look into detail. My main problem that I wanted to highlight here in this e-mail was not that. I wanted to highlight another example of the type of data us engineers need so we can make decisions about whether we should care if there is a shim or not.
On Wed, Oct 15, 2014 at 12:02 PM, James Forrester jforrester@wikimedia.org wrote:
On 15 October 2014 10:35, Jon Robson jrobson@wikimedia.org wrote:
I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
Huh, that's odd; we have a shim for JSON which provides this functionality for older browsers, as announced last month:
https://www.mail-archive.com/wikitech-l%40lists.wikimedia.org/msg78108.html
Is something failing to load it correctly?
J.
James D. Forrester Product Manager, Editing Wikimedia Foundation, Inc.
jforrester@wikimedia.org | @jdforrester
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
Understood, it is hard for you guys to work without knowing this data. I will try to get a user agent list for data from last month but, as I mentioned earlier, I think providing this data in a regular basis (monthly?) is a good goal for us.
On Wed, Oct 15, 2014 at 10:35 AM, Jon Robson jrobson@wikimedia.org wrote:
Anything would be useful. I just hit this situation again. I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
In the mean time while I don't have a fancy place to find out the answers to this how can I get these answers? Should I mail the analytics mailing list to ask these questions? Cc a point person on bugzilla with the question? Ping someone privately?
Jon
On Tue, Oct 14, 2014 at 10:30 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
Since we talked about this on IRC, restating here to keep the archives happy. We pull the ua parser jar from our archiva depot, an update will involve building a new jar, uploading it to archiva and updating our dependency
file
(pom.xml) to point to the newly updated version.
On Fri, Oct 10, 2014 at 9:59 PM, Oliver Keyes okeyes@wikimedia.org
wrote:
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote:
- A UDF for ua-parser or whatever we decide to use (this will
possibly
be necessary for pageviews, but not necessarily - it depends on our
spider/automaton detection strategy)
We got this one ready today:
https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote:
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote:
>At some point I believe we hope to just, you know. Have a regularly > updated browser matrix somewhere. I REALLY think this should make it into our goals, if it cannot be
done
this quarter it should for sure be done this quarter.
I agree it would be nice. It's one of those things that will either
come
as a side-effect of other stuff, OR require subsantially more work,
and
nothing in-between. Things we need for it:
- A UDF for ua-parser or whatever we decide to use (this will
possibly
be necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for text/html requests overall (depends on query runtime), maybe don't.
The ideal implementation, obviously, is to pair this up with a site
that
automatically parses the results into HTML. That should be the end
goal. but
in terms of engineering support we can get most of the way there
simply by
ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if there
are
urgent needs.
Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <-
as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names =
FALSE,
quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop
and
that setup, we can totally generate interim data from the sampled
logs using
the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the sampling,
and
people can convince [[Dario|Our Great Leader]] to authorise me to
spend 15
minutes of my time on it.
On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes <okeyes@wikimedia.org
wrote: > > Email Dario and I, if he prioritises it I'll run a check on more > recent data. > > At some point I believe we hope to just, you know. Have a regularly > updated browser matrix somewhere. This comes some time after
pageviews
> though. > > On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org
wrote:
>> >> Hi Jon -- I'm sure other folks will have more information but
here's
>> a link to a slide with some data from May[1]. We don't see a lot
of Windows
>> phone traffic. >> >> -Toby >> >> [1] >>
https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
>> >> On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson <
jrobson@wikimedia.org>
>> wrote: >>> >>> I was going through our backlog again today, and I noticed a bug >>> about >>> supporting editing on Windows Phones with IE9 [1] >>> >>> Yet again, I wondered 'how many of our users are using IE9' as I >>> wondered if because of this lack of support we are losing out on >>> lots >>> of potential editors. >>> >>> What's the easiest way to get this information now? Is it
available?
>>> >>> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599 >>> >>> _______________________________________________ >>> Analytics mailing list >>> Analytics@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > > > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation > > _______________________________________________ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics >
-- Oliver Keyes Research Analyst Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
So the answer to this question (with preliminary data) is that neither 2.1 nor 2.2 amount to 0.05% of traffic to the mobile site.
I have attached the list of user agents and devices (with percentages) for the last 30 days. I did not included any device/browser combo with less than 0.05% of traffic.
For about 4% of traffic we could not identify the browser, this might be cause the user agent was not there or because ua-parser could not figure it out, I understand this is not ideal but I am sending this cause I feel this list provides quite a bit of value and should help you triage bugs.
iOS takes the cake which does not cease to amaze me.
I described what I did to gather the data here (anyone with permits to 1002 can repro): https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive/QueryUsingUDF
On Wed, Oct 15, 2014 at 12:15 PM, Nuria Ruiz nuria@wikimedia.org wrote:
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
Understood, it is hard for you guys to work without knowing this data. I will try to get a user agent list for data from last month but, as I mentioned earlier, I think providing this data in a regular basis (monthly?) is a good goal for us.
On Wed, Oct 15, 2014 at 10:35 AM, Jon Robson jrobson@wikimedia.org wrote:
Anything would be useful. I just hit this situation again. I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
In the mean time while I don't have a fancy place to find out the answers to this how can I get these answers? Should I mail the analytics mailing list to ask these questions? Cc a point person on bugzilla with the question? Ping someone privately?
Jon
On Tue, Oct 14, 2014 at 10:30 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
Since we talked about this on IRC, restating here to keep the archives happy. We pull the ua parser jar from our archiva depot, an update will involve building a new jar, uploading it to archiva and updating our dependency
file
(pom.xml) to point to the newly updated version.
On Fri, Oct 10, 2014 at 9:59 PM, Oliver Keyes okeyes@wikimedia.org
wrote:
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote:
- A UDF for ua-parser or whatever we decide to use (this will
possibly
be necessary for pageviews, but not necessarily - it depends on our >spider/automaton detection strategy)
We got this one ready today:
https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote:
On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote: > > >At some point I believe we hope to just, you know. Have a regularly > > updated browser matrix somewhere. > I REALLY think this should make it into our goals, if it cannot be
done
> this quarter it should for sure be done this quarter. >
I agree it would be nice. It's one of those things that will either
come
as a side-effect of other stuff, OR require subsantially more work,
and
nothing in-between. Things we need for it:
- A UDF for ua-parser or whatever we decide to use (this will
possibly
be necessary for pageviews, but not necessarily - it depends on our spider/automaton detection strategy) 2. Pageviews data 3. A table somewhere.
Take 1, apply to 2, stick in 3. Maybe grab the same data for
text/html
requests overall (depends on query runtime), maybe don't.
The ideal implementation, obviously, is to pair this up with a site
that
automatically parses the results into HTML. That should be the end
goal. but
in terms of engineering support we can get most of the way there
simply by
ensuring we always have a recent snapshot to hand. I can probably put something together over the sampled logs and throw it in SQL if
there are
urgent needs.
> > Do we not have more recent data than May?
We don't, but thanks to the utilities library I built, the code for generating it would literally run:
library(WMUtils) uas <-
as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
uas <- uas[,j = list(requests = .N, by = c("os","browser")]
write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names =
FALSE,
quote = TRUE)
...assuming we didn't care about readability.
Point is, in the time until we have the new parser built into Hadoop
and
that setup, we can totally generate interim data from the sampled
logs using
the same parser at a tiny cost in research/programming time, iff (the mathematical if) we need it enough that we're cool with the
sampling, and
people can convince [[Dario|Our Great Leader]] to authorise me to
spend 15
minutes of my time on it.
> > > On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes <
okeyes@wikimedia.org>
> wrote: >> >> Email Dario and I, if he prioritises it I'll run a check on more >> recent data. >> >> At some point I believe we hope to just, you know. Have a regularly >> updated browser matrix somewhere. This comes some time after
pageviews
>> though. >> >> On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org
wrote:
>>> >>> Hi Jon -- I'm sure other folks will have more information but
here's
>>> a link to a slide with some data from May[1]. We don't see a lot
of Windows
>>> phone traffic. >>> >>> -Toby >>> >>> [1] >>>
https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
>>> >>> On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson <
jrobson@wikimedia.org>
>>> wrote: >>>> >>>> I was going through our backlog again today, and I noticed a bug >>>> about >>>> supporting editing on Windows Phones with IE9 [1] >>>> >>>> Yet again, I wondered 'how many of our users are using IE9' as I >>>> wondered if because of this lack of support we are losing out on >>>> lots >>>> of potential editors. >>>> >>>> What's the easiest way to get this information now? Is it
available?
>>>> >>>> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599 >>>> >>>> _______________________________________________ >>>> Analytics mailing list >>>> Analytics@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>> >> >> >> >> -- >> Oliver Keyes >> Research Analyst >> Wikimedia Foundation >> >> _______________________________________________ >> Analytics mailing list >> Analytics@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >
-- Oliver Keyes Research Analyst Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
On Wed, Oct 15, 2014 at 12:12 PM, Andrew Otto aotto@wikimedia.org wrote:
Jon,
Recent unsampled webrequest logs are available for querying in Hive now!
https://wikitech.wikimedia.org/wiki/Analytics/Cluster
:)
If you don’t already have access for this, submit an RT request to get access to stat1002 and the analytics-privatedata-group.
That's good to know. Thanks. I'm not sure if I have stat1002 access but every time you mention RT I shudder ;-)
Thanks for the dump of data Nuria. I assume these all add up to 100% (roughly) and are global? So if I understand correctly, if I get the above access and follow your instructions I can get this data when I do need it until we have some nice page I can go to to retrieve it :).
This is good to know when we have these sort of questions so thanks a bunch. We are currently interested in phablet traffic (big screen mobile devices) so this should be useful information for us thanks!
On Thu, Oct 16, 2014 at 7:15 PM, Nuria Ruiz nuria@wikimedia.org wrote:
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
So the answer to this question (with preliminary data) is that neither 2.1 nor 2.2 amount to 0.05% of traffic to the mobile site.
I have attached the list of user agents and devices (with percentages) for the last 30 days. I did not included any device/browser combo with less than 0.05% of traffic.
For about 4% of traffic we could not identify the browser, this might be cause the user agent was not there or because ua-parser could not figure it out, I understand this is not ideal but I am sending this cause I feel this list provides quite a bit of value and should help you triage bugs.
iOS takes the cake which does not cease to amaze me.
I described what I did to gather the data here (anyone with permits to 1002 can repro): https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive/QueryUsingUDF
On Wed, Oct 15, 2014 at 12:15 PM, Nuria Ruiz nuria@wikimedia.org wrote:
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
Understood, it is hard for you guys to work without knowing this data. I will try to get a user agent list for data from last month but, as I mentioned earlier, I think providing this data in a regular basis (monthly?) is a good goal for us.
On Wed, Oct 15, 2014 at 10:35 AM, Jon Robson jrobson@wikimedia.org wrote:
Anything would be useful. I just hit this situation again. I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
In the mean time while I don't have a fancy place to find out the answers to this how can I get these answers? Should I mail the analytics mailing list to ask these questions? Cc a point person on bugzilla with the question? Ping someone privately?
Jon
On Tue, Oct 14, 2014 at 10:30 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
Since we talked about this on IRC, restating here to keep the archives happy. We pull the ua parser jar from our archiva depot, an update will involve building a new jar, uploading it to archiva and updating our dependency file (pom.xml) to point to the newly updated version.
On Fri, Oct 10, 2014 at 9:59 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote:
>1. A UDF for ua-parser or whatever we decide to use (this will > possibly > be necessary for pageviews, but not necessarily - it depends on our > >spider/automaton detection strategy) We got this one ready today: https://gerrit.wikimedia.org/r/#/c/166142/
On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes okeyes@wikimedia.org wrote: > > > > On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote: >> >> >At some point I believe we hope to just, you know. Have a >> > regularly >> > updated browser matrix somewhere. >> I REALLY think this should make it into our goals, if it cannot be >> done >> this quarter it should for sure be done this quarter. >> > > I agree it would be nice. It's one of those things that will either > come > as a side-effect of other stuff, OR require subsantially more work, > and > nothing in-between. Things we need for it: > > 1. A UDF for ua-parser or whatever we decide to use (this will > possibly > be necessary for pageviews, but not necessarily - it depends on our > spider/automaton detection strategy) > 2. Pageviews data > 3. A table somewhere. > > Take 1, apply to 2, stick in 3. Maybe grab the same data for > text/html > requests overall (depends on query runtime), maybe don't. > > The ideal implementation, obviously, is to pair this up with a site > that > automatically parses the results into HTML. That should be the end > goal. but > in terms of engineering support we can get most of the way there > simply by > ensuring we always have a recent snapshot to hand. I can probably > put > something together over the sampled logs and throw it in SQL if > there are > urgent needs. > >> >> Do we not have more recent data than May? > > > We don't, but thanks to the utilities library I built, the code for > generating it would literally run: > > library(WMUtils) > uas <- > > as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent)) > > uas <- uas[,j = list(requests = .N, by = c("os","browser")] > > write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = > FALSE, > quote = TRUE) > > ...assuming we didn't care about readability. > > Point is, in the time until we have the new parser built into Hadoop > and > that setup, we can totally generate interim data from the sampled > logs using > the same parser at a tiny cost in research/programming time, iff > (the > mathematical if) we need it enough that we're cool with the > sampling, and > people can convince [[Dario|Our Great Leader]] to authorise me to > spend 15 > minutes of my time on it. > >> >> >> On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes >> okeyes@wikimedia.org >> wrote: >>> >>> Email Dario and I, if he prioritises it I'll run a check on more >>> recent data. >>> >>> At some point I believe we hope to just, you know. Have a >>> regularly >>> updated browser matrix somewhere. This comes some time after >>> pageviews >>> though. >>> >>> On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org >>> wrote: >>>> >>>> Hi Jon -- I'm sure other folks will have more information but >>>> here's >>>> a link to a slide with some data from May[1]. We don't see a lot >>>> of Windows >>>> phone traffic. >>>> >>>> -Toby >>>> >>>> [1] >>>> >>>> https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca... >>>> >>>> On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson >>>> jrobson@wikimedia.org >>>> wrote: >>>>> >>>>> I was going through our backlog again today, and I noticed a bug >>>>> about >>>>> supporting editing on Windows Phones with IE9 [1] >>>>> >>>>> Yet again, I wondered 'how many of our users are using IE9' as I >>>>> wondered if because of this lack of support we are losing out on >>>>> lots >>>>> of potential editors. >>>>> >>>>> What's the easiest way to get this information now? Is it >>>>> available? >>>>> >>>>> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599 >>>>> >>>>> _______________________________________________ >>>>> Analytics mailing list >>>>> Analytics@lists.wikimedia.org >>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>> >>>> >>> >>> >>> >>> -- >>> Oliver Keyes >>> Research Analyst >>> Wikimedia Foundation >>> >>> _______________________________________________ >>> Analytics mailing list >>> Analytics@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >> > > > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
Thanks for the dump of data Nuria. I assume these all add up to 100% (roughly)
and are global? Roughly and global. Yes to both. As I said, "very" preliminary data. Good enough to triage bugs though.
So if I understand correctly, if I get the above access and follow your
instructions I can get this data when I
do need it until we have some nice page I can go to to retrieve it :).
Yes. I also will be happy to re-run it if needed be.
On Fri, Oct 17, 2014 at 11:30 AM, Jon Robson jrobson@wikimedia.org wrote:
On Wed, Oct 15, 2014 at 12:12 PM, Andrew Otto aotto@wikimedia.org wrote:
Jon,
Recent unsampled webrequest logs are available for querying in Hive now!
https://wikitech.wikimedia.org/wiki/Analytics/Cluster
:)
If you don’t already have access for this, submit an RT request to get
access to stat1002 and the analytics-privatedata-group.
That's good to know. Thanks. I'm not sure if I have stat1002 access but every time you mention RT I shudder ;-)
Thanks for the dump of data Nuria. I assume these all add up to 100% (roughly) and are global? So if I understand correctly, if I get the above access and follow your instructions I can get this data when I do need it until we have some nice page I can go to to retrieve it :).
This is good to know when we have these sort of questions so thanks a bunch. We are currently interested in phablet traffic (big screen mobile devices) so this should be useful information for us thanks!
On Thu, Oct 16, 2014 at 7:15 PM, Nuria Ruiz nuria@wikimedia.org wrote:
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
So the answer to this question (with preliminary data) is that neither
2.1
nor 2.2 amount to 0.05% of traffic to the mobile site.
I have attached the list of user agents and devices (with percentages)
for
the last 30 days. I did not included any device/browser combo with less
than
0.05% of traffic.
For about 4% of traffic we could not identify the browser, this might be cause the user agent was not there or because ua-parser could not figure
it
out, I understand this is not ideal but I am sending this cause I feel
this
list provides quite a bit of value and should help you triage bugs.
iOS takes the cake which does not cease to amaze me.
I described what I did to gather the data here (anyone with permits to
1002
can repro): https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive/QueryUsingUDF
On Wed, Oct 15, 2014 at 12:15 PM, Nuria Ruiz nuria@wikimedia.org
wrote:
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
Understood, it is hard for you guys to work without knowing this data. I will try to get a user agent list for data from last month but, as I mentioned earlier, I think providing this data in a regular basis
(monthly?)
is a good goal for us.
On Wed, Oct 15, 2014 at 10:35 AM, Jon Robson jrobson@wikimedia.org wrote:
Anything would be useful. I just hit this situation again. I was reviewing some code and someone used JSON.stringify - this is not available in Android < 2.3 and I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic.
In the mean time while I don't have a fancy place to find out the answers to this how can I get these answers? Should I mail the analytics mailing list to ask these questions? Cc a point person on bugzilla with the question? Ping someone privately?
Jon
On Tue, Oct 14, 2014 at 10:30 AM, Nuria Ruiz nuria@wikimedia.org
wrote:
Woah! Nice :D How are definitions updates handled?
Since we talked about this on IRC, restating here to keep the
archives
happy. We pull the ua parser jar from our archiva depot, an update will involve building a new jar, uploading it to archiva and updating our
dependency
file (pom.xml) to point to the newly updated version.
On Fri, Oct 10, 2014 at 9:59 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Woah! Nice :D How are definitions updates handled?
On 10 October 2014 18:58, Nuria Ruiz nuria@wikimedia.org wrote: > > >1. A UDF for ua-parser or whatever we decide to use (this will > > possibly > > be necessary for pageviews, but not necessarily - it depends on
our
> > >spider/automaton detection strategy) > We got this one ready today: > https://gerrit.wikimedia.org/r/#/c/166142/ > > > > > On Fri, Oct 10, 2014 at 3:55 PM, Oliver Keyes <
okeyes@wikimedia.org>
> wrote: >> >> >> >> On 10 October 2014 16:02, Nuria Ruiz nuria@wikimedia.org wrote: >>> >>> >At some point I believe we hope to just, you know. Have a >>> > regularly >>> > updated browser matrix somewhere. >>> I REALLY think this should make it into our goals, if it cannot
be
>>> done >>> this quarter it should for sure be done this quarter. >>> >> >> I agree it would be nice. It's one of those things that will
either
>> come >> as a side-effect of other stuff, OR require subsantially more
work,
>> and >> nothing in-between. Things we need for it: >> >> 1. A UDF for ua-parser or whatever we decide to use (this will >> possibly >> be necessary for pageviews, but not necessarily - it depends on
our
>> spider/automaton detection strategy) >> 2. Pageviews data >> 3. A table somewhere. >> >> Take 1, apply to 2, stick in 3. Maybe grab the same data for >> text/html >> requests overall (depends on query runtime), maybe don't. >> >> The ideal implementation, obviously, is to pair this up with a
site
>> that >> automatically parses the results into HTML. That should be the end >> goal. but >> in terms of engineering support we can get most of the way there >> simply by >> ensuring we always have a recent snapshot to hand. I can probably >> put >> something together over the sampled logs and throw it in SQL if >> there are >> urgent needs. >> >>> >>> Do we not have more recent data than May? >> >> >> We don't, but thanks to the utilities library I built, the code
for
>> generating it would literally run: >> >> library(WMUtils) >> uas <- >> >>
as.data.table(ua_parse(data_sieve(do.call("rbind",lapply(seq(20140901,20140930,1),sampled_logs)))$user_agent))
>> >> uas <- uas[,j = list(requests = .N, by = c("os","browser")] >> >> write.table(uas, file = uas_for_jon.tsv, sep = "\t", row.names = >> FALSE, >> quote = TRUE) >> >> ...assuming we didn't care about readability. >> >> Point is, in the time until we have the new parser built into
Hadoop
>> and >> that setup, we can totally generate interim data from the sampled >> logs using >> the same parser at a tiny cost in research/programming time, iff >> (the >> mathematical if) we need it enough that we're cool with the >> sampling, and >> people can convince [[Dario|Our Great Leader]] to authorise me to >> spend 15 >> minutes of my time on it. >> >>> >>> >>> On Fri, Oct 10, 2014 at 12:45 PM, Oliver Keyes >>> okeyes@wikimedia.org >>> wrote: >>>> >>>> Email Dario and I, if he prioritises it I'll run a check on more >>>> recent data. >>>> >>>> At some point I believe we hope to just, you know. Have a >>>> regularly >>>> updated browser matrix somewhere. This comes some time after >>>> pageviews >>>> though. >>>> >>>> On 10 October 2014 14:38, Toby Negrin tnegrin@wikimedia.org >>>> wrote: >>>>> >>>>> Hi Jon -- I'm sure other folks will have more information but >>>>> here's >>>>> a link to a slide with some data from May[1]. We don't see a
lot
>>>>> of Windows >>>>> phone traffic. >>>>> >>>>> -Toby >>>>> >>>>> [1] >>>>> >>>>>
https://docs.google.com/a/wikimedia.org/presentation/d/19tZgTi6VUG04wfGWVzca...
>>>>> >>>>> On Fri, Oct 10, 2014 at 11:17 AM, Jon Robson >>>>> jrobson@wikimedia.org >>>>> wrote: >>>>>> >>>>>> I was going through our backlog again today, and I noticed a
bug
>>>>>> about >>>>>> supporting editing on Windows Phones with IE9 [1] >>>>>> >>>>>> Yet again, I wondered 'how many of our users are using IE9'
as I
>>>>>> wondered if because of this lack of support we are losing out
on
>>>>>> lots >>>>>> of potential editors. >>>>>> >>>>>> What's the easiest way to get this information now? Is it >>>>>> available? >>>>>> >>>>>> [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=55599 >>>>>> >>>>>> _______________________________________________ >>>>>> Analytics mailing list >>>>>> Analytics@lists.wikimedia.org >>>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Oliver Keyes >>>> Research Analyst >>>> Wikimedia Foundation >>>> >>>> _______________________________________________ >>>> Analytics mailing list >>>> Analytics@lists.wikimedia.org >>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>> >>> >> >> >> >> -- >> Oliver Keyes >> Research Analyst >> Wikimedia Foundation > >
-- Oliver Keyes Research Analyst Wikimedia Foundation
Hi,
[ leaving other things in this thread aside ]
On Fri, Oct 17, 2014 at 11:30:37AM -0700, Jon Robson wrote:
I assume these all add up to 100% (roughly) [...]
It sums to 67.328 for me [1].
Have fun, Christian
[1] _________________________________________________________________ christian@spencer // jobs: 0 // time: 23:18:19 // exit code: 0 cwd: ~ LC_ALL=C qalc $(cat user_agent_percentages_october_2014.txt | cut -f 1 -d % | tr '\n' '+') 0 0.05 + 0.05 + 0.05 + 0.05 + 0.051 + 0.051 + 0.051 + 0.051 + 0.052 + 0.052 + 0.053 + 0.053 + 0.053 + 0.053 + 0.054 + 0.054 + 0.055 + 0.055 + 0.055 + 0.055 + 0.056 + 0.057 + 0.057 + 0.058 + 0.058 + 0.058 + 0.058 + 0.058 + 0.058 + 0.058 + 0.059 + 0.059 + 0.059 + 0.059 + 0.06 + 0.06 + 0.06 + 0.061 + 0.061 + 0.061 + 0.063 + 0.063 + 0.064 + 0.064 + 0.065 + 0.065 + 0.066 + 0.066 + 0.066 + 0.066 + 0.067 + 0.067 + 0.067 + 0.067 + 0.068 + 0.068 + 0.068 + 0.069 + 0.069 + 0.069 + 0.07 + 0.071 + 0.073 + 0.074 + 0.074 + 0.075 + 0.076 + 0.077 + 0.079 + 0.08 + 0.081 + 0.081 + 0.082 + 0.083 + 0.084 + 0.085 + 0.085 + 0.089 + 0.09 + 0.091 + 0.092 + 0.093 + 0.095 + 0.096 + 0.096 + 0.097 + 0.097 + 0.101 + 0.103 + 0.104 + 0.104 + 0.105 + 0.106 + 0.111 + 0.112 + 0.116 + 0.119 + 0.127 + 0.134 + 0.137 + 0.138 + 0.139 + 0.139 + 0.14 + 0.141 + 0.141 + 0.141 + 0.145 + 0.146 + 0.147 + 0.148 + 0.15 + 0.156 + 0.156 + 0.159 + 0.159 + 0.163 + 0.172 + 0.184 + 0.185 + 0.19 + 0.197 + 0.198 + 0.204 + 0.207 + 0.21 + 0.22 + 0.222 + 0.223 + 0.229 + 0.229 + 0.232 + 0.238 + 0.24 + 0.257 + 0.272 + 0.288 + 0.307 + 0.324 + 0.342 + 0.351 + 0.357 + 0.375 + 0.385 + 0.409 + 0.428 + 0.473 + 0.502 + 0.575 + 0.607 + 0.747 + 0.8 + 0.841 + 1.017 + 1.199 + 1.227 + 1.38 + 1.397 + 1.928 + 2.709 + 3.05 + 4.713 + 5.098 + 8.823 + 12.764 + 0 = 67.328
Hi,
[ leaving other things in this thread aside ]
On Thu, Oct 16, 2014 at 07:15:03PM -0700, Nuria Ruiz wrote:
iOS takes the cake which does not cease to amaze me.
Not sure what exactly that means, but note that the long-tail got cut off. Due to the way the data is sliced, I expect the “long-tail cut off" to have thrown out a big part of requests from Android devices.
Have fun, Christian
It sums to 67.328 for me [1].
You are right. I should have clarified that. In the list attached I only included device+browser combinations that represent at least 0.05% of traffic. I removed the longtail. Will send Jon a file with the longtail as he probably would care to see that.
And I have no idea what our traffic for Android 2.1 and 2.2 is and if it is significant e.g. more than 1% of our traffic. So the answer to this question (with preliminary data) is that neither 2.1
nor 2.2 amount to 0.05% of traffic to the mobile site. Should have said "So the answer to this question (with preliminary data) is that neither 2.1 nor 2.2 amount to 1% of traffic to the mobile site"
On Fri, Oct 17, 2014 at 2:37 PM, Christian Aistleitner < christian@quelltextlich.at> wrote:
Hi,
[ leaving other things in this thread aside ]
On Thu, Oct 16, 2014 at 07:15:03PM -0700, Nuria Ruiz wrote:
iOS takes the cake which does not cease to amaze me.
Not sure what exactly that means, but note that the long-tail got cut off. Due to the way the data is sliced, I expect the “long-tail cut off" to have thrown out a big part of requests from Android devices.
Have fun, Christian
-- ---- quelltextlich e.U. ---- \ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christian@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax: +43 7946 / 20 5 81 Homepage: http://quelltextlich.at/
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics