Good news, everyone!
Oliver ran some stats for us on where tablet users were landing (desktop view or mobile view) a few days before and after the tablet redirect on Tuesday:
Before the redirect, *13% of pageviews from tablets were hitting the mobile site*, meaning a non-trivial chunk of users had opted into the mobile experience even before we were serving it to them as the default.
After the redirect, *95% of pageviews from tablets were hitting the mobile site*, meaning a fairly small number of users are opting out of the mobile experience we're serving – smaller than the number of people who were opting out of the desktop experience before the redirect :)
(Note that the total pageview numbers haven't changed significantly – the slightly downward trendline in the past few days just reflects the normal dip we observe toward the end of every week.)
So, tl;dr, it looks like we're serving the needs of the majority of our users. Graph below, courtesy of our beloved British data analyst/road warrior. We'll continue to monitor these numbers, of course, but the early results look good – nice work, team!
[image: Inline image 1]
Great to see the positive quantitate response.
Maryana, what's the high level of view of how your thinking about moving forward?
--tomasz
On Fri, Jun 20, 2014 at 10:33 AM, Maryana Pinchuk mpinchuk@wikimedia.org wrote:
Good news, everyone!
Oliver ran some stats for us on where tablet users were landing (desktop view or mobile view) a few days before and after the tablet redirect on Tuesday:
Before the redirect, *13% of pageviews from tablets were hitting the mobile site*, meaning a non-trivial chunk of users had opted into the mobile experience even before we were serving it to them as the default.
After the redirect, *95% of pageviews from tablets were hitting the mobile site*, meaning a fairly small number of users are opting out of the mobile experience we're serving – smaller than the number of people who were opting out of the desktop experience before the redirect :)
(Note that the total pageview numbers haven't changed significantly – the slightly downward trendline in the past few days just reflects the normal dip we observe toward the end of every week.)
So, tl;dr, it looks like we're serving the needs of the majority of our users. Graph below, courtesy of our beloved British data analyst/road warrior. We'll continue to monitor these numbers, of course, but the early results look good – nice work, team!
[image: Inline image 1]
-- Maryana Pinchuk Product Manager, Wikimedia Foundation wikimediafoundation.org
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On Fri, Jun 20, 2014 at 10:52 AM, Tomasz Finc tfinc@wikimedia.org wrote:
Great to see the positive quantitate response.
Maryana, what's the high level of view of how your thinking about moving forward?
I think we'll want to let the changes simmer for another week or two and then dig into a few specific things:
1) Do the pageview stats hold? e.g., are we still getting relatively few opt-outs? 2) How has this change impacted new mobile user numbers (registrations, 1st time editors, active editors, etc.)? Early results[1] show a healthy spike in new signups coming from mobile in the past few days, and we need to keep track of these users to make sure they're going through the new and active editor funnels at a reasonable rate. 3) How does the heatmap of taps look? e.g., how often are people accessing hamburger menu items, language selection, history, etc.
Based on the data from the above, we could do some UX optimization – the designers are very keen to refine the UI for page actions (instead of dumping features like language selection and page history at the top/bottom of the page kind of at random). If the new and active editor conversion rates are low, we'll need to prioritize work on new contribution funnels, but realistically that's probably not something we'll be able to tackle until Q2 of next fiscal.
1. http://ee-dashboard.wmflabs.org/graphs/enwiki_daily_reg_mobile
--tomasz
On Fri, Jun 20, 2014 at 10:33 AM, Maryana Pinchuk mpinchuk@wikimedia.org wrote:
Good news, everyone!
Oliver ran some stats for us on where tablet users were landing (desktop view or mobile view) a few days before and after the tablet redirect on Tuesday:
Before the redirect, *13% of pageviews from tablets were hitting the mobile site*, meaning a non-trivial chunk of users had opted into the mobile experience even before we were serving it to them as the default.
After the redirect, *95% of pageviews from tablets were hitting the mobile site*, meaning a fairly small number of users are opting out of the mobile experience we're serving – smaller than the number of people who were opting out of the desktop experience before the redirect :)
(Note that the total pageview numbers haven't changed significantly – the slightly downward trendline in the past few days just reflects the normal dip we observe toward the end of every week.)
So, tl;dr, it looks like we're serving the needs of the majority of our users. Graph below, courtesy of our beloved British data analyst/road warrior. We'll continue to monitor these numbers, of course, but the early results look good – nice work, team!
[image: Inline image 1]
-- Maryana Pinchuk Product Manager, Wikimedia Foundation wikimediafoundation.org
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Maryana Pinchuk, 21/06/2014 00:29:
- How has this change impacted [...]
1st time editors, active editors, etc.)? [...] we need to keep track of these users to make sure they're going through the new and active editor funnels at a reasonable rate.
Nice! I think this also needs to be reflected (at a very high level of course) in the yearly goals: https://www.mediawiki.org/w/index.php?title=Talk%3AWikimedia_Engineering%2F2...
Nemo
On Jun 23, 2014, at 1:57 AM, "Federico Leva (Nemo)" nemowiki@gmail.com wrote:
Maryana Pinchuk, 21/06/2014 00:29:
- How has this change impacted [...]
1st time editors, active editors, etc.)? [...] we need to keep track of these users to make sure they're going through the new and active editor funnels at a reasonable rate.
Nice! I think this also needs to be reflected (at a very high level of course) in the yearly goals: https://www.mediawiki.org/w/index.php?title=Talk%3AWikimedia_Engineering%2F2...
It is :) The quarterly quantitative targets in that draft doc mention maintaining and growing "new mobile active editors" -- this metric represents new users registering on mobile and then going on to hit the active editor threshold (5+ edits per month) on either mobile or desktop. I'll make that more explicit in the doc in case it's not clear.
On Fri, Jun 20, 2014 at 3:29 PM, Maryana Pinchuk mpinchuk@wikimedia.org wrote:
I think we'll want to let the changes simmer for another week or two and then dig into a few specific things:
Yes. We really need more data to make future decisions.
- Do the pageview stats hold? e.g., are we still getting relatively few
opt-outs?
Do we have this in LIMN ?
- How has this change impacted new mobile user numbers (registrations,
1st time editors, active editors, etc.)? Early results[1] show a healthy spike in new signups coming from mobile in the past few days, and we need to keep track of these users to make sure they're going through the new and active editor funnels at a reasonable rate.
Great. I tried to find any other metrics on the ee dashboard but I just got a white page. Do you have a better link for us? I looked on the mobile report card and found this which just can't be right
http://mobile-reportcard.wmflabs.org/graphs/successful-edits-main
- How does the heatmap of taps look? e.g., how often are people accessing
hamburger menu items, language selection, history, etc.
Eager to see the write ups of this.
Based on the data from the above, we could do some UX optimization – the
designers are very keen to refine the UI for page actions (instead of dumping features like language selection and page history at the top/bottom of the page kind of at random). If the new and active editor conversion rates are low, we'll need to prioritize work on new contribution funnels, but realistically that's probably not something we'll be able to tackle until Q2 of next fiscal.
That makes sense. Great work thus far.
--tomasz
On 23 June 2014 10:41, Tomasz Finc tfinc@wikimedia.org wrote:
On Fri, Jun 20, 2014 at 3:29 PM, Maryana Pinchuk mpinchuk@wikimedia.org wrote:
I think we'll want to let the changes simmer for another week or two and then dig into a few specific things:
Yes. We really need more data to make future decisions.
- Do the pageview stats hold? e.g., are we still getting relatively few
opt-outs?
Do we have this in LIMN ?
Nope, just static reports at the moment. Given the speedy nature of the request (Both figuratively - there was a narrow window to produce it - and literally, because I wrote most of the code while travelling through Oregon at 85 MP/H) I'm not tremendously confident in the ability of the code to indefinitely generate data.
- How has this change impacted new mobile user numbers (registrations,
1st time editors, active editors, etc.)? Early results[1] show a healthy spike in new signups coming from mobile in the past few days, and we need to keep track of these users to make sure they're going through the new and active editor funnels at a reasonable rate.
Great. I tried to find any other metrics on the ee dashboard but I just got a white page. Do you have a better link for us? I looked on the mobile report card and found this which just can't be right
http://mobile-reportcard.wmflabs.org/graphs/successful-edits-main
- How does the heatmap of taps look? e.g., how often are people
accessing hamburger menu items, language selection, history, etc.
Eager to see the write ups of this.
Based on the data from the above, we could do some UX optimization – the
designers are very keen to refine the UI for page actions (instead of dumping features like language selection and page history at the top/bottom of the page kind of at random). If the new and active editor conversion rates are low, we'll need to prioritize work on new contribution funnels, but realistically that's probably not something we'll be able to tackle until Q2 of next fiscal.
That makes sense. Great work thus far.
--tomasz
On Mon, Jun 23, 2014 at 10:47 AM, Oliver Keyes okeyes@wikimedia.org wrote:
Nope, just static reports at the moment. Given the speedy nature of the request (Both figuratively - there was a narrow window to produce it - and literally, because I wrote most of the code while travelling through Oregon at 85 MP/H) I'm not tremendously confident in the ability of the code to indefinitely generate data
That's fine.
Maryana, it would be good to keep track of this over the quarter.
Where would this sit on your priority list of analytics requests that need to have complete/scalable implementations ?
if there is already a backlog of these then feel free to point me to it.
--tomasz
Tomasz,
one of the analytics goals for Q1-2015 is to deliver traffic metric definitions (primarily: pageviews, unique clients) and their breakdown by target site, device or device class and geography.
We will keep monitoring page requests using the interim definitions Oliver applied to the sampled logs [1], but there’s more work that needs to be done to turn these into fully vetted, production-level reports generated from the unsampled logs.
We’re currently turning the mobile analytics priorities discussed with Howie, Maryana and Dan into cards and we’ll share the list once it’s completed.
Mobile is a focus area for Q1 with virtually two dedicated people from Research & Data supporting the team with traffic and contribution research. We’ll be also reinforcing our traffic crunching capacity with a new dedicated research position that we’ll be opening in Q1 and realistically we should expect to have onboard in Q2.
Dario
[1] https://trello.com/c/DCd58xGQ/334-daily-pv-from-sampled-logs
On Jun 23, 2014, at 10:58 AM, Tomasz Finc tfinc@wikimedia.org wrote:
On Mon, Jun 23, 2014 at 10:47 AM, Oliver Keyes okeyes@wikimedia.org wrote:
Nope, just static reports at the moment. Given the speedy nature of the request (Both figuratively - there was a narrow window to produce it - and literally, because I wrote most of the code while travelling through Oregon at 85 MP/H) I'm not tremendously confident in the ability of the code to indefinitely generate data
That's fine.
Maryana, it would be good to keep track of this over the quarter.
Where would this sit on your priority list of analytics requests that need to have complete/scalable implementations ?
if there is already a backlog of these then feel free to point me to it.
--tomasz
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
So; we have the second week of results.
To set the stage: ISP caches will be cleared, so we might see desktop drop. Weekend traffic will show up, which means we might see the type of people accessing wikipedia change, and bring new preferences and demographics into play. It's been longer since the switchover, so people who dislike the change have had more chances to find the opt-out. It's the data wheel of fortune, and nobody knows where your spin will take you!
The answer is "a good place". The weekend made absolutely no difference; it bumped the amount of desktop traffic by a tiny amount, compared to a big increase for mobile, which suggests not just that most people have happily switched over but that the people who have switched are our most frequent and active visitors. Day-on-day, we saw no significant increase in desktop opt-ins - in fact, a slight decrease from the (already tiny) 5-ish percent.
Looks like being WP:BOLD and switching our tablet users to mobile in one fell swoop was a good decision, and our readers think so too :).
On 25 June 2014 14:57, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
Tomasz,
one of the analytics goals for Q1-2015 is to deliver traffic metric definitions (primarily: pageviews, unique clients) and their breakdown by target site, device or device class and geography.
We will keep monitoring page requests using the interim definitions Oliver applied to the sampled logs [1], but there’s more work that needs to be done to turn these into fully vetted, production-level reports generated from the unsampled logs.
We’re currently turning the mobile analytics priorities discussed with Howie, Maryana and Dan into cards and we’ll share the list once it’s completed.
Mobile is a focus area for Q1 with virtually two dedicated people from Research & Data supporting the team with traffic and contribution research. We’ll be also reinforcing our traffic crunching capacity with a new dedicated research position that we’ll be opening in Q1 and realistically we should expect to have onboard in Q2.
Dario
[1] https://trello.com/c/DCd58xGQ/334-daily-pv-from-sampled-logs
On Jun 23, 2014, at 10:58 AM, Tomasz Finc tfinc@wikimedia.org wrote:
On Mon, Jun 23, 2014 at 10:47 AM, Oliver Keyes okeyes@wikimedia.org
wrote:
Nope, just static reports at the moment. Given the speedy nature of the request (Both figuratively - there was a narrow window to produce it -
and
literally, because I wrote most of the code while travelling through
Oregon
at 85 MP/H) I'm not tremendously confident in the ability of the code to indefinitely generate data
That's fine.
Maryana, it would be good to keep track of this over the quarter.
Where would this sit on your priority list of analytics requests that need to have complete/scalable implementations ?
if there is already a backlog of these then feel free to point me to it.
--tomasz
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On the contribution side, the switchover also had a positive impact on new user activation [1].
iPad users who were previously signing up on the desktop site saw a significantly (and substantially) higher activation rate [2] as a result of the redirection, for most days we sampled after the switchover.
It’s too early to tell if this change in new user engagement will persist (we probably had a large number of testers among the first signups) and we know that by design (anonymous edit restrictions, prominent CTAs) we should expect to see a higher conversion for editors on the mobile site, but these results are very encouraging: we’re seeing a much higher rate of new users to contribute to start editing Wikipedia in an environment that is more appropriate for tablet devices.
Dario
[1] measured as the proportion of newly registered users who complete at least 1 edit in their first 24 hours, English Wikipedia. [2] https://trello.com/c/LKVK7RfL/351-tablet-switchover-and-editor-activation
On Jun 27, 2014, at 1:37 PM, Oliver Keyes okeyes@wikimedia.org wrote:
So; we have the second week of results.
To set the stage: ISP caches will be cleared, so we might see desktop drop. Weekend traffic will show up, which means we might see the type of people accessing wikipedia change, and bring new preferences and demographics into play. It's been longer since the switchover, so people who dislike the change have had more chances to find the opt-out. It's the data wheel of fortune, and nobody knows where your spin will take you!
The answer is "a good place". The weekend made absolutely no difference; it bumped the amount of desktop traffic by a tiny amount, compared to a big increase for mobile, which suggests not just that most people have happily switched over but that the people who have switched are our most frequent and active visitors. Day-on-day, we saw no significant increase in desktop opt-ins - in fact, a slight decrease from the (already tiny) 5-ish percent.
Looks like being WP:BOLD and switching our tablet users to mobile in one fell swoop was a good decision, and our readers think so too :).
On 25 June 2014 14:57, Dario Taraborelli dtaraborelli@wikimedia.org wrote: Tomasz,
one of the analytics goals for Q1-2015 is to deliver traffic metric definitions (primarily: pageviews, unique clients) and their breakdown by target site, device or device class and geography.
We will keep monitoring page requests using the interim definitions Oliver applied to the sampled logs [1], but there’s more work that needs to be done to turn these into fully vetted, production-level reports generated from the unsampled logs.
We’re currently turning the mobile analytics priorities discussed with Howie, Maryana and Dan into cards and we’ll share the list once it’s completed.
Mobile is a focus area for Q1 with virtually two dedicated people from Research & Data supporting the team with traffic and contribution research. We’ll be also reinforcing our traffic crunching capacity with a new dedicated research position that we’ll be opening in Q1 and realistically we should expect to have onboard in Q2.
Dario
[1] https://trello.com/c/DCd58xGQ/334-daily-pv-from-sampled-logs
On Jun 23, 2014, at 10:58 AM, Tomasz Finc tfinc@wikimedia.org wrote:
On Mon, Jun 23, 2014 at 10:47 AM, Oliver Keyes okeyes@wikimedia.org wrote:
Nope, just static reports at the moment. Given the speedy nature of the request (Both figuratively - there was a narrow window to produce it - and literally, because I wrote most of the code while travelling through Oregon at 85 MP/H) I'm not tremendously confident in the ability of the code to indefinitely generate data
That's fine.
Maryana, it would be good to keep track of this over the quarter.
Where would this sit on your priority list of analytics requests that need to have complete/scalable implementations ?
if there is already a backlog of these then feel free to point me to it.
--tomasz
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- Oliver Keyes Research Analyst Wikimedia Foundation <2014-06-13_to_2014-06-25.png>
On Fri, Jun 27, 2014 at 2:26 PM, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
On the contribution side, the switchover also had a positive impact on new user activation [1].
iPad users who were previously signing up on the desktop site saw a significantly (and substantially) higher activation rate [2] as a result of the redirection, for most days we sampled after the switchover.
It’s too early to tell if this change in new user engagement will persist (we probably had a large number of testers among the first signups) and we know that by design (anonymous edit restrictions, prominent CTAs) we should expect to see a higher conversion for editors on the mobile site, but these results are very encouraging: we’re seeing a much higher rate of new users to contribute to start editing Wikipedia in an environment that is more appropriate for tablet devices.
Very cool data, thanks Dario! Is it possible to quantify before/after total edit volume to account for the impact of the anonymous edit restriction?
Erik
On Jun 27, 2014, at 2:56 PM, Erik Moeller erik@wikimedia.org wrote:
On Fri, Jun 27, 2014 at 2:26 PM, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
On the contribution side, the switchover also had a positive impact on new user activation [1].
iPad users who were previously signing up on the desktop site saw a significantly (and substantially) higher activation rate [2] as a result of the redirection, for most days we sampled after the switchover.
It’s too early to tell if this change in new user engagement will persist (we probably had a large number of testers among the first signups) and we know that by design (anonymous edit restrictions, prominent CTAs) we should expect to see a higher conversion for editors on the mobile site, but these results are very encouraging: we’re seeing a much higher rate of new users to contribute to start editing Wikipedia in an environment that is more appropriate for tablet devices.
Very cool data, thanks Dario! Is it possible to quantify before/after total edit volume to account for the impact of the anonymous edit restriction?
I am looking into overall edit volume as well. We should keep in mind that this will be a noisy measure (much less controlled than new user activation rates).
Other hypotheses we’re formulating on the expected impact of the tablet switchover are listed here [1], feel free to add more.
Dario
[1] http://etherpad.wikimedia.org/p/TabletSwitchover-UserAcquisition
On the contribution side, the switchover also had a positive impact on new user activation [1].
It’s too early to tell if this change in new user engagement will persist (we probably had a large number of testers among the first signups) and we know that by design (anonymous edit restrictions, prominent CTAs) we should expect to see a higher conversion for editors on the mobile site, but these results are very encouraging: we’re seeing a much higher rate of new users to contribute to start editing Wikipedia in an environment that is more appropriate for tablet devices.
Dario
[1] measured as the proportion of newly registered users who complete at least 1 edit in their first 24 hours, English Wikipedia. [2] https://trello.com/c/LKVK7RfL/351-tablet-switchover-and-editor-activation
Forwarding to mobile-l, because it needs approval, and commonsifying the graphic - https://commons.wikimedia.org/wiki/File:Tablet_desktop_to_mobile_switchover....
---------- Forwarded message ---------- From: Oliver Keyes okeyes@wikimedia.org Date: 27 June 2014 16:37 Subject: Re: [WikimediaMobile] Tablet pageview data pre and post redirect To: Dario Taraborelli dtaraborelli@wikimedia.org Cc: Tomasz Finc tfinc@wikimedia.org, mobile-l < mobile-l@lists.wikimedia.org>
So; we have the second week of results.
To set the stage: ISP caches will be cleared, so we might see desktop drop. Weekend traffic will show up, which means we might see the type of people accessing wikipedia change, and bring new preferences and demographics into play. It's been longer since the switchover, so people who dislike the change have had more chances to find the opt-out. It's the data wheel of fortune, and nobody knows where your spin will take you!
The answer is "a good place". The weekend made absolutely no difference; it bumped the amount of desktop traffic by a tiny amount, compared to a big increase for mobile, which suggests not just that most people have happily switched over but that the people who have switched are our most frequent and active visitors. Day-on-day, we saw no significant increase in desktop opt-ins - in fact, a slight decrease from the (already tiny) 5-ish percent.
Looks like being WP:BOLD and switching our tablet users to mobile in one fell swoop was a good decision, and our readers think so too :).
On 25 June 2014 14:57, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
Tomasz,
one of the analytics goals for Q1-2015 is to deliver traffic metric definitions (primarily: pageviews, unique clients) and their breakdown by target site, device or device class and geography.
We will keep monitoring page requests using the interim definitions Oliver applied to the sampled logs [1], but there’s more work that needs to be done to turn these into fully vetted, production-level reports generated from the unsampled logs.
We’re currently turning the mobile analytics priorities discussed with Howie, Maryana and Dan into cards and we’ll share the list once it’s completed.
Mobile is a focus area for Q1 with virtually two dedicated people from Research & Data supporting the team with traffic and contribution research. We’ll be also reinforcing our traffic crunching capacity with a new dedicated research position that we’ll be opening in Q1 and realistically we should expect to have onboard in Q2.
Dario
[1] https://trello.com/c/DCd58xGQ/334-daily-pv-from-sampled-logs
On Jun 23, 2014, at 10:58 AM, Tomasz Finc tfinc@wikimedia.org wrote:
On Mon, Jun 23, 2014 at 10:47 AM, Oliver Keyes okeyes@wikimedia.org
wrote:
Nope, just static reports at the moment. Given the speedy nature of the request (Both figuratively - there was a narrow window to produce it -
and
literally, because I wrote most of the code while travelling through
Oregon
at 85 MP/H) I'm not tremendously confident in the ability of the code to indefinitely generate data
That's fine.
Maryana, it would be good to keep track of this over the quarter.
Where would this sit on your priority list of analytics requests that need to have complete/scalable implementations ?
if there is already a backlog of these then feel free to point me to it.
--tomasz
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On Wed, Jun 25, 2014 at 11:57 AM, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
We’re currently turning the mobile analytics priorities discussed with Howie, Maryana and Dan into cards and we’ll share the list once it’s completed.
Thanks Dario. Do send out when its ready.
--tomasz
I wonder how much of these numbers - tablets on mobile site before the switchover and on desktop site after it - are due to discrepancies in tablet detection between analytics and varnish. Varnishes used to detect them by matching user-agent against (iPad|Android.3|(?i)tablet|PlayBook|Wii) - how does analytics do that?
On Fri, Jun 20, 2014 at 10:33 AM, Maryana Pinchuk mpinchuk@wikimedia.org wrote:
Good news, everyone!
Oliver ran some stats for us on where tablet users were landing (desktop view or mobile view) a few days before and after the tablet redirect on Tuesday:
Before the redirect, *13% of pageviews from tablets were hitting the mobile site*, meaning a non-trivial chunk of users had opted into the mobile experience even before we were serving it to them as the default.
After the redirect, *95% of pageviews from tablets were hitting the mobile site*, meaning a fairly small number of users are opting out of the mobile experience we're serving – smaller than the number of people who were opting out of the desktop experience before the redirect :)
(Note that the total pageview numbers haven't changed significantly – the slightly downward trendline in the past few days just reflects the normal dip we observe toward the end of every week.)
So, tl;dr, it looks like we're serving the needs of the majority of our users. Graph below, courtesy of our beloved British data analyst/road warrior. We'll continue to monitor these numbers, of course, but the early results look good – nice work, team!
[image: Inline image 1]
-- Maryana Pinchuk Product Manager, Wikimedia Foundation wikimediafoundation.org
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
We use tobie's ua-parser, which should indeed hit a slightly wider range of tablets and would explain some of that 5%. We should probably have a different conversation with ops about trying to rebalance the varnish tablet identification (I've seen the request logs, and that won't catch all of the tablets).
Re the drop - it's actually the mid-week drop (mobile devices, tablets included, gain a chunk of our user share on weekends). The script is set to loop every Friday, so there'll be additional data this time next week which should allow us to see if the percentages later as, e.g, ISP-side caching changes.
On Friday, June 20, 2014, Max Semenik maxsem.wiki@gmail.com wrote:
I wonder how much of these numbers - tablets on mobile site before the switchover and on desktop site after it - are due to discrepancies in tablet detection between analytics and varnish. Varnishes used to detect them by matching user-agent against (iPad|Android.3|(?i)tablet|PlayBook|Wii) - how does analytics do that?
On Fri, Jun 20, 2014 at 10:33 AM, Maryana Pinchuk <mpinchuk@wikimedia.org javascript:_e(%7B%7D,'cvml','mpinchuk@wikimedia.org');> wrote:
Good news, everyone!
Oliver ran some stats for us on where tablet users were landing (desktop view or mobile view) a few days before and after the tablet redirect on Tuesday:
Before the redirect, *13% of pageviews from tablets were hitting the mobile site*, meaning a non-trivial chunk of users had opted into the mobile experience even before we were serving it to them as the default.
After the redirect, *95% of pageviews from tablets were hitting the mobile site*, meaning a fairly small number of users are opting out of the mobile experience we're serving – smaller than the number of people who were opting out of the desktop experience before the redirect :)
(Note that the total pageview numbers haven't changed significantly – the slightly downward trendline in the past few days just reflects the normal dip we observe toward the end of every week.)
So, tl;dr, it looks like we're serving the needs of the majority of our users. Graph below, courtesy of our beloved British data analyst/road warrior. We'll continue to monitor these numbers, of course, but the early results look good – nice work, team!
[image: Inline image 1]
-- Maryana Pinchuk Product Manager, Wikimedia Foundation wikimediafoundation.org
Mobile-l mailing list Mobile-l@lists.wikimedia.org javascript:_e(%7B%7D,'cvml','Mobile-l@lists.wikimedia.org'); https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- Best regards, Max Semenik ([[User:MaxSem]])
On Fri, Jun 20, 2014 at 11:55 AM, Oliver Keyes okeyes@wikimedia.org wrote:
We should probably have a different conversation with ops about trying to rebalance the varnish tablet identification (I've seen the request logs, and that won't catch all of the tablets).
Luckily, Varnish doesn't need to detect tablets anymore - it now handles all mobile devices equally.
But how are 'mobile devices.' Defined?
(Let's split this thread off)
On Friday, June 20, 2014, Max Semenik maxsem.wiki@gmail.com wrote:
On Fri, Jun 20, 2014 at 11:55 AM, Oliver Keyes <okeyes@wikimedia.org javascript:_e(%7B%7D,'cvml','okeyes@wikimedia.org');> wrote:
We should probably have a different conversation with ops about trying to rebalance the varnish tablet identification (I've seen the request logs, and that won't catch all of the tablets).
Luckily, Varnish doesn't need to detect tablets anymore - it now handles all mobile devices equally.
On Fri, Jun 20, 2014 at 12:48 PM, Oliver Keyes okeyes@wikimedia.org wrote:
But how are 'mobile devices.' Defined?
(Let's split this thread off)
http://git.wikimedia.org/blob/operations%2Fpuppet.git/7cd814686241a19ba5b4fa...
lines 24, 25 and 27.
I agree these early numbers are very promising. Thanks for sharing! :)
Erik
