Hi Jon --

Great analysis, thanks for doing this and making it available publicly. As far as I know, this is the first time we've looked at the data this way.

Some questions/thoughts -- I think it is reassuring that an explanation for this page view drop we see is that we are still reaching the same number of people but they just browse less. But for me, it's still an issue that they are seeing less of our content.
  1. There's a strange artifact in the desktop page views where they drop by about 75mm at the end of 2013 but don't recover. Do we understand what happened here.
  2. Does this mean that mobile users are less likely to fall into the rabbit hole? I'd be interested in getting some more targeted research around how mobile and desktop use differ. Maybe the apps can help.
  3. It feels like we should keep things simple and see what we can do about engagement in the mobile web. We have a few ideas in the pipeline for this quarter; I wonder if there are more things we can do.
More analysis and questions are welcome.

-Toby







On Tue, Oct 13, 2015 at 10:14 PM, Jon Katz <jkatz@wikimedia.org> wrote:
Hi Folks,

TLDR: Directional data suggests that the project-wide drop we see in pageviews is, in part, caused by shorter sessions on mobile web compared to desktop (and a migration from desktop to mobile web)

Context:
Danny and I took some time last week to try and understand the dramatic drop in pageviews that we saw globally and in the global south just over the last quarter.  The numbers we quoted in the Q1 quarterly review last week were (pageviews across all projects, platforms and geographies):
  • -12.4% Quarter over quarter
  • -7% Year over year*  
*The YoY data is sampled and there may be anomalies that force us to consider an inaccuracy of up to +-5% difference possible (i.e. a YoY change from -2% to -12%) (source: Tilman, Hive).

Here is the answer to one question we had: Is this drop due to fewer people or fewer pages per user? I will preface this by saying that I erred on the side of getting this out sooner than in making it incredibly replicable, share-able.  If you are interested in the specific queries or access to the raw data, I will prep and send it out.  Otherwise, curious to hear your questions/concerns/suggestions.


Details:

Is this drop due to fewer people or fewer pages per user?  

The answer here, is interesting and the impact is more significant than I would have expected.  On Desktop the pageviews per visit (as internal referers/external+ unknown) is relatively constant.  On Mobile web, however, the pageviews per visit is much lower and appears to be dropping.  The following graphs explain:

Daily desktop pageviews, by referer 4/13--present (all wikipedias, all geos).  There is strong correlation between pageviews that come from the outside v. the inside:
Inline image 3



Inline image 4


Daily Mobile web pageviews, by referer 4/13--present (all wikipedias, all geos). 
Inline image 2


Compared the 60% we have on desktop, you can see that the ratio is 40% (33% smaller) on mobile and that this gap has widened (though not in the last 2 years):
Inline image 5


I don't know if we can explain all of our traffic decreases to the drop in session length, but it is certainly a big factor.  Basically 60% of our pageviews (internal) shrink by 33% on mobile.  So all else being equal, if we transfer all our traffic to mobile we lose 33% of our pageviews.  Right now we're at 50%.  This assumes that there is no change in numbers of sessions...on which we have no data right now.

Next Steps:
Unless I hear otherwise, I think the next steps are to start thinking through what the implications are.  
  • Do we try to identify reasons users might be skipping out earlier on mobile and fixing those?  
  • Do we try to make it easier for people to find content on mobile?
  • Maybe sessions across the internet are just shorter on mobile and we should focus our efforts on helping people find us?
Regardless, I find this a bit comforting, because having the same number of users who spend less time would be much better than reaching fewer people: controlling the experience once they found us is relatively easier than altering the channels by which people find us in the first place.    

Again questions/concerns/suggestions encouraged.

-J




mobile, which is a 15% drop in pageviews (again, assuming a 1:1 traffic switch).

_______________________________________________
reading-wmf mailing list
reading-wmf@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/reading-wmf