Hi all,
here is the usual look https://www.mediawiki.org/wiki/Reading/Readership_metrics_reports at our most important readership metrics. This time, highlights include the impact of the iOS app’s big release of a revamped version, a Google bug that brought the Android app a windfall of about half a million new users, and a first examination of the new unique devices dataset.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events, and facilitate thinking about core metrics in general. As always, feedback and discussion are welcome. Week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org. This edition of the report covers a timespan of eleven weeks.
Some other recent items of interest, in case they didn’t already catch your attention:
-
At the Foundation’s February metrics meeting, I gave an update on longer-term traffic trends since 2013. (TL;DR: Overall pageviews have been flat to slightly declining, mobile has been steadily rising but recently slowed down, desktop is declining.) See the slide deck here or the chart below (updated with March data):
[image: Wikimedia monthly pageviews (worldwide, mobile vs desktop), 2013-2016.png] https://commons.wikimedia.org/wiki/File:Wikimedia_monthly_pageviews_(worldwide,_mobile_vs_desktop),_2013-2016.png
-
The WMF Reading team published its quarterly review presentation https://commons.wikimedia.org/wiki/File:Reading_Quarterly_Review_Q3_2015-16.pdf for Q3 2015-16 (January-March), with lots of traffic and usage data. -
I recently gave a Tech Talk about “new readership data” https://commons.wikimedia.org/wiki/File:New_readership_data_(Wikimedia_Foundation_Tech_Talk).pdf - i.e. some data that, separately from the core metrics discussed here, gives insights on how Wikipedia is read.
Now to the usual data. (All numbers below are averages for February 8-April 24, 2016 unless otherwise noted.)
Our family of core metrics who are populating this report mourns a deplorable casualty (iOS daily active users are no longer available due to the app’s switch to opt-in data collection, T130432 https://phabricator.wikimedia.org/T130432), rejoices about the speedy convalescence of an accident victim thanks to expert therapy administered by the Analytics and iOS teams (iOS pageviews, T131824 https://phabricator.wikimedia.org/T131824), sends well-wishes for recovery to two other patients (Android app DAUs and pageviews, T132965 https://phabricator.wikimedia.org/T132965), and lastly welcomes a new member: unique devices.
Pageviews
Total: 538 million/day (+1.5% from the previous report)
(caveat: likely undercounting by 1-2 million/day since April due to a bug https://phabricator.wikimedia.org/T132965 related to the Android app’s gradual switch to RESTBase)
Context (April 2015-April 2016):
[image: Wikimedia daily pageviews, all vs. mobile (April 2015-2016-04-24).png]
Overall, pageviews were up compared to the previous report (which had still covered the slump around Christmas). There was a drop in early/mid March though, which probably can’t be fully explained by a change in the pageview definition https://meta.wikimedia.org/wiki/Research:Page_view#Change_log on March 9 that improved the exclusion of bot pageviews, nor does it look seasonal compared to March 2014 and March 2015.
See also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews.
Desktop: 55.2% (previous report: 54.3%)
Mobile web: 43.6% (previous report: 44.4%)
Apps: 1.2% (previous report: 1.3%)
(caveat: app percentage likely too low due to the aforementioned Android-related bug)
Context (April 2015-April 2016):
[image: Wikimedia daily pageviews, mobile percentage (April 2015..2016-04-24).png]
Mobile pageviews now regularly reach parity on weekends.
Global North ratio: 76.5% of total pageviews (previous report: 78.3%)
Context (April 2015-April 2016):
[image: Percentage of Wikimedia pageviews from the Global North (April 2015..2016-04-24).png] https://commons.wikimedia.org/wiki/File:Percentage_of_Wikimedia_pageviews_from_the_Global_North_(April_2015-).png
In the previous report, we could already witness the ratio of Global North pageviews falling (or conversely, the Global South ratio rising) from a peak at the beginning of January. Afterwards, it saw a somewhat conspicuous drop in mid-February, went back up a bit but is now back to the levels from a year ago, before the HTTPS-only rollout in June.
Unique devices
Recently, the Analytics team made a new metric available: Daily and monthly unique devices (see their announcement blog post https://blog.wikimedia.org/2016/03/30/unique-devices-dataset/ for background and details). These estimated numbers are provided for all Wikimedia projects (separately for the desktop and mobile web version), but because of the instrumentation method, there is no global metric for all projects. For now, we’ll pick the daily numbers of the English Wikipedia for inclusion in this report. As for other parts of this reports, the selection and presentation are always being optimized; feedback is welcome.
Daily unique devices estimate for English Wikipedia:
-
Mobile web: 33.2 million -
Desktop: 25.7 million -
Total: 58.9 million -
Average mobile web ratio: 56.5%
Like on many of our projects, mobile(-site using) devices outnumber desktop devices, but there are fewer pageviews per device on mobile. (For comparison, the English Wikipedia’s pageviews were at 55.9% desktop, 42.9% mobile web and 1.2% apps; i.e. slightly less mobile than our sites overall.)
Context (January-April 2016):
[image: Daily unique devices for English Wikipedia (estimate), mobile web vs. desktop, 2016-01-01..2016-04-24.png]
The early/mid March drop observed above for pageviews is visible here, too.
New app installations
Android: 31.6k/day (-17.8% from the previous report)
Daily installs per device, from Google Play
Context (last three months, log scale):
Around March 11/12, Google accidentally redirected Wikipedia search results directly to the app’s listing in Google Play, instead of offering the user to read the article on the mobile web version of Wikipedia. (This was a general bug affecting other apps/websites too.) While resulting spike of ca. 600k additional installs was partially negated by rising uninstalls, a month later the app’s install base still remained around half a million - ca. 3% - larger than before the incident.
Overall though, due to a general decrease, downloads per day are down from the previous report https://commons.wikimedia.org/w/index.php?title=File%3AReadership_metrics_for_the_two_months_until_February_7%2C_2016.pdf&page=4, which had included the “Best Apps of 2015” promotion that brought ca. 350k additional installs. (See also the summary https://commons.wikimedia.org/w/index.php?title=File%3AReading_Quarterly_Review_Q3_2015-16.pdf&page=57 in our quarterly review and the discussion of retention rates below)
iOS: 5.06k/day (+2.5% from the previous report)
Download numbers from App Annie
Context (last three months):
In March, the launch of the revamped 5.0 version of the app, with accompanying media attention, brought roughly 50k additional installs. There also was a smaller spike in mid-February whose reason we don’t know. App user retention
Android: 15.4% (previous report: 16.6%)
(Ratio of app installs opened again 7 days after installation, among all installed one week before a date that falls within this report. 1:100 sample.)
Context (last three months):
[image: Day-7 retention of Wikipedia Android app users (install dates 2015-10-01..2016-04-17).png] https://commons.wikimedia.org/wiki/File:Day-7_retention_of_Wikipedia_Android_app_users_(install_dates_2015-10-01..2016-04-17).png
As remarked in earlier reports, this data is a bit too noisy for drawing conclusions about whether retention changed significantly between different releases. But we can discern an effect of the aforementioned Google bug that inadvertently sent many surfers from the search results page directly to the app’s store page. While retention dropped a lot on the first day of the incident, it actually rose far above average later before the spike ended. Overall, the retention of users who came in during those three days (March 11-13) was 14.1%, not far below the average of 15.4%.
iOS: N/A
A new instrumentation has been built to measure retention in the same way as for the Android app, but unfortunately it turned out to be a bit buggy, yielding rosy retention rates above 100% ;) We’ll resume reporting after this is sorted out (T126693 https://phabricator.wikimedia.org/T126693).
Since improving retention was a central goal of the new 5.0 version (and the new instrumentation would not have been available for comparison in older versions of the app, anyway), the iOS team resorted to Apple’s metric once again in order to assess the effect of the rollout. Disregarding the imperfections that had made us stop relying on it earlier, it indicated that day-7 retention had risen from about 10% to 15% in the new version (see quarterly review https://commons.wikimedia.org/w/index.php?title=File%3AReading_Quarterly_Review_Q3_2015-16.pdf&page=10 ). Unique app users
Android: N/A
Unfortunately, there is currently no valid data for this metric because of issues having to do with the app’s recent switch to the RESTBase-based Content Service, see T132965 https://phabricator.wikimedia.org/T132965.
(I might still have included the usual 3-month chart showing the development up to that point, but as I’m writing this, Google Sheets doesn’t let you https://productforums.google.com/forum/#!msg/docs/yB_xNk6mn8U/QjRDR4WtNAAJ create timeline charts that include the month of February. True story; this report covers not one but two severe bugs in Google products ;) The linked bug contains a chart with a different timespan though - keep in mind that the decline visible from the beginning April is artificial.)
iOS: N/A
With the new 5.0 version that launched in March, only a small minority of devices continue to send the data that this metric is based on (T130432 https://phabricator.wikimedia.org/T130432), so it is no longer valid as a measure of the apps overall usage volume.
Data sources
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year>0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/77 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2016-02-08" AND "2016-04-24" GROUP BY access_method;
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/ (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE timestamp LIKE '2016%' AND userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android'; hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';