Hi all,
here is the usual look
<https://www.mediawiki.org/wiki/Reading/Readership_metrics_reports> at our
most important readership metrics. This time, highlights include the impact
of the iOS app’s big release of a revamped version, a Google bug that
brought the Android app a windfall of about half a million new users, and a
first examination of the new unique devices dataset.
As laid out earlier
<https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html>,
the main purpose is to raise awareness about how these are developing, call
out the impact of any unusual events, and facilitate thinking about core
metrics in general. As always, feedback and discussion are welcome.
Week-over-week and month-over-month changes are now being recorded on the
Product page <https://www.mediawiki.org/wiki/Wikimedia_Product#Reading> at
MediaWiki.org. This edition of the report covers a timespan of eleven weeks.
Some other recent items of interest, in case they didn’t already catch your
attention:
-
At the Foundation’s February metrics meeting, I gave an update on
longer-term traffic trends since 2013. (TL;DR: Overall pageviews have been
flat to slightly declining, mobile has been steadily rising but recently
slowed down, desktop is declining.) See the slide deck here or the chart
below (updated with March data):
[image: Wikimedia monthly pageviews (worldwide, mobile vs desktop),
2013-2016.png]
<https://commons.wikimedia.org/wiki/File:Wikimedia_monthly_pageviews_(worldwide,_mobile_vs_desktop),_2013-2016.png>
-
The WMF Reading team published its quarterly review presentation
<https://commons.wikimedia.org/wiki/File:Reading_Quarterly_Review_Q3_2015-16.pdf>
for Q3 2015-16 (January-March), with lots of traffic and usage data.
-
I recently gave a Tech Talk about “new readership data”
<https://commons.wikimedia.org/wiki/File:New_readership_data_(Wikimedia_Foundation_Tech_Talk).pdf>
- i.e. some data that, separately from the core metrics discussed here,
gives insights on how Wikipedia is read.
Now to the usual data. (All numbers below are averages for February 8-April
24, 2016 unless otherwise noted.)
Our family of core metrics who are populating this report mourns a
deplorable casualty (iOS daily active users are no longer available due to
the app’s switch to opt-in data collection, T130432
<https://phabricator.wikimedia.org/T130432>), rejoices about the speedy
convalescence of an accident victim thanks to expert therapy administered
by the Analytics and iOS teams (iOS pageviews, T131824
<https://phabricator.wikimedia.org/T131824>), sends well-wishes for
recovery to two other patients (Android app DAUs and pageviews, T132965
<https://phabricator.wikimedia.org/T132965>), and lastly welcomes a new
member: unique devices.
Pageviews
Total: 538 million/day (+1.5% from the previous report)
(caveat: likely undercounting by 1-2 million/day since April due to a bug
<https://phabricator.wikimedia.org/T132965> related to the Android app’s
gradual switch to RESTBase)
Context (April 2015-April 2016):
[image: Wikimedia daily pageviews, all vs. mobile (April
2015-2016-04-24).png]
Overall, pageviews were up compared to the previous report (which had still
covered the slump around Christmas). There was a drop in early/mid March
though, which probably can’t be fully explained by a change in the pageview
definition <https://meta.wikimedia.org/wiki/Research:Page_view#Change_log>
on March 9 that improved the exclusion of bot pageviews, nor does it look
seasonal compared to March 2014 and March 2015.
See also the Vital Signs dashboard
<https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews>.
Desktop: 55.2% (previous report: 54.3%)
Mobile web: 43.6% (previous report: 44.4%)
Apps: 1.2% (previous report: 1.3%)
(caveat: app percentage likely too low due to the aforementioned
Android-related bug)
Context (April 2015-April 2016):
[image: Wikimedia daily pageviews, mobile percentage (April
2015..2016-04-24).png]
Mobile pageviews now regularly reach parity on weekends.
Global North ratio: 76.5% of total pageviews (previous report: 78.3%)
Context (April 2015-April 2016):
[image: Percentage of Wikimedia pageviews from the Global North (April
2015..2016-04-24).png]
<https://commons.wikimedia.org/wiki/File:Percentage_of_Wikimedia_pageviews_from_the_Global_North_(April_2015-).png>
In the previous report, we could already witness the ratio of Global North
pageviews falling (or conversely, the Global South ratio rising) from a
peak at the beginning of January. Afterwards, it saw a somewhat conspicuous
drop in mid-February, went back up a bit but is now back to the levels from
a year ago, before the HTTPS-only rollout in June.
Unique devices
Recently, the Analytics team made a new metric available: Daily and monthly
unique devices (see their announcement blog post
<https://blog.wikimedia.org/2016/03/30/unique-devices-dataset/> for
background and details). These estimated numbers are provided for all
Wikimedia projects (separately for the desktop and mobile web version), but
because of the instrumentation method, there is no global metric for all
projects. For now, we’ll pick the daily numbers of the English Wikipedia
for inclusion in this report. As for other parts of this reports, the
selection and presentation are always being optimized; feedback is welcome.
Daily unique devices estimate for English Wikipedia:
-
Mobile web: 33.2 million
-
Desktop: 25.7 million
-
Total: 58.9 million
-
Average mobile web ratio: 56.5%
Like on many of our projects, mobile(-site using) devices outnumber desktop
devices, but there are fewer pageviews per device on mobile. (For
comparison, the English Wikipedia’s pageviews were at 55.9% desktop, 42.9%
mobile web and 1.2% apps; i.e. slightly less mobile than our sites
overall.)
Context (January-April 2016):
[image: Daily unique devices for English Wikipedia (estimate), mobile web
vs. desktop, 2016-01-01..2016-04-24.png]
The early/mid March drop observed above for pageviews is visible here, too.
New app installations
Android: 31.6k/day (-17.8% from the previous report)
Daily installs per device, from Google Play
Context (last three months, log scale):
<https://commons.wikimedia.org/wiki/File:Wikipedia_Android_app_-_Daily_installs_and_uninstalls_by_device_Jan_25-Apr_24,_2016_(Google_Play_Developer_Console)_log_scale.png>
Around March 11/12, Google accidentally redirected Wikipedia search results
directly to the app’s listing in Google Play, instead of offering the user
to read the article on the mobile web version of Wikipedia. (This was a
general bug affecting other apps/websites too.) While resulting spike of
ca. 600k additional installs was partially negated by rising uninstalls, a
month later the app’s install base still remained around half a million -
ca. 3% - larger than before the incident.
Overall though, due to a general decrease, downloads per day are down from the
previous report
<https://commons.wikimedia.org/w/index.php?title=File%3AReadership_metrics_for_the_two_months_until_February_7%2C_2016.pdf&page=4>,
which had included the “Best Apps of 2015” promotion that brought ca. 350k
additional installs. (See also the summary
<https://commons.wikimedia.org/w/index.php?title=File%3AReading_Quarterly_Review_Q3_2015-16.pdf&page=57>
in our quarterly review and the discussion of retention rates below)
iOS: 5.06k/day (+2.5% from the previous report)
Download numbers from App Annie
Context (last three months):
<https://commons.wikimedia.org/wiki/File:Wikipedia_iOS_app_daily_downloads,_Jan_25,_2016_~_Apr_24,_2016_(App_Annie).png>
In March, the launch of the revamped 5.0 version of the app, with
accompanying media attention, brought roughly 50k additional installs.
There also was a smaller spike in mid-February whose reason we don’t know.
App user retention
Android: 15.4% (previous report: 16.6%)
(Ratio of app installs opened again 7 days after installation, among all
installed one week before a date that falls within this report. 1:100
sample.)
Context (last three months):
[image: Day-7 retention of Wikipedia Android app users (install dates
2015-10-01..2016-04-17).png]
<https://commons.wikimedia.org/wiki/File:Day-7_retention_of_Wikipedia_Android_app_users_(install_dates_2015-10-01..2016-04-17).png>
As remarked in earlier reports, this data is a bit too noisy for drawing
conclusions about whether retention changed significantly between different
releases. But we can discern an effect of the aforementioned Google bug
that inadvertently sent many surfers from the search results page directly
to the app’s store page. While retention dropped a lot on the first day of
the incident, it actually rose far above average later before the spike
ended. Overall, the retention of users who came in during those three days
(March 11-13) was 14.1%, not far below the average of 15.4%.
iOS: N/A
A new instrumentation has been built to measure retention in the same way
as for the Android app, but unfortunately it turned out to be a bit buggy,
yielding rosy retention rates above 100% ;) We’ll resume reporting after
this is sorted out (T126693 <https://phabricator.wikimedia.org/T126693>).
Since improving retention was a central goal of the new 5.0 version (and
the new instrumentation would not have been available for comparison in
older versions of the app, anyway), the iOS team resorted to Apple’s metric
once again in order to assess the effect of the rollout. Disregarding the
imperfections that had made us stop relying on it earlier, it indicated
that day-7 retention had risen from about 10% to 15% in the new version
(see quarterly review
<https://commons.wikimedia.org/w/index.php?title=File%3AReading_Quarterly_Review_Q3_2015-16.pdf&page=10>
).
Unique app users
Android: N/A
Unfortunately, there is currently no valid data for this metric because of
issues having to do with the app’s recent switch to the RESTBase-based
Content Service, see T132965 <https://phabricator.wikimedia.org/T132965>.
(I might still have included the usual 3-month chart showing the
development up to that point, but as I’m writing this, Google Sheets doesn’t
let you
<https://productforums.google.com/forum/#!msg/docs/yB_xNk6mn8U/QjRDR4WtNAAJ>
create timeline charts that include the month of February. True story; this
report covers not one but two severe bugs in Google products ;) The linked
bug contains a chart with a different timespan though - keep in mind that
the decline visible from the beginning April is artificial.)
iOS: N/A
With the new 5.0 version that launched in March, only a small minority of
devices continue to send the data that this metric is based on (T130432
<https://phabricator.wikimedia.org/T130432>), so it is no longer valid as a
measure of the apps overall usage volume.
Data sources
For reference, the queries and source links used are listed below (access
is needed for each). Unless otherwise noted, all content of this report is
© Wikimedia Foundation and released under the CC BY-SA 3.0
<https://creativecommons.org/licenses/by-sa/3.0/> license. Most of the
above charts are available on Commons, too
<https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4>
.
hive (wmf)> SELECT year, month, day,
CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0"))
as date,
sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews,
SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year>0 AND
agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day
LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/77 FROM
wmf.projectview_hourly WHERE agent_type = 'user' AND
CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0"))
BETWEEN "2016-02-08"
AND "2016-04-24" GROUP BY access_method;
hive (wmf)> SELECT year, month, day,
CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")),
SUM(view_count) AS
all, SUM(IF (FIND_IN_SET(country_code,
'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM
wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY
year, month, day ORDER BY
year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_0281…
(“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/ (select
“Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1,
0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS
day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE timestamp LIKE
'2016%' AND userAgent LIKE '%-r-%' AND userAgent NOT LIKE
'%Googlebot%'
GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active
from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT
CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0"))
as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily
WHERE platform = 'Android';
hive (wmf)> SELECT
CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0"))
as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE
platform = 'iOS';
--
Tilman Bayer
Senior Analyst
Wikimedia Foundation
IRC (Freenode): HaeB