Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
----
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
Hey Tilman,
I would really like to thank you for producing these reports. They're wonderfully informative, great for transparency, and something more teams should be doing. I'm personally gonna be stealing a leaf from your book and doing something similar :).
On 18 February 2016 at 17:37, Tilman Bayer tbayer@wikimedia.org wrote:
Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Tilman, is it ok to cite "December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%)" on the Internetz?
On Thu, Feb 18, 2016 at 2:46 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Hey Tilman,
I would really like to thank you for producing these reports. They're wonderfully informative, great for transparency, and something more teams should be doing. I'm personally gonna be stealing a leaf from your book and doing something similar :).
On 18 February 2016 at 17:37, Tilman Bayer tbayer@wikimedia.org wrote:
Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Count Logula Wikimedia Foundation
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Feel free to. Are you planning an annual event of sorts? ;)
On Thu, Feb 18, 2016 at 4:00 PM, Volker Eckl volker@wikimedia.org wrote:
Tilman, is it ok to cite "December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%)" on the Internetz?
On Thu, Feb 18, 2016 at 2:46 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Hey Tilman,
I would really like to thank you for producing these reports. They're wonderfully informative, great for transparency, and something more teams should be doing. I'm personally gonna be stealing a leaf from your book and doing something similar :).
On 18 February 2016 at 17:37, Tilman Bayer tbayer@wikimedia.org wrote:
Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Count Logula Wikimedia Foundation
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Le 18/02/2016 23:37, Tilman Bayer a écrit :
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
Hello,
I would suspect it is related to Wikipedia turning 15 years old. The birthday has been well covered by news at least in France. A few acquaintance learned about the Wikipedia mobile apps by reading the news and started using it.
Quoting someone I met:
"I never heard you have a Wikipedia app. It is nice and usefull, much better than the web view".
Hearsay, I have no fact ...
Some of it, I suspect, is inevitable. I studied this a while back and a lot of prominent countries were very close to the 50% switchover, including (interestingly) Italy.
On 19 February 2016 at 06:51, Antoine Musso hashar+wmf@free.fr wrote:
Le 18/02/2016 23:37, Tilman Bayer a écrit :
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
Hello,
I would suspect it is related to Wikipedia turning 15 years old. The birthday has been well covered by news at least in France. A few acquaintance learned about the Wikipedia mobile apps by reading the news and started using it.
Quoting someone I met:
"I never heard you have a Wikipedia app. It is nice and usefull, much better than the web view".
Hearsay, I have no fact ...
-- Antoine "hashar" Musso
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Oliver Keyes, 19/02/2016 13:51:
a lot of prominent countries were very close to the 50% switchover, including (interestingly) Italy.
I compiled some links at https://meta.wikimedia.org/wiki/Research:The_sudden_decline_of_Italian_Wikip... on the likely social reasons.
Pew research also had some numbers on the matter for USA, recently.
Nemo
Dear Tilman,
Can you provide a link for this summary below?
Thank you,
*~Orsolya*
2016-02-18 23:37 GMT+01:00 Tilman Bayer tbayer@wikimedia.org:
Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Hi Orsolya,
a PDF copy is at https://commons.wikimedia.org/wiki/File:Readership_metrics_for_the_two_month... (previous reports and various charts are also available at https://commons.wikimedia.org/wiki/Category:Wikimedia_readership_metrics_rep... )
And of course these mailing lists have public archives too, although they don't play very well with HTML attachments ( https://lists.wikimedia.org/pipermail/mobile-l/2016-February/010096.html ).
On Fri, Feb 19, 2016 at 11:01 AM, Orsolya Gyenes < gyenes.orsolya@wiki.media.hu> wrote:
Dear Tilman,
Can you provide a link for this summary below?
Thank you,
*~Orsolya*
2016-02-18 23:37 GMT+01:00 Tilman Bayer tbayer@wikimedia.org:
Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Thanx a lot, PDF works fine!
*~Orsolya*
2016-02-19 20:05 GMT+01:00 Tilman Bayer tbayer@wikimedia.org:
Hi Orsolya,
a PDF copy is at https://commons.wikimedia.org/wiki/File:Readership_metrics_for_the_two_month... (previous reports and various charts are also available at https://commons.wikimedia.org/wiki/Category:Wikimedia_readership_metrics_rep... )
And of course these mailing lists have public archives too, although they don't play very well with HTML attachments ( https://lists.wikimedia.org/pipermail/mobile-l/2016-February/010096.html ).
On Fri, Feb 19, 2016 at 11:01 AM, Orsolya Gyenes < gyenes.orsolya@wiki.media.hu> wrote:
Dear Tilman,
Can you provide a link for this summary below?
Thank you,
*~Orsolya*
2016-02-18 23:37 GMT+01:00 Tilman Bayer tbayer@wikimedia.org:
Hi all,
this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November.
As laid out earlier https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome.
After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page https://www.mediawiki.org/wiki/Wikimedia_Product#Reading at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks.
See also the slides from the Reading team’s quarter review meeting https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarterly_reviews/Reading_and_Community_Tech,_January_2016 on January 20 for an in-depth look at various metrics.
Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.)
Pageviews
Total: 529 million/day
Context (April 2015-February 2016):
(see also the Vital Signs dashboard https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews)
Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop http://discovery.wmflabs.org/external/#traffic_by_engine in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now.
Desktop: 54.3%
Mobile web: 44.4%
Apps: 1.3%
Context (April 2015-February 2016):
December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents.
Global North ratio: 78.3% of total pageviews
Context (April 2015-February 2016):
New app installations
Android: 38.4k/day
Daily installs per device, from Google Play
Context (January 2015-February 2016):
As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs.
The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-through-app-indexing.html in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015.
iOS: 4.94k/day
Download numbers from App Annie
Context (last three months):
Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us.
App user retention
Android: 16.8%
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample)
Context (last four months):
Retention for installations since around Christmas appears to be higher; but the data is quite noisy.
iOS: N/A
(Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users)
As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working https://phabricator.wikimedia.org/T126693 to measure retention ourselves via EventLogging, like we do on Android. Unique app users
Android: 1.206 million / day
Context (last four months):
A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially).
iOS: 303 k / day
Context (last four months):
Similar to Android, a clearly visible and lasting increase around Christmas.
For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0/ license. Most of the above charts are available on Commons, too https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20150923010000&user=Tbayer+%28WMF%29&ilshowall=1&limit=4 .
hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method;
hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE
agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07";
hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US')
0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly
WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000;
https://console.developers.google.com/storage/browser/pubsite_prod_rev_02812... (“overview”)
https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdow... (select “Total”)
SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE;
(with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course)
https://analytics.itunes.apple.com/#/retention?app=324715238
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android';
hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS';
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l