Re: [Wikitech-l] [Analytics] Readership metrics for the timespan until December 4, 2016

Yes, thanks so much for sharing this! It's fascinating and encouraging to see the November impressions bump. On Wed, Dec 14, 2016 at 11:12 AM, Toby Negrin &lt;tnegrin(a)wikimedia.org&gt; wrote: Thank you so much for this Zareen! It's really great to see this report -- so much interesting data to think about! -Toby On Mon, Dec 12, 2016 at 5:45 PM, Zareen Farooqui &lt;zareenf(a)gmail.com&gt; wrote: Hi all, This resumes the usual look <https://www.mediawiki.org/wiki/Reading/Readership_metrics_reports> at our most important readership metrics. This time we can report that daily pageviews are up 4.8% (since the last report), with an interesting recent peak which meant that November’s pageviews surpassed those of November 2015 (coincidentally also) by 4.8%, after October had already seen a 2.1% year-over-year increase. The iOS Wikipedia app saw increased downloads, while the Android app’s install base has stopped its previous downward trend. As laid out earlier <https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html>, the main purpose is to raise awareness about how these metrics are developing, call out the impact of any unusual events, and facilitate thinking about core metrics in general. As always; feedback and discussion welcome. Week-over-week and month-over-month changes are now being recorded on the Product page <https://www.mediawiki.org/wiki/Wikimedia_Product#Reading> at MediaWiki.org. This edition of the report covers a timespan of eighteen weeks. Some other recent items of interest, in case they didn’t already catch your attention: - The WMF Reading team published its quarterly review presentation <https://commons.wikimedia.org/wiki/File:Reading_and_Comm_Tech_Quarterly_Review_Q1_2016-17.pdf> for Q1 2016-17 (July-September), which includes lots of traffic and usage data. - At the Foundation’s August metrics meeting <https://commons.wikimedia.org/w/index.php?title=File%3AAugust_2016_Monthly_Metrics_Meeting.pdf&page=14>, the Reading team gave an update on longer-term traffic trends since 2013. (TL;DR: Overall pageviews have been flat to slightly declining, mobile has been steadily rising but recently slowed down, desktop has declining during these three years. However, total pageviews have been slightly increasing year-over-year in the last few months.) See the chart below, updated with data until November: [image: Wikimedia monthly pageviews (desktop+mobile), 2013-2016 (version December 2016).png] In particular, as mentioned, the number of total pageview saw year-over-year increases of +2.1% for October and +4.8% in November, in contrast to e.g. the -10.5% we had for May 2015-May 2016. Now to the usual data. (All numbers below are averages for August 1-December 4, 2016 unless otherwise noted.) Pageviews Total: 529 million/day (+4.76% from the previous report timeframe, with corrected numbers for anomalously high traffic on some main pages <https://phabricator.wikimedia.org/T141506>) Context (April 2015-December 2016): See also the Vital Signs dashboard <https://analytics.wikimedia.org/dashboards/vital-signs/#projects=all/metrics=Pageviews> (Small caveats: iOS app’s pageviews were undercounted by about 1.6 million/day from mid September to early November due to a bug <https://phabricator.wikimedia.org/T148663>.) Overall pageviews increased steadily during the timespan of this report aside from the week ending October 30th (right before Halloween) and the end of November. There appears to be a peak in pageviews in November. To facilitate our understanding of which traffic movements are seasonal and which may indicate lasting changes, here is a chart overlaying the total pageview numbers back to May 2013 (the earliest time for which we have data according to the current pageview definition): The blue line indicates a non-seasonal rise peaking around November 12. We checked whether this peak came from a particular country and were able to exclude that possibility. Wikimedia Daily Pageviews from US Pageviews in US do not show any drastic changes (even around the time of the US elections). Wikimedia Daily Pageviews from Mexico In Mexico, there seems to been a huge drop starting October 27th (perhaps some sort of local outage). Wikimedia Daily Pageviews from Ecuador Ecuador shows a huge spike on October 30th, followed by a drop for several days. Desktop: 54.1% ​(previous report: ​54.1%) Mobile web: 44.8% ​(previous report: 44.6%) Apps: 1.1% ​(previous report: ​1.3%) (missing some iOS pageviews, cf. above) Context (December 2015 - December 2016): Overall mobile percentage is similar to the last report, but we did see a small increase (besides one week in August) until late September. As a reminder, mobile already has a solid majority in terms of unique devices, cf. below. Global North ratio: 75.3% of total pageviews (previous report: 75.5%) Context (January - December 2016): There is a slight increase in Global North percentage in late October and November, but overall this number is relatively steady. NB: We are currently rethinking this metric and might replace it with a different country selection constructed as part of the work on the New Readers project. Unique devices See the announcement blog post <https://blog.wikimedia.org/2016/03/30/unique-devices-dataset/> from March for background and details on this recently introduced metric. These estimated numbers are provided for all Wikimedia language projects (separately for the desktop and mobile web version). Because of the instrumentation method, there is no global metric for all projects and all languages, but following some recent discussions it is now planned <https://phabricator.wikimedia.org/T138027> to extend it to a cross­-language global metric per project at least. For now, we track the daily numbers of the English Wikipedia in this report. Daily unique devices estimate for English Wikipedia: - Mobile web: 33.9 million (previous report: 32.2 million) - Desktop: 24.1 million (previous report: 22.3 million) - Total: 57.9 million (previous report: 54.5 million) - Average mobile web ratio: 58.7% (previous report: 59.3%) Context (January- December 2016): This metric hasn’t existed long enough yet to get a good sense of what yearly seasonalities may exist, but since the last report time frame daily unique desktop devices show an increasing upward trend until early November, where there is a peak and then an apparent drop (which parallels the peak in pageviews). To understand the peak in daily unique devices, we plotted the ratio of daily pageviews to daily unique devices. There is a small bump in the mobile ratio in early November, but it is too small to fully explain the rise in pageviews around this time. New app installations Android: 19.9k/day (+1.21% from the previous report) Daily installs per device, from Google Play Context (last seven months): Download numbers increased in September and have sustained previous levels so far, possibly helped by the app’s inclusion on the Education category page <https://play.google.com/store/apps/category/EDUCATION> in Google’s Play Store. There was an additional bump in mid September (reason unknown). The number of daily uninstalls decreased slightly. During the timespan of the previous report, installs had markedly decreased (except for a so far unexplained bump during two weeks in July), which combined with the high uninstall numbers had caused the app’s install base to shrink from 15.5 million on March 31 to 15.1 million at the beginning of August. This trend has now been stopped, although the number was still at 15.1 million on November. (Google has since discontinued <https://support.google.com/googleplay/android-developer/answer/7003402#active_devices> this “current installs” metric in favor of a new “installs on active devices" metric.) iOS: 6.35k/day (+99.9% from the previous report) Download numbers from App Annie Daily downloads (with the largest countries broken out): [image: Wikipedia iOS app daily downloads by country, Aug 1 ~ Dec 7, 2016 (App Annie).png] The iOS Wikipedia app was promoted on iTunes in September, with huge impact (twice as many downloads than resulted from a comparable promotion in September 2015). Total daily downloads remained on a higher level afterwards (4.8k/day during the last week of this report, Nov 28-Dec 4). Around October 24, downloads from the US spiked, possibly helped by some organic media coverage such as this <http://thetechbeard.com/best-free-ios-10-widgets/>. On November 11, we registered an extra 9000 downloads from Sweden, for unknown reasons (no app store promotions or media coverage that we are aware of). App user retention Android: 16.4% (previous report: 16.7%) (Ratio of app installs opened again 7 days after installation, among all installed one week before a date that falls within this report. 1:100 sample) Context (last seven months): As remarked in earlier reports, this data is a bit too noisy for drawing conclusions about whether retention changed significantly between different releases. Retention appears to have dropped somewhat in September after having gradually grown earlier during the timespan of the last report and after. iOS: N/A <https://phabricator.wikimedia.org/T126693> (We recently began measuring this on the iOS app - using the same method as on Android, restricted to users who opt-in to data collection - but there is not enough data yet.) Unique app users Android: 1.146 million / day (+0.0% from the previous report) Context (last six months): In September/October, there seems to be a very small growth in daily average users. iOS: N/A <https://phabricator.wikimedia.org/T130432> Zareen Farooqui, Data Analyst Intern, Wikimedia Foundation Tilman Bayer, Senior Analyst, Wikimedia Foundation Data sources For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 <https://creativecommons.org/licenses/by-sa/3.0/> license. Most of the above charts are available on Commons, too. <https://commons.wikimedia.org/wiki/Category:Wikimedia_readership_metrics_reports> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year>0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT LEFT(timestamp, 10) AS date, sum(IF(access_method <> 'desktop', pageviews, null)) AS mobileviews, SUM(pageviews) AS allviews FROM staging.pageviews05 WHERE is_spider = FALSE AND is_automata = FALSE GROUP BY date; Pivot URLs: United States <https://pivot.wikimedia.org/#pageviews-daily/line-chart/2/EQUQLgxg9AqgKgYWAGgN7APYAdgC5gQAWAhgJYB2KwApgB5YBO1Azs6RpbutnsEwGZVyxALbVeAfQlhSY4AF9kwYhBkdmeANroVazsApU6jFmw55uOfABtSYag2LWqANycBXcV2DMwxBmC8AEwADACMAGwAtCEAnDFhcCEhuMmpIQB0ySEAWkbkACbB4dFhQTEAHEkpaclZyXmKwGAAnlhewHAAkgCyIBIASgCCAHIA4iAKijqq7PrEhUb0TKxzFphWBCSGSsYrZpyWvAJCoh0QGO7kYAwtCkq6a96P5vgYLg7WxDi7y6ZPRxsdgcTlcHi86GY1DAcDaHQAynABl1xkZrNQxNcNLhNMAYORgQUAATwvz2DQAXSarXavHhIDgU3k02Us1eykWvxMq1egK2ZEoXP2AI2x2ogiUwjk+GIAHNqNdpHD7qy9OsXvp3p9vktuQd1jwgfZHM4lG5rJ51lCYcr8IjkajdujMWBsbj3FCGMAqUoaQiGUz5D7tDQ/jzDqL8CdJWdJNJZOImgAjdwQADW0KGbIjGt4MjEACFUxnAkoCu5HGr8AAFMIAERVzAwASzVZmVZ8zdLob1IsNfHFp2lwCk+cTZdITGzvGIzAgCoKFFlUwpyBunjNpGoAHcJBcrqXcS4t7v99dvchNKvyO5rNYgA==> Mexico <https://pivot.wikimedia.org/#pageviews-daily/line-chart/2/EQUQLgxg9AqgKgYWAGgN7APYAdgC5gQAWAhgJYB2KwApgB5YBO1Azs6RpbutnsEwGZVyxALbVeAfQlhSY4AF9kwYhBkdmeANroVazsApU6jFmw55uOfABtSYag2LWqANycBXcV2DMwxBmC8AEwADACMAGwAtCEAnDFhcCEhuMmpIQB0ySEAWkbkACbB4dFhQTEAHEkpaclZyXmKwGAAnlhewHAAkgCyIBIASgCCAHIA4iAKijqq7PrEhUb0TKxzFphWBCSGSsYrZpyWvAJCoh0QGO7kYAwtCkq6a96P5vgYLg7WxDi7y6ZPRxsdgcTlcHi86GY1DAcDaHQAynABl1xkZrNQxNcNLhNMAenRSBdgABdJqtdq8eEgOBTeTTZSzV7KRa/EyrV6ArZkSis/YAjbHaiCJTCOT4YgAc2o12kcPuDL06xe+nen2+SzZB3WPCB9kcziUbmsnnWUJhcvwiORqN26MxYGxuPcUIYJLJFuAVJpdLppOQ2hof3ZhwF+BOIrOkmksnETQARu4IABraFDRkh5W8GRiABCiZTgSUBXcjkV+AACmEACLy5gYAJpsszMs+euFwOa/k6vhC05i4BSbOxoukJjp3jEZgQaUFCgSqbE5A3TyG0jUADuEguV0LuJca832+uJP9i/I7ms1iAA=> Ecuador <https://pivot.wikimedia.org/#pageviews-daily/line-chart/2/EQUQLgxg9AqgKgYWAGgN7APYAdgC5gQAWAhgJYB2KwApgB5YBO1Azs6RpbutnsEwGZVyxALbVeAfQlhSY4AF9kwYhBkdmeANroVazsApU6jFmw55uOfABtSYag2LWqANycBXcV2DMwxBmC8AEwADACMAGwAtCEAnDFhcCEhuMmpIQB0ySEAWkbkACbB4dFhQTEAHEkpaclZyXmKwGAAnlhewHAAkgCyIBIASgCCAHIA4iAKijqq7PrEhUb0TKxzFphWBCSGSsYrZpyWvAJCoh0QGO7kYAwtCkq6a96P5vgYLg7WxDi7y6ZPRxsdgcTlcHi86GY1DAcDaHQAynABl1xkZrNQxNcNLhNKAIO5iAUMAxgABdJqtdq8eEgOBTeTTZSzV7KRa/EyrV6ArZkSjs/YAjbHaiCJTCOT4YgAc2o12kcPuTL06xe+nen2+Sw5B3WPCB9kcziUbmsnnWUJhCvwiORqN26MxYGxuPcUJJ5KUlIRtPp8g92hof05hyF+BOYrOkmksnETQARu4IABraFDZkh1W8GRiABCiZTgSUBXcjmV+AACmEACKK5jEsBpsszMs+etagVc0N8EWnCXAKTZ2NF0hMdO8YjMCCygoUKVTUnIG6eY2kagAdwkFyuhdxLlXG631zJyE0C/I7ms1iAA=> SELECT access_method, SUM(view_count)/(7*18) FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2016-08-01" AND "2016-12-04" GROUP BY access_method; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US') WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, SUM(IF(uri_host LIKE 'en.m.wikipedia%', uniques_estimate, 0)) AS enwiki_mobile_web FROM wmf.last_access_uniques_daily WHERE year=2016 GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, SUM(IF(uri_host LIKE 'en.wikipedia%', uniques_estimate, 0)) AS enwiki_desktop FROM wmf.last_access_uniques_daily WHERE year=2016 GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) AS date, SUM(IF(access_method = 'mobile web', view_count, null)) AS mobilewebviews, SUM(IF(access_method = 'desktop', view_count, null)) AS desktopviews FROM wmf.projectview_hourly WHERE year=2016 AND agent_type = 'user' AND project = 'en.wikipedia' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE timestamp LIKE '201510%' AND userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE; (with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course) https://analytics.itunes.apple.com/#/retention?app=324715238 SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android'; https://console.developers.google.com/storage/browser/pubsite_prod_rev_0281… (“overview”) https://www.appannie.com/dashboard/252257/item/324715238/downloads/ (select “Total”) Zareen Farooqui _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics -- Caitlin Cogdill Senior Fundraising Email Manager Wikimedia Foundation Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us make it a reality! *https://donate.wikimedia.org <https://donate.wikimedia.org/>* _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics