Wikitech-l February 2017

wikitech-l@lists.wikimedia.org

68 participants
72 discussions

Discovery Weekly Update for the week starting 2017-02-13
by Chris Koerner 19 Feb '17

19 Feb '17

Hello, Here are the updates from the Discovery department for the week of 13 February. As always, feedback on these updates are welcome. I hope you're having a good day. == Discussions == === Search === * The inter-wiki search test has been modified this week to allow for more bucketing/sampling of users so that we can get more data to analyze. This is an A/B test of a new search results page which includes inter-wiki search results that is deployed to Persian, Italian, Catalan and Polish Wikipedias. Take a look and give us feedback! [0] [1] * Our codfw elasticsearch cluster has been upgraded to Debian Jessie, its partitioning has been standardised and a 12 new servers have been added [2] [3] [4] * Wikidata Query Service has two more brand new servers [5] [6] [0] https://fr.wikipedia.org/wiki/Special:Search?search=passion+for+food&fullte… [1] https://phabricator.wikimedia.org/T149806 [2] https://phabricator.wikimedia.org/T151326 [3] https://phabricator.wikimedia.org/T151328 [4] https://phabricator.wikimedia.org/T154251 [5] https://phabricator.wikimedia.org/T152643 [6] https://phabricator.wikimedia.org/T152643 ---- The archive of all past updates can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Discovery/Status_updates Interested in getting involved? See tasks marked as "Easy" or "Volunteer needed" in Phabricator. [1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R [2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R Yours, Chris Koerner Community Liaison - Discovery Wikimedia Foundation

1 0

Your feedback matters: Last reminder about the global Wikimedia survey
by Edward Galvez 18 Feb '17

18 Feb '17

Hello Wikimedia technical contributors & developers: Just sending one last reminder. The Wikimedia Foundation is asking for your feedback in a survey. ***The survey will close on February 15, 2017.*** We want to know how well we are supporting your contributions on and off wiki, and how we can change or improve things in the future.[1] The opinions you share will directly affect the current and future work of the Wikimedia Foundation. To say thank you for your time, we are giving away 10 Wikimedia T-shirts to randomly selected people who take the survey.[2] The survey is available in various languages and will take between 20 and 40 minutes. Use this link to take the survey now: https://wikimedia.qualtrics.com/SE/?SID=SV_6mTVlPf6O06r3mt&Aud=DEV&Src=DEV You can find more information about this project here[3]. This survey is hosted by a third-party service and governed by this privacy statement[4]. Please visit our frequently asked questions page to find more information about this survey[5]. If you need additional help or have questions about this survey, send an email to surveys(a)wikimedia.org. Thank you! Edward Galvez Community Engagement Wikimedia Foundation [1] This survey is primarily meant to get feedback on the Wikimedia Foundation's current work, not long-term strategy. [2]Legal information we have to share: No purchase necessary. Must be the age of majority to participate. Sponsored by the Wikimedia Foundation located at 149 New Montgomery, San Francisco, CA, USA, 94105. Ends February 16, 2017. Void where prohibited. Follow this link for the contest rules: https://meta.wikimedia.org/wiki/Community_Engagement_ Insights/2017_second_contest_rules [3] About this survey: https://meta.wikimedia.org/wiki/Community_Engagement_Insight s/About_CE_Insights [4] Privacy statement: https://wikimediafoundation.org/wiki/ Community_Engagement_Insights_2016_Survey_Privacy_Statement [5] FAQ: https://meta.wikimedia.org/wiki/Community_Engagement_Insight s/Frequently_asked_questions -- Edward Galvez Evaluation Strategist (Survey Specialist), and Affiliations Committee Liaison Learning & Evaluation Community Engagement Wikimedia Foundation

1 2

SECURITY: Flow security fix to make sure EnableFlow is always attributed
by Matthew Flaschen 18 Feb '17

18 Feb '17

There is a security fix to ensure that EnableFlow is always properly attributed. This may be an issue if you see users maliciously using Special:EnableFlow on pages that already exist. It should be merged shortly, but in the meantime, you can download it from Gerrit (https://gerrit.wikimedia.org/r/#/c/333301/): git fetch ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Flow refs/changes/01/333301/1 && git checkout FETCH_HEAD Matt Flaschen

1 2

[ANNOUNCEMENT] Node 6 in WMF production
by Marko Obrovac 17 Feb '17

17 Feb '17

Hello, In the second half of January 2017, the version of Node running in WMF production has been updated from v4 to v6~[1]. Parsoid, RESTBase, AQS and the services running on the SCB cluster (Graphoid, Mathoid, Mobile Content Service, to mention just a few) are now running on Node v6. The only outstanding Node service which is still running Node v4 is Maps, but we are making good progress on moving it to Node v6 and expect it to happen soon~[2]. In the case of services running in WMF production, we have seen a slight increase in performance as well as a substantial decrease in memory consumption across the board, which improves the stability of our services. You can read more about the experience of the update and its results in the post published on Wikimedia's blog~[3]. Big thanks to all of the people that helped to make it happen! Happy Friday, Marko && the Services Team [1] https://phabricator.wikimedia.org/T149331 [2] https://phabricator.wikimedia.org/T150354 [3] https://blog.wikimedia.org/2017/02/17/node-6-wikimedia/ -- Marko Obrovac, PhD Senior Services Engineer Wikimedia Foundation

4 3

This week in logspam
by Tyler Cipriani 17 Feb '17

17 Feb '17

Hi all! Logspam makes it difficult to glance at error logs after a deployment and reason about a deployment's impact [0]. Release Engineering is making a conscious effort (in Scrum of Scrums, in Phabricator, and on mailing lists) to connect logspam tasks with folks that can make an impact on these tasks (and, consequentially, make an impact on deployments). Sometimes, as now, our effort takes the form of a broad appeal to help investigate high impact logspam tasks. This week in logspam: * wfShellExec errors end up in HHVM log [1] These errors are the noisiest of the noisy recently. They take various forms, but many relate to PDF handling and start with "SyntaxError" * Warning: Cannot modify header information - headers already sent [2] The latest iteration of this error seems to have started with the release of 1.29.0-wmf.10 * Warning: timed out after 0.2 seconds when connecting to rdb1001.eqiad.wmnet [110]: Connection timed out [3] saw a bit of movement last week, but there are some unanswered questions and the message is still going strong. * Couple of session related ones ** Session "{session}": Unverified user provided and no metadata to auth it [4] ** Session "{session}": Metadata merge failed: {exception} [5] * Throttler: throttler data not found for {user} [6] If you or anyone you know has information that can lead to the cessation of these errors, please add that person or a comment on the tasks listed here. Thus concludes another exciting week in logspam. <3 -- RelEngers [0] <https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Logspam> [1] <https://phabricator.wikimedia.org/T157646> [2] <https://phabricator.wikimedia.org/T157392> [3] <https://phabricator.wikimedia.org/T125735> [4] <https://phabricator.wikimedia.org/T158367> [5] <https://phabricator.wikimedia.org/T158365> [6] <https://phabricator.wikimedia.org/T158364>

1 0

2017-02-15 Scrum of Scrums meeting notes
by Grace Gellerman 16 Feb '17

16 Feb '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-02-15 = 2017-02-15 = == Technology == == Analytics == - Finished rebuilding pageview API for resiliency (10x improvments on throughput and latency were done last quarter). A “programatic way to get pageviews” was one of the most requested items on community wishlist last year. - Still working with DBAs to get data from labs, working functionally, now working on performance - AB testing document design done, will publish design document. - Working on changes to parse user agent on eventlogging data so it is not raw in tables - Merged many patches to delete data aon hadoop on our 60 days schedule === Security=== Reviews: - PageForms - 3D extension - Internal File Server == Product == === Reading === ==== iOS native app ==== * Last Week ** Continued work on 5.4 https://phabricator.wikimedia.org/project/view/2326/ *** Nearby map view is feature complete *** Completed login updates and addition of 2FA * This week ** Continue work on 5.4 *** Location permission changes (friendlier prompt, users to opt-in instead of an intrusive pop-up) *** Other bug fixes and enhancements ==== Web ==== * Current sprint: https://phabricator.wikimedia.org/project/board/2460/ * Last week: ** Create Reading Depth schema: T155639 (WIP) ** Spikes related to problems we're trying to solve from technical, design, and product points of view: T156259, T157071, and T157071. * Next week: ** Page Previews to be rolled out to 90% of users of italian, russian, greek, catalan, and hebrew wikis: T157700 ** Deploy new branding work to Catalan and Italian wikipedia mobile website: T156794 ====Android==== * Last week: ** 2.5.188 beta released; epicly low crash rates. Production is coming soon! (Final beta release likely today) ** Mapbox library upgrade in progress (some issues with method limit) ** Miscellaneous release time fixes * Next week (https://phabricator.wikimedia.org/project/view/2352/ ): ** Continue improving the offline experience ====Reading Infrastructure==== * working on TemplateStyles, ORES, random log/test errors * TemplateStyles RfC: https://phabricator.wikimedia.org/T155813 * https://gerrit.wikimedia.org/r/#/c/336963/ (more machine-readability in MediaWiki change list pages) could use review * not blocking/blocked: * Mobile Content Service: Kanban Board: https://phabricator.wikimedia.org/project/view/2445/ ** New endpoint for OnThisDay (anniversary) deployed soon (example: /feed/onthisday/selected/mm/dd) ** Working on: minor improvements for onthisday + more tests ** Next: expanding TFA (today's featured article) to more languages using FeaturedFeed extension ====Community Tech==== * Cookie blocks will be tested on testwiki, and hopefully out to enwiki next week https://phabricator.wikimedia.org/T158129 * Bot that generates popular pages report, BRFA soon https://phabricator.wikimedia.org/T156857 * Starting rewrite on XTools ArticleInfo tool https://phabricator.wikimedia.org/T157706 * Other minor feature additions to CopyPatrol, along with porting to Spanish Wikipedia ** https://phabricator.wikimedia.org/T157120 and https://phabricator.wikimedia.org/T155933 * Starting investigation on increasing length of edit summaries for non-Latin languages https://phabricator.wikimedia.org/T153333 Blockers: * Collation version numbering patch needs review https://gerrit.wikimedia.org/r/#/c/272419/ ** Only the authors are apparently that familiar with it, blocking effort to make UCA default for MediaWiki https://phabricator.wikimedia.org/T47611 * Range Contributions test queries and the plan for the infrastructure needs feedback https://phabricator.wikimedia.org/T156318#3023578 === Discovery === * No blockers * A/B test for inter-wiki search results is running on Persian, Italian, Catalan & Polish Wikis. https://phabricator.wikimedia.org/T149806 * TextCat language detection enabled on Dutch wiki and improved on German, English, Spanish, French, Italian, Japanese, Portuguese, and Russian Wikipedias (https://phabricator.wikimedia.org/T149324) * Published analysis for Stempel text analyzer in Polish: https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Stempel_Analyzer_Ana… * Wikidata Query Service data reloaded, many old data errors should be gone now. * Working on ES5 upgrade & wikidata search === Fundraising Tech === * Hiring replacement for Casey, who's turning devops * More work on new PayPal integration: https://phabricator.wikimedia.org/T107372, https://phabricator.wikimedia.org/T153720 * Method to purge CentralNotice banners from caches: https://phabricator.wikimedia.org/T154954 == Wikidata == * Still focussing on "federation" (allow using properties and items from an other Wikibase installation, needed for Commons) … * … as well as support for all the freedom new entity types need (for Wiktionary). * Enabled search engine indexing for a few ArticlePlaceholder pages. == German Technical Wishlist == * TwoColConflict betafeature deployed to dewiki * Work on an extension to copy files to Commons from other wikis while maintaining file & description histories * Drafting a prototype for an improved Special:Search form * No blockers == RelEng == * Blocking ** Dashiki extension for meta wiki (Analytics) * Blocked ** [none] * Updates ** Logspam in current train, please help: https://phabricator.wikimedia.org/T155527#3029895 ** new version of git-fat (thanks otto!), for those that care === Services === * Blockers: none * Updates: ** RFC Discussion *REST API language variant support* *** https://phabricator.wikimedia.org/T154190 ** RFC Discussion *Thumb API* *** https://phabricator.wikimedia.org/T66214 ** Deployed /feed/onthisday endpoint *** request for MCS: add an x-ample in your spec so we can remove it from RB ** REST API 1.0 *** Deprecation and removal of a series of endpoints **** will send announcement soon, deletion on May 1st **** /page/title/ and sub-endpoints - https://phabricator.wikimedia.org/T158100 **** /page/mobile-text - https://phabricator.wikimedia.org/T158128 ** Container infrastructure requirements **** https://docs.google.com/a/wikimedia.org/document/d/1QsCVooqxkeE6tKYTxgoRvRd… == Editing == === Parsing team === * No blockers * Starting to work on audio/video support in Parsoid (VE will follow) * Continuing work on language variant support, needed for zhwiki and others === Collaboration === * Blocking * Blocked * Updates ** Continued improvements to RCFilters. *** More front-end patches have been merged, e.g. handling for conflicting and interacting filters, AJAX support so results update immediately. *** Back-end patch is up for review. This sends structured filter information to the client, and changes how RecentChanges and Watchlist filters are registered. It's still a WIP, but you might want to take a look if your extension uses or might use such filters: https://gerrit.wikimedia.org/r/#/c/337546/ Other changes in Collaboration-maintained extensions ** Echo *** Fix mentions failing editing empty pages *** Use internal diff engine ** Flow *** Bottom-positioned toolbar

3 2

Weigh in on whether to finalize "Amendments" section of Code of Conduct
by Matthew Flaschen 16 Feb '17

16 Feb '17

Please comment on whether to approve the "Amendments" section of the draft Code of conduct for technical spaces. The draft text is at https://www.mediawiki.org/w/index.php?title=Code_of_Conduct/Draft&oldid=238… . You can comment at https://www.mediawiki.org/wiki/Talk:Code_of_Conduct/Draft#Finalize_.22Amend… . A position and brief comment is fine. You can also send private feedback to conduct-discussion(a)wikimedia.org . Thanks again, Matt Flaschen * The "Creation and renewal of the Committee" section has been approved. Thanks to everyone who helped discuss, edit, and approve this section.

1 0

Results of the Developer Wishlist are in
by Gergo Tisza 15 Feb '17

15 Feb '17

Hi all, voting for the Developer Wishlist [1] has ended. Thanks to everyone who participated! And extra thanks to everyone who helped. [2] You can find the the results, with links to the full proposals, at: https://www.mediawiki.org/wiki/User:Community_Tech_bot/WishlistSurvey/Votes As you might have read on the wishlist page, this is a primarily-volunteer run experiment. It can only be successful with your help! [3] If you can afford the time, please look through the highly-voted propsals and see if you can find something you would be interested to work on. I also invite everyone to participate in these three follow-up tasks: https://phabricator.wikimedia.org/T158148 - promote wishlist proposals https://phabricator.wikimedia.org/T158149 - find owners for the top10 proposals https://phabricator.wikimedia.org/T158150 - post-mortem thanks Gergő [1] https://www.mediawiki.org/wiki/Developer_Wishlist [2] Including, but not limited to: Srishti Sethi who handled most of the communications; Quim Gil and James Forrester who helped a lot with the planning; Leon Ziemba who ran the survey maintenance bot; Jeph Paul and Jonathan Morgan who wrote the original version of the voting button gadget; all the people working on the Community Wishlist from which I stole the idea and most of the process/design. [3] https://i.imgflip.com/1jnzy6.jpg

5 5

MediaWiki-Vagrant needs testers for Debian Jessie changes
by Bryan Davis 15 Feb '17

15 Feb '17

TL;DR See <https://phabricator.wikimedia.org/T136429> and try the new jessie-migration branch. The Wikimedia Foundation has been working on converting the majority of its production server farm from Ubuntu to Debian for quite some time. In August of 2016 that project advanced to the point of actually converting the 252 servers running MediaWiki [0]. Today that work has largely been done and the vast majority of MediaWiki is running on Jessie. Now it's time (past time really) for MediaWiki-Vagrant to catch up. We have done this sort of thing once before with the switch from using Ubuntu Precise to Ubuntu Trusty as our base image [1] in mid-2014. We had fewer users then and only one supported virtual machine type. The switch from Trusty to Jessie is also slightly more complicated because the init system used is changing as well. This means that we have more Puppet code changes to make than last time and more people who will be impacted by the change. For a little over a month several of us have been working on the jessie-migration branch to get ready for the change over. I think we are at the point that we could change over, but I want to have a period of testing by a wider audience to see if we can iron out a few more common issues before forcing everyone to either upgrade or explicitly pin their MediaWiki-Vagrant virtual machines to the git tag for the last Trusty compatible build. == Testing the Jessie base image and Puppet profiles == Its recommended to test with a fresh MediaWiki-Vagrant checkout so if things go badly you can easily switch back to your original install and keep working. $ git clone --recursive https://gerrit.wikimedia.org/r/mediawiki/vagrant mwv-jessie $ cd mwv-jessie $ git checkout jessie-migration $ ./setup.sh $ vagrant up You can run vagrant roles list -e -1 to get a nice list of the roles you have enabled on your normal Trusty VM install to copy over to your Jessie testing VM. This one-liner liner might even do it for you: $ cd mwv-jessie $ vagrant roles enable $(cd ../vagrant; vagrant roles list -e -1) $ vagrant provision Give things a try and report issues as children of the tracking task for this migration [2]. Barring major issues effecting many people, I would like to merge the jessie-migration branch with the master branch the week of March 13th. [0]: https://phabricator.wikimedia.org/T143536 [1]: https://phabricator.wikimedia.org/rMWVAdc73e2bee9dff1e0755d15cfe1376ee2dc6e… [2]: https://phabricator.wikimedia.org/T136429 Bryan -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

4 3

Readership metrics for the timespan until February 5, 2017
by Zareen Farooqui 15 Feb '17

15 Feb '17

Link to PDF of report in Commons <https://commons.wikimedia.org/wiki/File:Wikimedia_Readership_metrics_for_th…> Hi all, Here is the usual look <https://www.mediawiki.org/wiki/Reading/Readership_metrics_reports> at our most important readership metrics. This time we see an overall rise in pageviews following the seasonal winter slump, examine the recent year-over-year growth in pageviews more closely, and introduce a new day-7 retention metric for the Wikipedia iOS app. As laid out earlier <https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html>, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events, and facilitate thinking about core metrics in general. As always; feedback and discussion welcome. Week-over-week and month-over-month changes are being recorded on the Product page <https://www.mediawiki.org/wiki/Wikimedia_Product#Reading> at MediaWiki.org. This edition of the report covers a timespan of five weeks. You can also find lots of other traffic and usage data in the quarterly metrics presentation <https://commons.wikimedia.org/wiki/File:Wikimedia_Foundation_Reading_metric…> for Q2 2016-2017 (October - December) that was just published by the WMF Reading team. All numbers below are averages for January 2 - February 5, 2017 unless otherwise noted. Pageviews Total: 582 million/day (+10.14% from the previous report) Context (April 2015-February 2017): See also the Vital Signs dashboard <https://analytics.wikimedia.org/dashboards/vital-signs/#projects=all/metric…> After the seasonal winter slump, we see a rise in desktop pageviews, as expected. Mobile pageviews continue to remain at higher levels than before christmas. The previously mentioned iOS app’s pageview increase is still under investigation, and may turn out to be an anomaly <https://phabricator.wikimedia.org/T154735> inflating mobile pageviews by roughly 5 million views per day. This chart looks at long-term traffic trends from May 2013 - January 2017. This shows that over this timespan, the annual change in overall pageviews was -2%, desktop has been down 15%, and mobile (web + apps) has been trending upwards at a rate of 23% per year. However, the past few months have seen total pageviews increasing year-over-year (chart further below). To facilitate our understanding of which traffic movements are seasonal and which may indicate lasting changes, here is a chart overlaying the total pageview numbers back to May 2013 (the earliest time for which we have data according to the current pageview definition): Total pageviews have continued rising and are now higher than before the winter holidays. The blue line shows that the increase in overall pageviews year-over-year remains (January 2017 is up 5% from January 2016). It is possible that a smaller part of this is due to unidentified bot traffic (e.g. we just updated <https://phabricator.wikimedia.org/T157528> the pageview definition to exclude a recently discovered bot that had been causing up to 0.9% of total pageviews). But overall it is starting to look like a small but sustained rise in real human pageviews. Here we see that the changed trend in recent months can be attributed to desktop pageviews, which have mostly stopped declining year-over-year. This chart also shows that the seasonal christmas dip in pageviews comes predominantly from desktop views. Most recently, January 2017 desktop pageviews are again down 1.8% from January 2016, but that is still much less than the aforementioned -15% yearly trend since 2013. Year-over-year mobile pageviews are still increasing at a notable rate, but slower than in 2013-15. January 2017 mobile pageviews are up 13% from January 2016, less than the yearly mobile growth of 23% in the overall 2013-17 timespan. We can see how mobile pageviews always increase around the winter holidays and maintain higher levels afterwards. Desktop: 50.9% (previous report: 51.5%) Mobile web: 46.7% (previous report: 46.8%) Apps: 2.4% (previous report: 1.7%) Mobile percentage dropped a bit following the christmas spike, but remains high at an average of 49% mobile views. The previously mentioned iOS app’s pageview increase is still under investigation, and may turn out to be an anomaly <https://phabricator.wikimedia.org/T154735> inflating this metric by about 1%. Global North ratio: 77.3% of total pageviews (previous report: 76.8%) Context (January 2016-February 2017): Although Global North pageview percentage fell following the winter spike, but overall this metric grew since the last report. Levels remain in the high 70 percentile range. NB: We are currently rethinking this metric and might replace it with a different country selection constructed as part of the work on the New Readers project Unique devices See the announcement blog post <https://blog.wikimedia.org/2016/03/30/unique-devices-dataset/> from March 2016 for background and details on this metric. These estimated numbers are provided for all Wikimedia language projects (separately for the desktop and mobile web version). Because of the instrumentation method, there is no global metric for all projects and all languages, but it is currently being extended <https://phabricator.wikimedia.org/T138027> to a cross-language global metric per project at least. For now, we track the daily numbers of English Wikipedia in this report. Daily unique devices estimate for English Wikipedia: - Mobile web: 36.3 million - Desktop: 24.3 million - Total: 60.6 million - Average mobile web ratio: 60.0% Context (January 2016- February 2017): Since the last report, both daily unique desktop devices and mobile devices grew. Mobile web daily uniques mirrors the daily pageviews chart - mobile web unique levels have remained higher than before the winter bump. January 2017 was the first month we could compare year-over-year changes in unique devices. While pageviews on English Wikipedia are up from January 2016, the number of monthly unique devices is down 16%. For this report timespan (Jan 2 - Feb 5), daily mobile web uniques are up 2.6%, while daily desktop uniques are down 8.2%. This is currently being investigated. While the number of daily unique desktop devices has grown overall, the ratio of pageviews to devices on desktop has steadily declined back to pre christmas levels. The small, but notable increase in daily views per device for mobile web which happened in November has held up. New app installations Android: 18.0k/day (-5.9% from the previous report) Daily installs per device, from Google Play Context (last nine months): The number of daily installs dropped after the christmas spike, but remains higher than the number of uninstalls (except for January 26). Overall, the number of daily installs has dropped nearly 6% and daily uninstalls dropped about 8% since the last report. iOS: 5.67k/day (+6.5% from the previous report) Download numbers from App Annie Context (last two months): [image: Wikipedia iOS app daily downloads by country, Dec 6, 2016 - Feb 5, 2017 (App Annie).png] Similar to other mobile metrics measured in this report, the iOS app download metric has remained higher than usual following the winter bump. During the timespan of this report, iOS app downloads remain fairly steady besides two large, unusual spikes at the end of January in the United Arab Emirates and the United States (which follow the spike from the Netherlands in December) which are likely anomalies. We are looking into filing a bug report with App Annie to investigate this. App user retention Android: 15.6% (previous report: 16.8%) (Ratio of app installs opened again 7 days after installation for all dates that fall within this report. 1:100 sample) Context (last six months): As remarked in earlier reports, this data is a bit too noisy for drawing conclusions about whether retention changed significantly between different releases. However, we can at least rule out the existence of major shifts during this timespan. There was a small spike in day 7 retention for installs on January 22nd. iOS: 19.2% (previous report: N/A <https://phabricator.wikimedia.org/T126693> ) (Ratio of app installs opened again 7 days after installation for all dates that fall within this report. 100:100 sample since iOS app only sends data when users opt-in so there is a small user base) Context (last two months): After a long break where we refrained from reporting iOS retention here due to data quality issues with the number provided by Apple, we are happy to report iOS app 7 day retention again, based on a new EventLogging instrumentation that parallels the one on Android. There doesn’t seem to have been any significant changes in retention levels during the timespan of this report, even after new releases of the app. Looking back at December though, it appears that there was a period of lower retention which roughly corresponds with stability issues (crashes) the app was having at this time, and which have since been fixed. Unique app users Android: 1.211 million / day (+2.7% from the previous report) Context (last eight months): Android daily average levels have remained higher following the christmas and new year’s bump. This matches the mobile pageviews and unique mobile devices metrics. iOS: N/A <https://phabricator.wikimedia.org/T130432> Zareen Farooqui, Data Analyst Intern, Wikimedia Foundation Tilman Bayer, Senior Analyst, Wikimedia Foundation Data sources For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 <https://creativecommons.org/licenses/by-sa/3.0/> license. Most of the above charts are available on Commons, too. <https://commons.wikimedia.org/wiki/Category:Wikimedia_readership_metrics_re…> SELECT year, month, CONCAT(year,"-",LPAD(month,2,"0")) as yearmonth, SUM(view_count) AS allhuman, sum(IF(access_method = 'desktop', view_count, null)) AS desktophuman FROM wmf.projectview_hourly WHERE year = 2017 AND agent_type = 'user' GROUP BY year, month ORDER BY year, month LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year = 2017 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT access_method, SUM(view_count)/(7*5) FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2017-01-02" AND "2017-02-05" GROUP BY access_method; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,I S,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO, NZ,JP,SG,KR,TW,US') > 0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly WHERE year = 2017 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, SUM(IF(uri_host LIKE 'en.m.wikipedia%', uniques_estimate, 0)) AS enwiki_mobile_web FROM wmf.last_access_uniques_daily WHERE year=2017 GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, SUM(IF(uri_host LIKE 'en.wikipedia%', uniques_estimate, 0)) AS enwiki_desktop FROM wmf.last_access_uniques_daily WHERE year=2017 GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) AS date, SUM(IF(access_method = 'mobile web', view_count, null)) AS mobilewebviews, SUM(IF(access_method = 'desktop', view_count, null)) AS desktopviews FROM wmf.projectview_hourly WHERE year=2017 AND agent_type = 'user' AND project = 'en.wikipedia' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; https://console.cloud.google.com/storage/browser/pubsite_prod_rev_028125227… https://www.appannie.com/dashboard/252257/item/324715238/downloads/ SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE timestamp LIKE '2017%' AND userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE; SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%iPhone%' OR userAgent LIKE '%iOS%' GROUP BY date ORDER BY DATE; SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE year = 2017 AND platform = 'Android'; Zareen Farooqui

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l February 2017