Wikitech-l February 2016

wikitech-l@lists.wikimedia.org

103 participants
104 discussions

External images in VE/Parsoid
by James Montalvo 23 Feb '16

23 Feb '16

I'm trying to make images from an external source, provided by a parser function, work with VisualEditor and Parsoid. For a very simplified illustration I added the following to the bottom of LocalSettings.php ``` $wgExtensionMessagesFiles['myPfTest'] = "$IP/myPfTest.php"; $wgHooks['ParserFirstCallInit'][] = function( &$parser ) { $parser->setFunctionHook( 'test_func', function ( &$parser ) { // Output a wiki image and an external image $output = "[[File:Test.png|frameless|300px]]" . "<br />" . "<img src='http://goo.gl/fh3yKh' />"; return array( $output, "noparse" => true, "isHTML" => true, ); }, SFH_OBJECT_ARGS ); }; $wgAllowExternalImages = false; $wgAllowImageTag = false; ``` To satisfy the $wgExtensionMessagesFiles requirement, I also added a file at $IP/myPfTest.php with the contents: ``` <?php $magicWords = array(); $magicWords['en'] = array( 'test_func' => array( 0, 'test_func' ), ); ``` When I add a page with wikitext `{{#test_func: }}`, the page displays the raw wikitext of the File:Test.png and shows the actual image for the external image. This is as expected. However, when I click "edit" for Visual Editor, File:Test.png DOES display (not expected), but the external image does not (also not expected). Instead the external image displays the raw text "<img src=' http://goo.gl/fh3yKh' />" Adjusting $wgAllowExternalImages and $wgAllowImageTag to all possible combinations has no effect. Adjusting "noparse" to true makes File:Test.png display in view-mode (expected) as well as VE (expected) but has no effect on the external image. "isHTML" appears to have no effect. Is there some way to make external images play nicely with VE and/or Parsoid? I've tested this on two configurations: MW 1.25.5 PHP 5.6.14 VE be84313 (head of the REL1_25 branch) Parsoid ba26a55 (latest commit I've been able to get working with REL1_25) MW 1.26.2 PHP 5.6.14 VE 34a21d8 (head of the REL1_26 branch) Parsoid 8976ab9 (latest commit as of a week or so ago, I think) Thanks, James

2 7

GSoC 2016
by Mehul Sawarkar 23 Feb '16

23 Feb '16

Hello, I am Mehul Sawarkar, a Computer Science student from India . I want to be part of wikimedia in GSoC 2016.Pplease help me getting started .Is there any small bug that I can fix? Thanks, Mehul Sawarkar.

2 1

Re: [Wikitech-l] GSoC 2016
by Bahodir Mansurov 23 Feb '16

23 Feb '16

Hi and welcome, Here are some easy tasks: https://phabricator.wikimedia.org/project/view/169/ Baha On Tue, 23 Feb 2016 11:04:28 -0500, Mehul Sawarkar <mehulsawarkar(a)gmail.com> wrote: > Hello, > > I am Mehul Sawarkar, a Computer Science student from India . I want to be > part of wikimedia in GSoC 2016.Pplease help me getting started .Is there > any small bug that I can fix? > > Thanks, > Mehul Sawarkar. > _______________________________________________ > Wikitech-l mailing list > Wikitech-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Baha

3 2

OOjs UI 0.16.0 release (breaking change)
by James Forrester 22 Feb '16

22 Feb '16

Hey all, I released OOjs UI 0.16.0 today. It will be in MediaWiki core from 1.27.0-wmf.15 which will be deployed to Wikimedia production in the regular train, starting on Tuesday 1 March. As there are four breaking changes (albeit some nominal) please look carefully over them to determine if they affect your code. Breaking and deprecating changes since last release: - DraggableGroupElement: Add default implementation of re-order (Ed Sanders) This mixin previously wouldn't re-oder its own contents, which instead needed to be implemented (and duplicated) by each downstream. We now provide a basic reorder() method, so that this is no longer needed. Widgets with more complex requirements can over-ride the method. The events which we previously emitted for downstream widgets to implement this themselves are still emitted but are deprecated and will be removed. The only known users of this are in Editing's own VisualEditor-MediaWiki and TemplateData, both of which we shall fix. - SelectFileWidget: Remove deprecated config 'dragDropUI' (Prateek Saxena) This option had been deprecated some time ago, in v0.12.7, but we (I) clearly forgot to remove it. :-) Prateek noticed this whilst doing wider clean-up for this widget, and so we're now (finally) removing it. This option has not been used by anyone since August last year, as far as we know. - MenuOptionsWidgets: Drop jQuery autoEllipsis support (Bartosz Dziewoński) This feature was very expensive in terms of both execution performance and in resource terms, and was only used as far as we know inside VisualEditor. Thus we have killed this feature, which only provided the option to add '…' inside labels rather than at their end when they didn't fit inside the menu. Thus we no longer support the 'autoFitLabel' configuration option, and the getLabel() call is now deprecated. - Element#scrollIntoView: Replace callback with promise (Ed Sanders) Having a callback for scrollIntoView was out of keeping with the architecture of the rest of the library, so we deprecated it with a returned promise which does the same. - Remove 'noimages' distribution (Bartosz Dziewoński) As part of our on-going work to slim down the library for wider usage, during the 0.15.x series of release we (mostly MatmaRex) split out the core parts in to a 'core' distribution, with the rest of the library's components available in other distributions. These changes removed the justification for retaining our 'noimages' distribution of the library without the images, as it was only of use to MediaWiki, which no longer needs it. This, this is only nominally a breaking change in the names of files to be loaded by downstream environments which should not have any effect on developers or end-users. - Require PHP 5.5.9+; drop old array syntax (James D. Forrester) We have simplified our server-side code, as we now require PHP version 5.5.9, or higher, for the execution environment. This change was done following that by MediaWiki, dropping of support for PHP 5.3.x, as the only significant user of the library. This is thus a nominal breaking change in the environments we support, which should not have any effect on developers or end-users. Additional details are in the full changelog[0]. If you have any further queries or need help dealing with deprecations, please let me know. As always, a general set of library documentation is available on mediawiki.org[1], and there is some comprehensive generated code-level documentation and interactive demos hosted on doc.wikimedia.org[2]. [0] - https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md [1] - https://www.mediawiki.org/wiki/OOjs_UI [2] - https://doc.wikimedia.org/oojs-ui/master/ J. -- James D. Forrester Lead Product Manager, Editing Wikimedia Foundation, Inc. jforrester(a)wikimedia.org | @jdforrester

1 0

Issue with separate webserver port for Parsoid
by James Montalvo 22 Feb '16

22 Feb '16

The mw:Parsoid/Troubleshooting page ( https://www.mediawiki.org/wiki/Parsoid/Troubleshooting) says the following: > If your MediaWiki runs via SSL, make sure that your Parsoid server can access your MediaWiki without certificate errors. In other words; make sure that the certificate is valid and added to the certificate storage on your Parsoid-running system. Alternatively and similarly, you can set up your webserver to listen on another port, that is not open on your firewall, thus it can be standard http without SSL, therefore be accessed by Parsoid. I chose option two, using a separate port. My relevant Apache config is below: # standard https entry Listen 443 <VirtualHost *:443> SSLEngine on SSLCertificateFile /path/to/cert.crt SSLCertificateKeyFile /path/to/key.key <Directory /opt/meza/htdocs> AllowOverride All Options Indexes FollowSymLinks Require all granted Options All -Indexes </Directory> DocumentRoot /opt/meza/htdocs ServerName Meza </VirtualHost> # parsoid entry (port 9000 not open in firewall) Listen 9000 <VirtualHost *:9000> <Directory /opt/meza/htdocs> AllowOverride All Options Indexes FollowSymLinks Require all granted Options All -Indexes </Directory> DocumentRoot /opt/meza/htdocs ServerName MezaParsoidEntryPoint </VirtualHost> Parsoid and VisualEditor are working great with this setup for everything except images. When I first add an image (VE --> Insert --> Media) it works as expected. The image displays and is configurable. When I save the page everything functions properly. However, when I click edit again the image does not show. Shortly thereafter the request for the image times out and I get the following error in the browser console: GET http://<my-domain>:9000/bme/img_auth.php/thumb/6/66/BME_sign.jpg/400px-BME_sign.jpg net::ERR_CONNECTION_TIMED_OUT Note that it's attempting to load the image over port 9000, not 443. Is there a way to tell images to load over the standard entry point? Thanks, James

2 2

setting $wgDefaultUserOptions['language'] = 'de'; fails
by planetenxin 22 Feb '16

22 Feb '16

Hi, for technical reasons (Semantic), I need to set a wiki's (1.25.x) $wgLanguageCode to English but want to have the user's default language set to German. $wgLanguageCode = 'en'; $wgDefaultUserOptions['language'] = 'de'; The $wgDefaultUserOptions['language'] = 'de'; seems to be ignored. New users still get default 'en'. Did I miss something? Many thanks,

1 0

New PHP profiling tool available on WMF cluster
by Ori Livneh 22 Feb '16

22 Feb '16

Hello, I set up XHGui on the Wikimedia Foundation's production cluster: https://performance.wikimedia.org/xhgui/ XHGui is a web application for browsing and analyzing PHP profiling data collected via the XHProf hierarchical profiler. It's now available on the Wikimedia Foundation's production cluster: https://performance.wikimedia.org/xhgui/ You can see an example of a profiled request here: https://performance.wikimedia.org/xhgui/run/view?id=56ca93be7ed6ccf971b5b693 I set things up so all requests which bear the X-Wikimedia-Debug header (see <https://wikitech.wikimedia.org/wiki/X-Wikimedia-Debug>) are profiled and entered into XHGui's database. Anyone can use this setup to profile requests and view profiling data, but for the moment POSTing to XHGui (to add function watches, create custom views, etc.) requires WMF staff or NDA LDAP credentials. Note that cookie, client IP, form data, and all query parameters except 'action', are stripped from requests. Also note that the app server that handles X-Wikimedia-Debug requests is configured to process messages in all log channels, which grossly inflates the runtime of log-processing functions.

1 0

Re: [Wikitech-l] [WikimediaMobile] Readership metrics for the two months until February 7, 2016
by Tilman Bayer 19 Feb '16

19 Feb '16

Hi Orsolya, a PDF copy is at https://commons.wikimedia.org/wiki/File:Readership_metrics_for_the_two_mont… (previous reports and various charts are also available at https://commons.wikimedia.org/wiki/Category:Wikimedia_readership_metrics_re… ) And of course these mailing lists have public archives too, although they don't play very well with HTML attachments ( https://lists.wikimedia.org/pipermail/mobile-l/2016-February/010096.html ). On Fri, Feb 19, 2016 at 11:01 AM, Orsolya Gyenes < gyenes.orsolya(a)wiki.media.hu> wrote: > Dear Tilman, > > Can you provide a link for this summary below? > > Thank you, > > *~Orsolya* > > 2016-02-18 23:37 GMT+01:00 Tilman Bayer <tbayer(a)wikimedia.org>: > >> Hi all, >> >> this resumes the usual look at our most important readership metrics. >> Among other things, this time we observe the annual Christmas slump in >> pageviews, hail the advent of the mobile singularity (December saw the >> first ever day with >50% mobile pageviews), and resolve the mystery of the >> Android app’s installation drop since mid November. >> >> As laid out earlier >> <https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html>, >> the main purpose is to raise awareness about how these are developing, call >> out the impact of any unusual events in the preceding week, and facilitate >> thinking about core metrics in general. We are still iterating on the >> presentation; feedback and discussion welcome. >> >> After switching away from the weekly schedule (week-over-week and >> month-over-month changes are now being recorded on the Product page >> <https://www.mediawiki.org/wiki/Wikimedia_Product#Reading> at >> MediaWiki.org) we also skipped an issue and added one week, so this edition >> of the report covers a timespan of nine weeks. >> >> See also the slides from the Reading team’s quarter review meeting >> <https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarter…> >> on January 20 for an in-depth look at various metrics. >> >> Now to the usual data. (All numbers below are averages for December 7, >> 2015 to February 7, 2016 unless otherwise noted.) >> >> Pageviews >> >> Total: 529 million/day >> >> Context (April 2015-February 2016): >> >> >> >> (see also the Vital Signs dashboard >> <https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews>) >> >> Total pageviews saw a sharp drop about a week before Christmas (almost >> entirely on desktop), coinciding with a drop >> <http://discovery.wmflabs.org/external/#traffic_by_engine> in Google >> referrals. But both recovered in early January, even rising above earlier >> levels, thanks to mobile (cf. below). Historical data from one and two >> years ago shows a very similar slump in total and desktop pageviews in the >> second half of December (with a lasting increase in mobile around >> Christmas, too), so I’m interpreting this as seasonal even though it >> preceded the actual holidays by a few days. In January 2014, traffic did >> not fully recover to the levels of November/early December. In January >> 2015, it rose slightly above them, similar to now. >> >> Desktop: 54.3% >> >> Mobile web: 44.4% >> >> Apps: 1.3% >> >> Context (April 2015-February 2016): >> >> >> December 20, 2015 will forever be engraved in Wikimedia history as the >> first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). >> It was a Sunday - as we have known for a long time, mobile usage is higher >> on weekends. In general, the mobile percentage over a whole week still >> remains well below 50%. But even as desktop pageviews recovered from the >> Christmas slump at the beginning of January, the mobile percentage remains >> roughly 2% higher than before mid-December - quite likely due to a lot of >> new phones entering usage as Christmas presents. >> >> Global North ratio: 78.3% of total pageviews >> >> Context (April 2015-February 2016): >> >> New app installations >> >> Android: 38.4k/day >> >> Daily installs per device, from Google Play >> >> Context (January 2015-February 2016): >> >> As reported earlier, the app was featured twice on Google Play in recent >> months, and we can now estimate how many additional installs each may have >> caused. November’s placement in the “new and updated” section appears to >> have brought in more than 200k installs, and from early December to early >> January, the apps was featured as one of the “Best Apps of 2015” in many >> countries, resulting in around 350k additional installs. >> >> The preceding reports also described how we noticed a sharp drop in the >> new install rate around November 12, and started investigating the cause >> with our contact at Google. After some detective work, it turned out that >> the app had been benefitting (since April) from a new feature >> <https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-thro…> >> in Google Search showing install buttons for a website’s app next to search >> results from that site for general search terms. Google ended this >> experiment in November, causing the app’s baseline install rate to drop >> significantly (coincidentally around the end of the “new and updated” >> promotion). Going forward, this means that the ongoing growth rate >> (installs minus uninstalls) remains much lower than it was during most of >> 2015. >> >> >> iOS: 4.94k/day >> >> Download numbers from App Annie >> >> Context (last three months): >> >> Like for the Android app, there was a notable bump around Christmas, but >> also an even larger spike on January 14 - the reason is not clear to us. >> >> App user retention >> >> Android: 16.8% >> >> (Ratio of app installs opened again 7 days after installation, among all >> installed during the previous week. 1:100 sample) >> >> Context (last four months): >> >> Retention for installations since around Christmas appears to be higher; >> but the data is quite noisy. >> >> iOS: N/A >> >> (Ratio of app installs opened again 7 days after installation, among all >> installed during the previous week. From iTunes Connect, opt-in only = ca. >> 20-30% of all users) >> >> As reported earlier, we encountered data quality issues with this metric. >> We eventually received an explanation from Apple last week, but it doesn’t >> look very satisfactory for our purposes. We are now working >> <https://phabricator.wikimedia.org/T126693> to measure retention >> ourselves via EventLogging, like we do on Android. >> Unique app users >> >> Android: 1.206 million / day >> >> Context (last four months): >> >> A clearly visible and lasting increase around Christmas, which again >> indicates that the retention rates for the install peak around that time >> were higher than during the install peaks caused by the Google Play >> promotions (even though they resulted in more installs initially). >> >> iOS: 303 k / day >> >> Context (last four months): >> >> Similar to Android, a clearly visible and lasting increase around >> Christmas. >> >> ---- >> >> For reference, the queries and source links used are listed below (access >> is needed for each). Unless otherwise noted, all content of this report is >> © Wikimedia Foundation and released under the CC BY-SA 3.0 >> <https://creativecommons.org/licenses/by-sa/3.0/> license. Most of the >> above charts are available on Commons, too >> <https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20…> >> . >> >> hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS >> avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = >> 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN >> "2015-12-07" AND "2016-02-07"; >> >> hive (wmf)> SELECT year, month, day, >> CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, >> sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, >> SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND >> agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day >> LIMIT 1000; >> >> hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM >> wmf.projectview_hourly WHERE agent_type = 'user' AND >> CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" >> AND "2016-02-07" GROUP BY access_method; >> >> hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, >> 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US') >> > 0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE >> agent_type = 'user' AND >> CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" >> AND "2016-02-07"; >> >> hive (wmf)> SELECT year, month, day, >> CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS >> all, SUM(IF (FIND_IN_SET(country_code, >> 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US') >> > 0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly >> WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY >> year, month, day LIMIT 1000; >> >> >> https://console.developers.google.com/storage/browser/pubsite_prod_rev_0281… >> (“overview”) >> >> >> https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdo… >> (select “Total”) >> >> SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, >> 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS >> day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent >> LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY >> DATE; >> >> (with the retention rate calculated as day7_active divided by day0_active >> from seven days earlier, of course) >> >> https://analytics.itunes.apple.com/#/retention?app=324715238 >> >> hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) >> as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily >> WHERE platform = 'Android'; >> >> hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) >> as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE >> platform = 'iOS'; >> >> >> -- >> Tilman Bayer >> Senior Analyst >> Wikimedia Foundation >> IRC (Freenode): HaeB >> >> _______________________________________________ >> Mobile-l mailing list >> Mobile-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> >> > -- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB

1 0

Re: [Wikitech-l] [WikimediaMobile] Readership metrics for the two months until February 7, 2016
by Oliver Keyes 19 Feb '16

19 Feb '16

Some of it, I suspect, is inevitable. I studied this a while back and a lot of prominent countries were very close to the 50% switchover, including (interestingly) Italy. On 19 February 2016 at 06:51, Antoine Musso <hashar+wmf(a)free.fr> wrote: > Le 18/02/2016 23:37, Tilman Bayer a écrit : >> Context (last three months): >> >> Like for the Android app, there was a notable bump around Christmas, but >> also an even larger spike on January 14 - the reason is not clear to us. > > Hello, > > I would suspect it is related to Wikipedia turning 15 years old. The > birthday has been well covered by news at least in France. A few > acquaintance learned about the Wikipedia mobile apps by reading the news > and started using it. > > Quoting someone I met: > > "I never heard you have a Wikipedia app. It is nice and usefull, much > better than the web view". > > > Hearsay, I have no fact ... > > -- > Antoine "hashar" Musso > > > _______________________________________________ > Mobile-l mailing list > Mobile-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/mobile-l -- Oliver Keyes Count Logula Wikimedia Foundation

2 1

Readership metrics for the two months until February 7, 2016
by Tilman Bayer 18 Feb '16

18 Feb '16

Hi all, this resumes the usual look at our most important readership metrics. Among other things, this time we observe the annual Christmas slump in pageviews, hail the advent of the mobile singularity (December saw the first ever day with >50% mobile pageviews), and resolve the mystery of the Android app’s installation drop since mid November. As laid out earlier <https://lists.wikimedia.org/pipermail/mobile-l/2015-September/009773.html>, the main purpose is to raise awareness about how these are developing, call out the impact of any unusual events in the preceding week, and facilitate thinking about core metrics in general. We are still iterating on the presentation; feedback and discussion welcome. After switching away from the weekly schedule (week-over-week and month-over-month changes are now being recorded on the Product page <https://www.mediawiki.org/wiki/Wikimedia_Product#Reading> at MediaWiki.org) we also skipped an issue and added one week, so this edition of the report covers a timespan of nine weeks. See also the slides from the Reading team’s quarter review meeting <https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/Quarter…> on January 20 for an in-depth look at various metrics. Now to the usual data. (All numbers below are averages for December 7, 2015 to February 7, 2016 unless otherwise noted.) Pageviews Total: 529 million/day Context (April 2015-February 2016): (see also the Vital Signs dashboard <https://vital-signs.wmflabs.org/#projects=all/metrics=Pageviews>) Total pageviews saw a sharp drop about a week before Christmas (almost entirely on desktop), coinciding with a drop <http://discovery.wmflabs.org/external/#traffic_by_engine> in Google referrals. But both recovered in early January, even rising above earlier levels, thanks to mobile (cf. below). Historical data from one and two years ago shows a very similar slump in total and desktop pageviews in the second half of December (with a lasting increase in mobile around Christmas, too), so I’m interpreting this as seasonal even though it preceded the actual holidays by a few days. In January 2014, traffic did not fully recover to the levels of November/early December. In January 2015, it rose slightly above them, similar to now. Desktop: 54.3% Mobile web: 44.4% Apps: 1.3% Context (April 2015-February 2016): December 20, 2015 will forever be engraved in Wikimedia history as the first day where mobile pageviews surpassed desktop pageviews (51% vs. 49%). It was a Sunday - as we have known for a long time, mobile usage is higher on weekends. In general, the mobile percentage over a whole week still remains well below 50%. But even as desktop pageviews recovered from the Christmas slump at the beginning of January, the mobile percentage remains roughly 2% higher than before mid-December - quite likely due to a lot of new phones entering usage as Christmas presents. Global North ratio: 78.3% of total pageviews Context (April 2015-February 2016): New app installations Android: 38.4k/day Daily installs per device, from Google Play Context (January 2015-February 2016): As reported earlier, the app was featured twice on Google Play in recent months, and we can now estimate how many additional installs each may have caused. November’s placement in the “new and updated” section appears to have brought in more than 200k installs, and from early December to early January, the apps was featured as one of the “Best Apps of 2015” in many countries, resulting in around 350k additional installs. The preceding reports also described how we noticed a sharp drop in the new install rate around November 12, and started investigating the cause with our contact at Google. After some detective work, it turned out that the app had been benefitting (since April) from a new feature <https://googlewebmastercentral.blogspot.com/2015/04/drive-app-installs-thro…> in Google Search showing install buttons for a website’s app next to search results from that site for general search terms. Google ended this experiment in November, causing the app’s baseline install rate to drop significantly (coincidentally around the end of the “new and updated” promotion). Going forward, this means that the ongoing growth rate (installs minus uninstalls) remains much lower than it was during most of 2015. iOS: 4.94k/day Download numbers from App Annie Context (last three months): Like for the Android app, there was a notable bump around Christmas, but also an even larger spike on January 14 - the reason is not clear to us. App user retention Android: 16.8% (Ratio of app installs opened again 7 days after installation, among all installed during the previous week. 1:100 sample) Context (last four months): Retention for installations since around Christmas appears to be higher; but the data is quite noisy. iOS: N/A (Ratio of app installs opened again 7 days after installation, among all installed during the previous week. From iTunes Connect, opt-in only = ca. 20-30% of all users) As reported earlier, we encountered data quality issues with this metric. We eventually received an explanation from Apple last week, but it doesn’t look very satisfactory for our purposes. We are now working <https://phabricator.wikimedia.org/T126693> to measure retention ourselves via EventLogging, like we do on Android. Unique app users Android: 1.206 million / day Context (last four months): A clearly visible and lasting increase around Christmas, which again indicates that the retention rates for the install peak around that time were higher than during the install peaks caused by the Google Play promotions (even though they resulted in more installs initially). iOS: 303 k / day Context (last four months): Similar to Android, a clearly visible and lasting increase around Christmas. ---- For reference, the queries and source links used are listed below (access is needed for each). Unless otherwise noted, all content of this report is © Wikimedia Foundation and released under the CC BY-SA 3.0 <https://creativecommons.org/licenses/by-sa/3.0/> license. Most of the above charts are available on Commons, too <https://commons.wikimedia.org/w/index.php?title=Special:ListFiles&offset=20…> . hive (wmf)> SELECT SUM(view_count)/(7000000*9) AS avg_daily_views_millions FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07"; hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, sum(IF(access_method <> 'desktop', view_count, null)) AS mobileviews, SUM(view_count) AS allviews FROM wmf.projectview_hourly WHERE year > 0 AND agent_type = 'user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; hive (wmf)> SELECT access_method, SUM(view_count)/7 FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07" GROUP BY access_method; hive (wmf)> SELECT SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US') > 0, view_count, 0))/SUM(view_count) FROM wmf.projectview_hourly WHERE agent_type = 'user' AND CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) BETWEEN "2015-12-07" AND "2016-02-07"; hive (wmf)> SELECT year, month, day, CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")), SUM(view_count) AS all, SUM(IF (FIND_IN_SET(country_code, 'AD,AL,AT,AX,BA,BE,BG,CH,CY,CZ,DE,DK,EE,ES,FI,FO,FR,FX,GB,GG,GI,GL,GR,HR,HU,IE,IL,IM,IS,IT,JE,LI,LU,LV,MC,MD,ME,MK,MT,NL,NO,PL,PT,RO,RS,RU,SE,SI,SJ,SK,SM,TR,VA,AU,CA,HK,MO,NZ,JP,SG,KR,TW,US') > 0, view_count, 0)) AS Global_North_views FROM wmf.projectview_hourly WHERE year > 0 AND agent_type='user' GROUP BY year, month, day ORDER BY year, month, day LIMIT 1000; https://console.developers.google.com/storage/browser/pubsite_prod_rev_0281… (“overview”) https://www.appannie.com/dashboard/252257/item/324715238/downloads/?breakdo… (select “Total”) SELECT LEFT(timestamp, 8) AS date, SUM(IF(event_appInstallAgeDays = 0, 1, 0)) AS day0_active, SUM(IF(event_appInstallAgeDays = 7, 1, 0)) AS day7_active FROM log.MobileWikiAppDailyStats_12637385 WHERE userAgent LIKE '%-r-%' AND userAgent NOT LIKE '%Googlebot%' GROUP BY date ORDER BY DATE; (with the retention rate calculated as day7_active divided by day0_active from seven days earlier, of course) https://analytics.itunes.apple.com/#/retention?app=324715238 hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS Android_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'Android'; hive (wmf)> SELECT CONCAT(year,"-",LPAD(month,2,"0"),"-",LPAD(day,2,"0")) as date, unique_count AS iOS_DAU FROM wmf.mobile_apps_uniques_daily WHERE platform = 'iOS'; -- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB

2 1

← Newer
1
2
3
4
5
6
7
...
11
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l February 2016