Hi, we at the mobile team are currently working on improving our current hit rate, publishing the half-implemented plan here for review: == Current status == * X-Device header is generated by frontend Varnish from user-agent. * There are currently 21 possible X-Device values, which we decreased to 20 this week. * X-Device is used for HTML variance (roughly, Vary: X-Device). * Depending on X-Device, we alter skin HTML, serve it full or limited resources. * Because some phones need CSS tweaks and don't support media queries, we have to serve them device-specific CSS. * Device-specific CSS is served via separate ResourceLoader modules e.g. mobile.device.android. == What's bad about it? == Cache fragmentation is very high, resulting in ~55% hit rate. == Proposed strategy == * We don't vary pages on X-Device anymore. * Because we still need to give really ancient WAP phones WML output, we create a new header, X-WAP, with just two values, yes or not[1] * And we vary our output on X-WAP instead of X-Device[2] * Because we still need to serve device-specific CSS but can't use device name in page HTML, we create a single ResourceLoader module, mobile.device.detect, which outputs styles depending on X-Device.[2] This does not affect bits cache fragmentation because it simply changes the way the same data is varied, but not adds the new fragmentation factors. Bits hit rate currently is very high, by the way. * And because we need X-Device, we will need to direct mobile load.php requests to the mobile site itself instead of bits. Not a problem because mobile domains are served by Varnish just like bits. * Since now we will be serving ResourceLoader to all devices, we will blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3] == Commits == [1] https://gerrit.wikimedia.org/r/#/c/32866/ - adds X-WAP to Varnish [2] https://gerrit.wikimedia.org/r/55226 - main MobileFrontend change [3] https://gerrit.wikimedia.org/r/#/c/55446/ - ResourceLoader change, just a sketch of a real solution as of the moment I'm writing this Your comments are highly appreciated! :) -- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Max, do we still plan to detect javascript support for mobile devices, or do you want to fold that into "isWAP" ?

Non-js-supporting devices need very different handling, as all HTML has to be pre-built for them on the server.

* Since now we will be serving ResourceLoader to all devices, we will blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]

This approach will require either: 1) Adding device detection to bits for device variance 2) Using mobile varnish to handle load.php requests for resources requested from .m domains From conversations with Max and some folks from ops, it sounds like #2 is the preferred approach, but I am a little nervous about it since mobile varnish caches will have to handle a significant increase in requests. It looks like a typical article load results in 6 load.php requests. Also, we'll need to duplicate some configuration from the bits VCL. Ops, is this OK given current architecture? On Fri, Mar 29, 2013 at 11:18 AM, Max Semenik <maxsem.wiki(a)gmail.com> wrote: or they -- Arthur Richards Software Engineer, Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687 _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi, we at the mobile team are currently working on improving our current hit rate, publishing the half-implemented plan here for review:

== Proposed strategy == * We don't vary pages on X-Device anymore. * Because we still need to give really ancient WAP phones WML output, we create a new header, X-WAP, with just two values, yes or not[1] * And we vary our output on X-WAP instead of X-Device[2] * Because we still need to serve device-specific CSS but can't use device name in page HTML, we create a single ResourceLoader module, mobile.device.detect, which outputs styles depending on X-Device.[2] This does not affect bits cache fragmentation because it simply changes the way the same data is varied, but not adds the new fragmentation factors. Bits hit rate currently is very high, by the way.

* And because we need X-Device, we will need to direct mobile load.php requests to the mobile site itself instead of bits. Not a problem because mobile domains are served by Varnish just like bits. * Since now we will be serving ResourceLoader to all devices, we will blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]

Your comments are highly appreciated! :)