Hi, we at the mobile team are currently working on improving our current hit rate, publishing the half-implemented plan here for review:
== Current status == * X-Device header is generated by frontend Varnish from user-agent. * There are currently 21 possible X-Device values, which we decreased to 20 this week. * X-Device is used for HTML variance (roughly, Vary: X-Device). * Depending on X-Device, we alter skin HTML, serve it full or limited resources. * Because some phones need CSS tweaks and don't support media queries, we have to serve them device-specific CSS. * Device-specific CSS is served via separate ResourceLoader modules e.g. mobile.device.android.
== What's bad about it? == Cache fragmentation is very high, resulting in ~55% hit rate.
== Proposed strategy == * We don't vary pages on X-Device anymore. * Because we still need to give really ancient WAP phones WML output, we create a new header, X-WAP, with just two values, yes or not[1] * And we vary our output on X-WAP instead of X-Device[2] * Because we still need to serve device-specific CSS but can't use device name in page HTML, we create a single ResourceLoader module, mobile.device.detect, which outputs styles depending on X-Device.[2] This does not affect bits cache fragmentation because it simply changes the way the same data is varied, but not adds the new fragmentation factors. Bits hit rate currently is very high, by the way. * And because we need X-Device, we will need to direct mobile load.php requests to the mobile site itself instead of bits. Not a problem because mobile domains are served by Varnish just like bits. * Since now we will be serving ResourceLoader to all devices, we will blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
== Commits == [1] https://gerrit.wikimedia.org/r/#/c/32866/ - adds X-WAP to Varnish [2] https://gerrit.wikimedia.org/r/55226 - main MobileFrontend change [3] https://gerrit.wikimedia.org/r/#/c/55446/ - ResourceLoader change, just a sketch of a real solution as of the moment I'm writing this
Your comments are highly appreciated! :)
This sounds like a great plan. Thank you!
On Fri, Mar 29, 2013 at 2:45 AM, Max Semenik maxsem.wiki@gmail.com wrote:
Hi, we at the mobile team are currently working on improving our current hit rate, publishing the half-implemented plan here for review:
== Current status ==
- X-Device header is generated by frontend Varnish from user-agent.
- There are currently 21 possible X-Device values, which we decreased to 20
this week.
- X-Device is used for HTML variance (roughly, Vary: X-Device).
- Depending on X-Device, we alter skin HTML, serve it full or limited
resources.
- Because some phones need CSS tweaks and don't support media queries, we
have to serve them device-specific CSS.
- Device-specific CSS is served via separate ResourceLoader modules e.g.
mobile.device.android.
== What's bad about it? == Cache fragmentation is very high, resulting in ~55% hit rate.
== Proposed strategy ==
- We don't vary pages on X-Device anymore.
- Because we still need to give really ancient WAP phones WML output, we
create a new header, X-WAP, with just two values, yes or not[1]
- And we vary our output on X-WAP instead of X-Device[2]
- Because we still need to serve device-specific CSS but can't use device
name in page HTML, we create a single ResourceLoader module, mobile.device.detect, which outputs styles depending on X-Device.[2] This does not affect bits cache fragmentation because it simply changes the way the same data is varied, but not adds the new fragmentation factors. Bits hit rate currently is very high, by the way.
- And because we need X-Device, we will need to direct mobile load.php
requests to the mobile site itself instead of bits. Not a problem because mobile domains are served by Varnish just like bits.
- Since now we will be serving ResourceLoader to all devices, we will
blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
== Commits == [1] https://gerrit.wikimedia.org/r/#/c/32866/ - adds X-WAP to Varnish [2] https://gerrit.wikimedia.org/r/55226 - main MobileFrontend change [3] https://gerrit.wikimedia.org/r/#/c/55446/ - ResourceLoader change, just a sketch of a real solution as of the moment I'm writing this
Your comments are highly appreciated! :)
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Max, do we still plan to detect javascript support for mobile devices, or do you want to fold that into "isWAP" ?
Non-js-supporting devices need very different handling, as all HTML has to be pre-built for them on the server.
On Fri, Mar 29, 2013 at 2:45 AM, Max Semenik maxsem.wiki@gmail.com wrote:
Hi, we at the mobile team are currently working on improving our current hit rate, publishing the half-implemented plan here for review:
== Current status ==
- X-Device header is generated by frontend Varnish from user-agent.
- There are currently 21 possible X-Device values, which we decreased to 20
this week.
- X-Device is used for HTML variance (roughly, Vary: X-Device).
- Depending on X-Device, we alter skin HTML, serve it full or limited
resources.
- Because some phones need CSS tweaks and don't support media queries, we
have to serve them device-specific CSS.
- Device-specific CSS is served via separate ResourceLoader modules e.g.
mobile.device.android.
== What's bad about it? == Cache fragmentation is very high, resulting in ~55% hit rate.
== Proposed strategy ==
- We don't vary pages on X-Device anymore.
- Because we still need to give really ancient WAP phones WML output, we
create a new header, X-WAP, with just two values, yes or not[1]
- And we vary our output on X-WAP instead of X-Device[2]
- Because we still need to serve device-specific CSS but can't use device
name in page HTML, we create a single ResourceLoader module, mobile.device.detect, which outputs styles depending on X-Device.[2] This does not affect bits cache fragmentation because it simply changes the way the same data is varied, but not adds the new fragmentation factors. Bits hit rate currently is very high, by the way.
- And because we need X-Device, we will need to direct mobile load.php
requests to the mobile site itself instead of bits. Not a problem because mobile domains are served by Varnish just like bits.
- Since now we will be serving ResourceLoader to all devices, we will
blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
== Commits == [1] https://gerrit.wikimedia.org/r/#/c/32866/ - adds X-WAP to Varnish [2] https://gerrit.wikimedia.org/r/55226 - main MobileFrontend change [3] https://gerrit.wikimedia.org/r/#/c/55446/ - ResourceLoader change, just a sketch of a real solution as of the moment I'm writing this
Your comments are highly appreciated! :)
-- Best regards, Max Semenik ([[User:MaxSem]]) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 29.03.2013, 21:47 Yuri wrote:
Max, do we still plan to detect javascript support for mobile devices, or do you want to fold that into "isWAP" ?
Non-js-supporting devices need very different handling, as all HTML has to be pre-built for them on the server.
ResourceLoader has a small stub module called startup. It checks browser compatibility and then loads jQuery and various MediaWiki modules (including ResourceLoader core). We just need to imporove the checks, as the my original message states:
- Since now we will be serving ResourceLoader to all devices, we will
blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
This approach will require either:
1) Adding device detection to bits for device variance 2) Using mobile varnish to handle load.php requests for resources requested from .m domains
From conversations with Max and some folks from ops, it sounds like #2 is
the preferred approach, but I am a little nervous about it since mobile varnish caches will have to handle a significant increase in requests. It looks like a typical article load results in 6 load.php requests. Also, we'll need to duplicate some configuration from the bits VCL. Ops, is this OK given current architecture?
On Fri, Mar 29, 2013 at 11:18 AM, Max Semenik maxsem.wiki@gmail.com wrote:
On 29.03.2013, 21:47 Yuri wrote:
Max, do we still plan to detect javascript support for mobile devices, or do you want to fold that into "isWAP" ?
Non-js-supporting devices need very different handling, as all HTML has
to
be pre-built for them on the server.
ResourceLoader has a small stub module called startup. It checks browser compatibility and then loads jQuery and various MediaWiki modules (including ResourceLoader core). We just need to imporove the checks, as the my original message states:
- Since now we will be serving ResourceLoader to all devices, we will
blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
-- Best regards, Max Semenik ([[User:MaxSem]])
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Why don't we continue to use the bits cache for all things resourceloader. Can you provide a different path for these requests, such as instead of:
http://bits.wikimedia.org/en.wikipedia.org/load.php?..
use something like:
http://bits.wikimedia.org/m/en.wikipedia.org/load.php?..
Then we can if (req.url ~ "^/m/") { tag_carrier + strip the /m/ }, so the overhead only effects mobile requests.
Faidon has raised that it's still advantageous to shard page resources across more than one domain for browser pipelining.
On Fri, Mar 29, 2013 at 1:55 PM, Arthur Richards arichards@wikimedia.orgwrote:
This approach will require either:
- Adding device detection to bits for device variance
- Using mobile varnish to handle load.php requests for resources requested
from .m domains
From conversations with Max and some folks from ops, it sounds like #2 is the preferred approach, but I am a little nervous about it since mobile varnish caches will have to handle a significant increase in requests. It looks like a typical article load results in 6 load.php requests. Also, we'll need to duplicate some configuration from the bits VCL. Ops, is this OK given current architecture?
On Fri, Mar 29, 2013 at 11:18 AM, Max Semenik maxsem.wiki@gmail.com wrote:
On 29.03.2013, 21:47 Yuri wrote:
Max, do we still plan to detect javascript support for mobile devices,
or
do you want to fold that into "isWAP" ?
Non-js-supporting devices need very different handling, as all HTML has
to
be pre-built for them on the server.
ResourceLoader has a small stub module called startup. It checks browser compatibility and then loads jQuery and various MediaWiki modules (including ResourceLoader core). We just need to imporove the checks, as the my original message states:
- Since now we will be serving ResourceLoader to all devices, we will
blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if
they
degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
-- Best regards, Max Semenik ([[User:MaxSem]])
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Arthur Richards Software Engineer, Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687 _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 30.03.2013, 2:40 Asher wrote:
Why don't we continue to use the bits cache for all things resourceloader. Can you provide a different path for these requests, such as instead of:
use something like:
Then we can if (req.url ~ "^/m/") { tag_carrier + strip the /m/ }, so the overhead only effects mobile requests.
Faidon has raised that it's still advantageous to shard page resources across more than one domain for browser pipelining.
Like https://gerrit.wikimedia.org/r/56774 ?
Hi Max,
On Mar 29, 2013, at 10:45 AM, Max Semenik maxsem.wiki@gmail.com wrote:
Hi, we at the mobile team are currently working on improving our current hit rate, publishing the half-implemented plan here for review:
== Proposed strategy ==
- We don't vary pages on X-Device anymore.
- Because we still need to give really ancient WAP phones WML output, we
create a new header, X-WAP, with just two values, yes or not[1]
- And we vary our output on X-WAP instead of X-Device[2]
- Because we still need to serve device-specific CSS but can't use device
name in page HTML, we create a single ResourceLoader module, mobile.device.detect, which outputs styles depending on X-Device.[2] This does not affect bits cache fragmentation because it simply changes the way the same data is varied, but not adds the new fragmentation factors. Bits hit rate currently is very high, by the way.
Yes. It does add Vary processing on the bits caches for these requests though. But we can change that by including the X-Device header into the hash for cache lookups, if we want to.
- And because we need X-Device, we will need to direct mobile load.php
requests to the mobile site itself instead of bits. Not a problem because mobile domains are served by Varnish just like bits.
- Since now we will be serving ResourceLoader to all devices, we will
blacklist all the incompatible devices in the startup module to prevent them from choking on the loads of JS they can't handle (and even if they degrade gracefully, still no need to force them to download tens of kilobytes needlessly)[3]
Good work! This should help a great deal.
Your comments are highly appreciated! :)
I've been pondering a bit about the two options for serving mobile ResourceLoader requests with Varnish: on the bits caches or on the mobile caches. I don't fully like either option to be honest. On one hand I'd like to keep mobile device detection off our currently very efficient bits caches, on the other hand I don't like the idea of mixing in the RL requests into the high churn of mobile request LRU cache eviction of the frontend caches. Unfortunately Varnish currently doesn't allow us to separate/specify cache backends for objects well.
So let's go with Asher's suggestion indeed, and add the device detection to the bits servers. Let's keep it such that it'll always be easy to distinguish these requests, so we can easily decide to move these to another Varnish cluster at any point.
On 02.04.2013, 20:16 Mark wrote:
I've been pondering a bit about the two options for serving mobile ResourceLoader requests with Varnish: on the bits caches or on the mobile caches. I don't fully like either option to be honest. On one hand I'd like to keep mobile device detection off our currently very efficient bits caches, on the other hand I don't like the idea of mixing in the RL requests into the high churn of mobile request LRU cache eviction of the frontend caches. Unfortunately Varnish currently doesn't allow us to separate/specify cache backends for objects well.
So let's go with Asher's suggestion indeed, and add the device detection to the bits servers. Let's keep it such that it'll always be easy to distinguish these requests, so we can easily decide to move these to another Varnish cluster at any point.
The puppet changes for this are at https://gerrit.wikimedia.org/r/#/c/56774/ Anything else needed?
wikitech-l@lists.wikimedia.org