To be clear Joaquin is asking people to record their experiences on 2G connections of loading the Barack Obama article it is not about displaying video. It would also be interesting for people to try the experience on their desktop browser using 2g and record those experiences as many people around the world connect to our desktop site on 2g. We can and should do better. On 12 Jan 2016 7:58 a.m., "Bernardo Sulzbach" <mafagafogigante(a)gmail.com> wrote:

Sure Bernardo, happy to share. This was the best take I got from around 4 runs. I have other video that I haven't uploaded that goes on for 4 minutes and tries to close chrome 3 times (it is painful and boring to watch). With high speed connections I haven't experienced this behaviour, the worst I've experienced have been around 1s-2s of locked browser (not being able to interact at all) but it quickly recovers and it doesn't show the unresponsive app dialog. I tried diagnosing using chrome developer tools connected to the device but the timeline profiles I recorded didn't show anything really evident (and it should, since it spent 20s blocked). It was pretty confusing, and my best guess is that the sheer size of the HTML (ungzipping, parsing, laying it out, etc) (+1mb of html for B. Obama), and the 100+ images that are being fetched (both the network requests, and the multiple (slow) streams of image data that need to be decoded by the phone) are somehow creating a huge bottleneck on the phone's CPU. I'm thinking of repeating the experiment disabling images and see what difference it makes, to find out if it is the multiple images being decoded in parallel really slowly (because of network speed). You're right Bernardo in that Nexus5 is supposed to be a good phone (not the best, but "modern" for sure), that's why I asked if more people could try doing the same thing and reporting back. I'll also try to run the same experiment with different devices (got a few of old ones for testing), that should yield interesting results. As far as my conclusions go, we need more data from real tests on real devices, and to really care more about this, given the industry numbers and predictions around 2G ( https://commons.wikimedia.org/w/index.php?title=File:Paradigm.pdf&page=8) I'll report back when I get around to testing more! On Tue, Jan 12, 2016 at 10:24 AM, Brian Wolff <bawolff(a)gmail.com> wrote:

Can you clarify what you consider "splitting"? I'm interested in knowing more. I'm of the opinion that there's only so much that browsers can do, there's ways of serving the content like lazy loading that still get you the full content but help the browser do less work. I also think that it's responsible to use those techniques where required (low end network conditions on underpowered devices like phones, for example) as long as the experience is not broken (which needs careful implementation and consideration!). Whatever we need to reach as much people as we can! (and make google not redirect wikipedia to googleweblight on certain countries) On Wed, Jan 13, 2016 at 12:29 PM, Bernardo Sulzbach < mafagafogigante(a)gmail.com&gt; wrote:

I understand, thanks for the response! I don't think in any case any implementation would mean splitting article content in the sense you mentioned here. As you said, it doesn't make sense. Cheers! On Wed, Jan 13, 2016 at 2:35 PM, Bernardo Sulzbach < mafagafogigante(a)gmail.com&gt; wrote:

I like videos because it's so easy for non technical people to understand the value of performance. For me it has been a really good way to show and explain performance changes and what it looks like for the user. Picking a representative connection is hard and the good thing is I think we don't need to. Connections in the real world vary so much: You could have 3G but a really bad connection to the nearest station because there's a lot of other phones connected etc. We have tests running with real connections 2G/3G from Bangalore and LTE in San Francisco, you can check it out how it looks here: https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest?panelId=30&am… I really like that graph because it shows how much variances you can have on the same connection type from the same location. Check out how it looks like in Bangalore. When we throttle the connection for WebPageTest (or other tools) we don't get that type of variations, we just use hard coded values for a specific connection. And that's good, because it makes it possible for us to measure improvements so that when we do fixes, we can easily see if it makes it better or worse. Picking a slow connection like 2G is good because then it is easier to spot what impact we get from our changes. I like automated testing with real devices, we have automatic tests up and running with a Iphone 6, Ipad Mini and Motorola G (using the ones provided in WebPageTest) and it looks like this: https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest?panelId=28&am… You can click on the each row in the table to only show one graph, that makes it easier to see what numbers we have. Click on Dulles_iPhone6_iPhone_6_iOS_9.render and you can see that the start rendering time (when something is first showed on the screen) is pretty stable for the Iphone 6 but if you choose Dulles_iPhone6_iPhone_6_iOS_9.SpeedIndex (SpeedIndex is a way of calculate when content within the viewport is visible for the end user) vary a lot. The thing is that we only have limited access to automated tests with real phones (we don't have anything setup for ourselves). So we use it now to just get a feeling of what it looks like on real devices. Most IMPORTANT: When we measure performance it is important that we always do it exactly the same way, so we try to limit the changes to as few as possible (=our code change). Using WebPageTest I think it's ok to use Mobile 2G Fast, emulating mobile using Chrome. When we run the automated tests in Jenkins we test each URL 5 times, it seems to be pretty stable. Our tests will not give us the same numbers as the users gets, but it will give us constant numbers so that we can see that the changes we do impact the site. When we push these changes to production we can hopefully pick them up with our RUM metrics. One last thing: We will not pickup things as CPU usage faking real devices but to do that, I think we need to look at another tool than WebPageTest to do that correctly. Do we have one already? On Mon, Jan 18, 2016 at 1:20 PM, Federico Leva (Nemo) &lt;nemowiki(a)gmail.com&gt; wrote:

https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest?panelId=28&am… yes sorry I forgot to change that, now it is easier to understand. On Wed, Jan 20, 2016 at 5:09 PM, Joaquin Oltra Hernandez < jhernandez(a)wikimedia.org&gt; wrote: > Thanks for the responses Federico & Peter. > Federico, I fully agree that a video is not a replacement for data, but > they serve different purposes. Peter expanded on it better than I what I > could do, but basically data driven tests serve a great deal for > development purposes to assessing impact of changes since they give us a > controlled environment to see the impact of changes trying to improve > performance. > The reality is a lot messier and hard to measure, and I find it greatly > useful to see real experiences on real devices, where the network > conditions vary a lot, and there are multiple applications/services > competing for CPU and bandwidth on the device, making the experience > completely different to what it is on the numbers that the performance > tests provide. > Surely it is anecdotal data, but it is an anecdotal experience that a lot > of people probably experience, so I end up caring about it. > --- > Thanks Peter for the links and the thoughtful response. Great points and > advice. I didn't know about that dashboard, it is great! > What are the units on the Y axis here for example? > https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest?panelId=28&am… > One last thing: We will not pickup things as CPU usage faking real devices > > but to do that, I think we need to look at another tool than WebPageTest > to > > do that correctly. Do we have one already? > Not that I know of, we were thinking about doing some searching to see what > we could find. It would be a great addition to have. > Joaquin > On Wed, Jan 20, 2016 at 10:28 AM, Peter Hedenskog &lt;peter(a)wikimedia.org> wrote: > > I like videos because it's so easy for non technical people to understand > > the value of performance. For me it has been a really good way to show > and > > explain performance changes and what it looks like for the user. > > > Picking a representative connection is hard and the good thing is I think > > we don't need to. Connections in the real world vary so much: You could > > have 3G but a really bad connection to the nearest station because > there's > > a lot of other phones connected etc. We have tests running with real > > connections 2G/3G from Bangalore and LTE in San Francisco, you can check > it > > out how it looks here: > > > https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest?panelId=30&am… > > I really like that graph because it shows how much variances you can have > > on the same connection type from the same location. Check out how it > looks > > like in Bangalore. > > > When we throttle the connection for WebPageTest (or other tools) we don't > > get that type of variations, we just use hard coded values for a specific > > connection. And that's good, because it makes it possible for us to > measure > > improvements so that when we do fixes, we can easily see if it makes it > > better or worse. Picking a slow connection like 2G is good because then > it > > is easier to spot what impact we get from our changes. > > > I like automated testing with real devices, we have automatic tests up > and > > running with a Iphone 6, Ipad Mini and Motorola G (using the ones > provided > > in WebPageTest) and it looks like this: > > > https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest?panelId=28&am… > > > You can click on the each row in the table to only show one graph, that > > makes it easier to see what numbers we have. Click > > on Dulles_iPhone6_iPhone_6_iOS_9.render and you can see that the start > > rendering time (when something is first showed on the screen) is pretty > > stable for the Iphone 6 but if you > > choose Dulles_iPhone6_iPhone_6_iOS_9.SpeedIndex (SpeedIndex is a way of > > calculate when content within the viewport is visible for the end user) > > vary a lot. The thing is that we only have limited access to automated > > tests with real phones (we don't have anything setup for ourselves). So > we > > use it now to just get a feeling of what it looks like on real devices. > > > Most IMPORTANT: When we measure performance it is important that we > always > > do it exactly the same way, so we try to limit the changes to as few as > > possible (=our code change). Using WebPageTest I think it's ok to use > > Mobile 2G Fast, emulating mobile using Chrome. When we run the automated > > tests in Jenkins we test each URL 5 times, it seems to be pretty stable. > > Our tests will not give us the same numbers as the users gets, but it > will > > give us constant numbers so that we can see that the changes we do impact > > the site. When we push these changes to production we can hopefully pick > > them up with our RUM metrics. > > > One last thing: We will not pickup things as CPU usage faking real > devices > > but to do that, I think we need to look at another tool than WebPageTest > to > > do that correctly. Do we have one already? > > > On Mon, Jan 18, 2016 at 1:20 PM, Federico Leva (Nemo) < > nemowiki(a)gmail.com> > wrote: > > > > I'm allergic to videos as replacement of data, but does your video > > > substantially differ from what tests give us, e.g. > > > http://www.webpagetest.org/video/view.php?id=160118_YK_HKT.1.0 ? > > > > > It's easy to test multiple speeds (some tests still pending): > > > > http://www.webpagetest.org/video/compare.php?tests=160118_AZ_JH3,160118_VB_… > > > > > Some of the tests show significant CPU usage after the "everything and > a > > > kitchen sink" ResourceLoader phase in which some stuff for > CentralNotice, > > > Gather etc. etc. is loaded. > > > > > Mainly, the issue is to pick representative connections. There is some > > > official data, some of which based on actual tests by probes, although > > it's > > > limited: > > > * > > > > https://ec.europa.eu/digital-agenda/en/news/quality-broadband-services-eu > > > (no mobile) > > > * > > > > https://ec.europa.eu/digital-agenda/en/news/use-commercial-mobile-networks-… > > > (needs some parsing!) > > > > > Nemo > > > > > > > _______________________________________________ > > > Wikitech-l mailing list > > > Wikitech-l(a)lists.wikimedia.org > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > _______________________________________________ > > Wikitech-l mailing list > > Wikitech-l(a)lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l