[Wikitech-l] Update from the Wikimedia Performance Team

Hello, This is the monthly report from the Wikimedia Performance team. ## Our progress ## * Availability. We've done a major overhaul of the ObjectCache interfaces. Many factory methods were deprecated or removed, reducing it to just four simple entry points. New docs at https://doc.wikimedia.org/mediawiki-core/master/php/classObjectCache.html#d… We've written a new IExpiringStore interface for convenient TTL constants, e.g. $cache::TTL_WEEK. See https://doc.wikimedia.org/mediawiki-core/master/php/interfaceIExpiringStore… We've migrated most use of wfGetMainCache() to WANObjectCache. Work continued on the librarization of BagOStuff, Memcached, and other object cache classes. * Performance testing infrastructure. We've created dedicated dashboards for portals: https://grafana.wikimedia.org/dashboard/db/webpagetest-portals And for mobile: https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest We now test one page using real 3G connections (from San Francisco and Bangalore) and test other pages using the following physical devices: iPhone 6, iPad mini 2 and Moto G. * Media stack. We've extended Thumbor with 12 small plugins to meet our needs and match our existing thumbnailing feature set. This includes support for all the file formats in use on Commons. The Thumbor Vagrant stack is now very close to having all the moving parts needed in production, with basic Vagrant roles for Varnish and Swift having been written to that end. Our objective is to finish the work on VM by the holidays and have it ready to be showcased and discussed collectively at the developer summit in a breakout session. * ResourceLoader. We've written a new mw.requestIdleCallback API for scheduling deferred tasks. We've removed usage of the msg_resource_links DB table. We now use message config from the module registry directly. We've migrated MessageBlobStore msg_resource DB table to an object cache (to be deployed in January 2016): https://phabricator.wikimedia.org/T113092 ## How are we doing? ## Client-side performance has remained stable over the past month. Save Timing has also remained stable, around the 1s median mark. The job queue's health improved greatly after adding a new server to the pool, with the job queue size dropping drastically and the 99th percentile job processing time going from one day to one hour: * https://grafana.wikimedia.org/dashboard/db/job-queue-health There was a small scare about a sudden increase of the SpeedIndex value across the board: https://grafana.wikimedia.org/dashboard/db/webpagetest But it was entirely explained by the fundraising banner, which doesn't appear immediately on pageload. SpeedIndex measures the time it takes for the above-the-fold area to "settle" visually. The banner appears late and pushes the content down, which delays the time when visual changes stop happening for the above-the-fold area. Until next time, Aaron, Gilles, Peter, Timo, and Ori.