New subject: [Wikitech-l] Update from the Wikimedia Performance Team

8 Dec 2015


      Hello,
This is the monthly report from the Wikimedia Performance team.
## Our progress ##
* Availability. We've done a major overhaul of the ObjectCache interfaces. Many
factory methods were deprecated or removed, reducing it to just four simple
entry points. New docs at
https://doc.wikimedia.org/mediawiki-core/master/php/classObjectCache.html#de...
We've written a new IExpiringStore interface for convenient TTL constants,
e.g. $cache::TTL_WEEK. See
https://doc.wikimedia.org/mediawiki-core/master/php/interfaceIExpiringStore....
We've migrated most use of wfGetMainCache() to WANObjectCache. Work
continued on the librarization of BagOStuff, Memcached, and other object
cache classes.
* Performance testing infrastructure. We've created dedicated dashboards
for portals:
https://grafana.wikimedia.org/dashboard/db/webpagetest-portals
And for mobile:
https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest
We now test one page using real 3G connections (from San Francisco and
Bangalore) and test other pages using the following physical devices:
iPhone 6, iPad mini 2 and Moto G.
* Media stack. We've extended Thumbor with 12 small plugins to meet our
needs and match our existing thumbnailing feature set. This includes
support for all the file formats in use on Commons. The Thumbor Vagrant
stack is now very close to having all the moving parts needed in
production, with basic Vagrant roles for Varnish and Swift having been
written to that end. Our objective is to finish the work on VM by the
holidays and have it ready to be showcased and discussed collectively at
the developer summit in a breakout session.
* ResourceLoader. We've written a new mw.requestIdleCallback API for
scheduling deferred tasks. We've removed usage of the  msg_resource_links
DB table. We now use message config from the module registry directly.
We've migrated MessageBlobStore msg_resource DB table to an object cache
(to be deployed in January 2016): https://phabricator.wikimedia.org/T113092
## How are we doing? ##
Client-side performance has remained stable over the past month. Save Timing
has also remained stable, around the 1s median mark.
The job queue's health improved greatly after adding a new server to the
pool, with the job queue size dropping drastically and the 99th percentile
job processing time going from one day to one hour:
* https://grafana.wikimedia.org/dashboard/db/job-queue-health
There was a small scare about a sudden increase of the SpeedIndex value
across the board:
https://grafana.wikimedia.org/dashboard/db/webpagetest
But it was entirely explained by the fundraising banner, which doesn't
appear immediately on pageload. SpeedIndex measures the time it takes for
the above-the-fold area to "settle" visually. The banner appears late and
pushes the content down, which delays the time when visual changes stop
happening for the above-the-fold area.
Until next time,
Aaron, Gilles, Peter, Timo, and Ori.