Hello all,
This is a quick email to give you an update on the DevOps Sprint[0].
== What we focused on ==
As you can see on the project page, there were three main areas of (attempted) focus: Monitoring, Cache improvements, and Deployment. That may sound like a lot, and it was. It was too much, in fact. There were also items under those categories that weren't as well defined as they should have been (see below).
== What we did ==
* We (Tim and Brad) did fix a couple long standing annoying cache improvement bugs ([1] and [2]).
* We (Ori) improved scap a bit ** IRC logging of commits ** Reporting time to graphite - This is useful in that it gives us a real baseline to judge any future improvements against. ** Made it more atomic by having rsync defer updating of files until they've all been transmitted to the server, thus minimizing the amount of time code is in an inconsistent state ** Made rsync use --compress which helped in over scap time, but we don't know exactly how much (the 'time reporting' thing above happened after)
* We (Aaron, Antoine, Bryan, Ori) setup Logstash ** This is still ongoing, but it is close to being setup in production. ** There is also a labs instance[3] you can look at that uses log info from the Beta Cluster.
* We (Ori, Bryan, Aaron) wrote an RFC for adding structured logging to MediaWiki [4]
== Things we didn't get to ==
Much of the deployments category. This is mostly due to the above mentioned issues (too much, too broad).
Specifically: * Trebuchet (aka: git-deploy) ** Ryan put a lot of work into Trebuchet during this sprint which brought it to a great position ** We (Platform) weren't able to devote the requisite time to it during the high level of churn (which was good). ** We (Platform) hope to work on this in the coming months from the deployer's experience end (ie: not the backend of transferring files around, specifically): more on this later[5].
Best,
Greg and the DevOps Sprint team.
[0] https://www.mediawiki.org/wiki/DevOps_Sprint_2013 [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=5382 [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=27935 [3] http://logstash.instance-proxy.wmflabs.org/#/dashboard user/password: https://office.wikimedia.org/wiki/User:BDavis_%28WMF%29/logstash [4] https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging [5] Basically, there will be a documentation sprint to produce a overview of the current dev and deploy process followed by a "where we want to go" high level thing (text or flowchart, something). From that we can create a pretty solid design doc for deployment 2.0.