On 16/04/11 03:07, Brion Vibber wrote:
In the meantime, we should be running 1.18 on live servers, with a maximum of a week lag from trunk, and preferably much less. Ongoing work on trunk should always be keeping stability in mind, and code review should concentrate on ensuring that code is being actively tested and used.
Yeah, I've heard this before. It didn't work the first time around, and I don't think it can work now. We can't use Wikipedia as a testing site for alpha-quality code anymore.
I think we should have a cycle of:
* Development branch merges and other major work in trunk. * Review and stabilisation of the course of a couple of months, alongside general development work. * Branch point. * A period of backports and review to ensure the stability of the new branch. * Testing, for 1-2 weeks. * Deployment.
This is what we did for 1.17, and it worked well, leading to a 1.17 deployment which caused a minimum of disruption.
Unexercised code is dangerous code that will break when you least expect it; we need to get code into use fast, where it won't sit idle until we push it live with a thousand other things we've forgotten about.
This certainly wasn't my experience with the 1.17 deployment. We had a great deal of review and testing of the 1.17 branch, and many bugs were fixed without having to get Wikipedians to tell us about them.
translatewiki.net is a great help, but don't forget that it doesn't run all the same extensions as are used in Wikimedia production sites.
No it doesn't, that's why we set up public test wikis which did have a similar set of extensions: first a set of wikis separate from the main cluster on prototype.wikimedia.org, and then a test wiki which was part of the cluster. Then we did a staged deployment, deploying 1.17 to several wikis at a time.
CT and Robla were very supportive of this deployment strategy, and setting up permanent systems for deploying different versions to different wikis is now a high priority project.
We had a significant amount of manpower dedicated to testing the software on prototype.wikimedia.org, both Wikimedia staff and experts contracted via Calcey QA.
It's not the same site as it was when you first proposed this policy.
-- Tim Starling