On 16/04/11 03:07, Brion Vibber wrote:
In the meantime, we should be running 1.18 on live
servers, with a maximum
of a week lag from trunk, and preferably much less. Ongoing work on trunk
should always be keeping stability in mind, and code review should
concentrate on ensuring that code is being actively tested and used.
Yeah, I've heard this before. It didn't work the first time around,
and I don't think it can work now. We can't use Wikipedia as a testing
site for alpha-quality code anymore.
I think we should have a cycle of:
* Development branch merges and other major work in trunk.
* Review and stabilisation of the course of a couple of months,
alongside general development work.
* Branch point.
* A period of backports and review to ensure the stability of the new
branch.
* Testing, for 1-2 weeks.
* Deployment.
This is what we did for 1.17, and it worked well, leading to a 1.17
deployment which caused a minimum of disruption.
Unexercised code is dangerous code that will break
when you least expect it;
we need to get code into use fast, where it won't sit idle until we push it
live with a thousand other things we've forgotten about.
This certainly wasn't my experience with the 1.17 deployment. We had a
great deal of review and testing of the 1.17 branch, and many bugs
were fixed without having to get Wikipedians to tell us about them.
translatewiki.net is a great help, but don't
forget that it doesn't run all
the same extensions as are used in Wikimedia production sites.
No it doesn't, that's why we set up public test wikis which did have a
similar set of extensions: first a set of wikis separate from the main
cluster on
prototype.wikimedia.org, and then a test wiki which was
part of the cluster. Then we did a staged deployment, deploying 1.17
to several wikis at a time.
CT and Robla were very supportive of this deployment strategy, and
setting up permanent systems for deploying different versions to
different wikis is now a high priority project.
We had a significant amount of manpower dedicated to testing the
software on
prototype.wikimedia.org, both Wikimedia staff and experts
contracted via Calcey QA.
It's not the same site as it was when you first proposed this policy.
-- Tim Starling