<quote name="Leslie Carr" date="2013-05-07" time="11:43:47 -0700">
I think this is awesome for features ... but if we're putting work into this, I would love even more to have a clustered a+b production environment, such that 10% of folks are put on the new release (cluster a) and then it gets pushed over to cluster b. Then we can also test performance in a real world environment, and breakages only happen for 10% (PS the 10% number was pulled out of thin air).
I really like this idea, I think. So bringing various concurrent (email) threads together; long term we could have something that looks like this (roughly):
1) Change proposed - Jenkins runs tests on a throw away labs instance
2) Change merged to master - Jenkins/etc runs tests on betalabs
3) New wmfXX released to 10% of cluster - "10%" being something like: test, test2, mediawiki.org, and some of the non-'pedia project sites - Our users do the testing ;-)
4) That wmfXX goes to the rest of the cluster - Hopefully all is good.
Is that kind of what you had in mind, Leslie?
Greg