<quote name="C. Scott Ananian" date="2015-05-29" time="13:38:49 -0400">
I'll echo those nervous about the faster pace for deploys -- although not so nervous as to dig my feet in and yell stop. Mostly my concerns boil down to the fact that the beta environment isn't really a good test for anything other than absolute crashers. Hardly anyone uses the Collection extension (for example) on beta, for example.
Sounds like you need more manual testing then? Who is tasked with doing that for your team?
I'd *really* like to see some effort put into improving beta. In particular, running beta with an up-to-date (but sanitized, perhaps) mirror of the main WP databases would ensure that we have a decent amount of test cases on beta. We recently spent a couple of hours trying to test Parsoid on beta, only to find out that the behavior we were chasing down was caused by the fact that beta's copy of the IPA formatting templates on enwiki (!) were out-of-date and incomplete. We really need to do better than [[English language]] as far as articles to test on enbeta.
Yeah, that'd be nice.
See related tasks: https://phabricator.wikimedia.org/T51779 https://phabricator.wikimedia.org/T54382
Greg