On Fri, Mar 7, 2014 at 10:30 AM, Jon Robson jdlrobson@gmail.com wrote:
Let's also take this into a new thread. There are a lot of different conversations now going on....
On Fri, Mar 7, 2014 at 9:21 AM, Brad Jorsch (Anomie) bjorsch@wikimedia.org wrote:
On Fri, Mar 7, 2014 at 12:08 PM, C. Scott Ananian cananian@wikimedia.orgwrote:
I agree. I think a better technical solution would be to halt jenkins' auto-merge for the 24 hour period, so that +2'ed changes are not automatically merged until after the branch is cut.
I don't see how that's any better. Things still aren't getting merged.
If anything, the "cut using master@{24 hours ago}" is a much better idea.[1] Although it might be useful to see if Wednesday tends to be a relatively active bug-fixing day as the community on non-Wikipedia sites finds issues in the version that was deployed to them on Tuesday, in which case keeping those from making it into the new cut on Thursday (and so requiring more backports or waiting an extra week for fixes) might not be so great.
[1]: And yes, 'master@{24 hours ago}' is valid git syntax.
In another project in another place with another team... I solved a very similar problem by creating release branches from tags that the testing system placed on revisions that had passed the integration test suite rather than whatever HEAD happened to be at the time that the branch was cut. We had a very heavy (~2hr wall clock time; ~24hr cpu time) integration test suite that ran once a day. When the master job that kicked off these tests found that no child tests had failed it would tag the revisions that had been tested across all of the involved repositories with something like 'integration-YYYYMMDD'. Our release branch was forked from these tags. In the MW situation it might be nicer to run such test processes more often that once a day but the fundamental idea could/should work.
This type of gated promotion process won't stop all problems from getting to the next stage, but it might make the inclusion of code into the weekly branch slightly safer. It also could be seen as stepping stone to further automation that could decrease the time between production minor version deploys. Someday we might be able to deploy "continuously" to some set of wikis where continuously doesn't mean every individual commit but every time the integration test suite says that things are stable.
Bryan