Hello,
As you know we had to rollback 1.27-wmf.11 a few times in the last two weeks due to issues in the new SessionManager code base. The last time was this weekend (Saturday) and thus right now (Monday Feb 1st and 18:00 UTC) Wikimedia production is running only[0] wmf.10.
After discussions with Brad (Anomie), Bryan (bd808), Dan (marxarelli), Antoine (hashar), and myself we decided on this plan moving forward:
1) Stay at wmf.10 everywhere today/Mon Feb 1st
2) Brad/Bryan prepare master *without* SessionManager
3) Brad to prepare session invalidation improvement (speed up the invalidation process)
4) Tomorrow/Tuesday Feb 2nd: cut wmf.12 (without SessionManager)
5) Roll wmf.12 as normal this week (ie: skip wmf.11 entirely)
6) Rest of week: Brad and Bryan to get some more end-to-end tests running with Dan
7) Next week: Hopefully get SessionManager back into wmf.13 (with session invalidation improvement incorporated)
Apologies to all for the issues we've been having and apologies to Bryan, Brad, and Gergo for delaying the rollout of SessionManager (and probably AuthManager). I think this is the best solution we have right now given the situation.
Best,
Greg
[0] If you care about relatively tiny details, the mw1017 appserver is running wmf.11, but the only users that get that server set a special debug header. This is for the team to test/debug what they can.