+1 to DJ's question about timing. Also, one might wish to be mindful of the number of recent trains that were supposed to be boring but involved interesting surprises; this makes me wonder whether trains that one thinks will be boring are actually OK in this circumstance even if they turn out to be "interesting".
Pine ( https://meta.wikimedia.org/wiki/User:Pine )
-------- Original message --------From: Derk-Jan Hartman d.j.hartman+wmf_ml@gmail.com Date: 8/30/18 2:54 AM (GMT-08:00) To: Wikimedia developers wikitech-l@lists.wikimedia.org Subject: Re: [Wikitech-l] Datacenter switchover and switchback While I think these regular switches are a very good idea, from an outside perspective I do have to question a process that puts a significant plug in the velocity of various teams working on major projects (esp. in a time of year that could probably be seen as one of the most productive). What are plans to reduce the disruption of this exercise in the future ?
DJ
On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo jcrespo@wikimedia.org wrote:
Let me explain the rationale of the bellow request for clarification:
On Wed, Aug 29, 2018 at 11:30 PM MA strigiwm@gmail.com wrote:
Hello:
For the duration of the switchover (1 month), deployers are kindly requested to refrain from large db schema changes and avoid deploying any kind of new feature that requires creation of tables. There will be a train freeze in the week of Sept 10th and Oct 8th.
During the failover, some schema changes will be finalized on the current active datacenter (plus some major server and network maintenance may be done)- our request is mostly to refrain from quickly enabling those large new unlocked features (e.g. the ongoing comment refactoring, actor/user refactoring, Multi Content Revision, JADE, major wikidata or structured comons structure changes, new extensions not ever deployed to the cluster, etc.) at the same time than the ongoing maintenance to reduce variables of things that can go bad- enabling those features may be unblocked during the switchover time, but we ask you to hold until being back on the current active datacenter. Basically, ask yourself if you are enabling a large new core feature or want to start a heavy-write maintenance script and there is a chance you will need DBA/system support. Sadly, we had some instances of this happening last year and we want to explicitly discourage this during these 2 weeks.
In own my opinion, enabling existing features on smaller projects (size here is in amount of server resources, not that they are less important) is equivalent to a swat change, and I am not against it happening. I would ask contributors to use their best judgement on every case, and ask people on the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt. My plea is to not enable major structural changes during that time may affect thousands of edits per minute. Swat-like changes and "boring" :-) trains are ok.
For new wiki creations I would prefer if those were delayed but CC #DBA s on the phabricator task to check with us. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l