+1 to DJ's question about timing. Also, one might wish to be mindful of the number of
recent trains that were supposed to be boring but involved interesting surprises; this
makes me wonder whether trains that one thinks will be boring are actually OK in this
circumstance even if they turn out to be "interesting".
-------- Original message --------From: Derk-Jan Hartman
<d.j.hartman+wmf_ml(a)gmail.com> Date: 8/30/18 2:54 AM (GMT-08:00) To: Wikimedia
developers <wikitech-l(a)lists.wikimedia.org> Subject: Re: [Wikitech-l] Datacenter
switchover and switchback
While I think these regular switches are a very good idea, from an outside
perspective I do have to question a process that puts a significant plug in
the velocity of various teams working on major projects (esp. in a time of
year that could probably be seen as one of the most productive). What are
plans to reduce the disruption of this exercise in the future ?
On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo <jcrespo(a)wikimedia.org> wrote:
Let me explain the rationale of the bellow request for
On Wed, Aug 29, 2018 at 11:30 PM MA <strigiwm(a)gmail.com> wrote:
>For the duration of the switchover (1 month), deployers are kindly
>requested to refrain from large db schema changes and avoid deploying
>any kind of new feature that requires creation of tables.
>There will be a train freeze in the week of Sept 10th and Oct 8th.
During the failover, some schema changes will be finalized on the current
active datacenter (plus some major server and network maintenance may be
done)- our request is mostly to refrain from quickly enabling those large
new unlocked features (e.g. the ongoing comment refactoring, actor/user
refactoring, Multi Content Revision, JADE, major wikidata or structured
comons structure changes, new extensions not ever deployed to the cluster,
etc.) at the same time than the ongoing maintenance to reduce variables of
things that can go bad- enabling those features may be unblocked during the
switchover time, but we ask you to hold until being back on the current
active datacenter. Basically, ask yourself if you are enabling a large new
core feature or want to start a heavy-write maintenance script and there is
a chance you will need DBA/system support. Sadly, we had some instances of
this happening last year and we want to explicitly discourage this during
these 2 weeks.
In own my opinion, enabling existing features on smaller projects (size
here is in amount of server resources, not that they are less important) is
equivalent to a swat change, and I am not against it happening. I would ask
contributors to use their best judgement on every case, and ask people on
the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt.
My plea is to not enable major structural changes during that time may
affect thousands of edits per minute. Swat-like changes and "boring" :-)
trains are ok.
For new wiki creations I would prefer if those were delayed but CC #DBA s
on the phabricator task to check with us.
Wikitech-l mailing list
Wikitech-l mailing list