On Thu, Aug 30, 2018 at 12:55 PM Derk-Jan Hartman
While I think these regular switches are a very good idea, from an outside
perspective I do have to question a process that puts a significant plug in
the velocity of various teams working on major projects (esp. in a time of
year that could probably be seen as one of the most productive).
That is absolutely true.
plans to reduce the disruption of this exercise in the future ?
There are indeed plans about making this easier, more automated and
shorter in duration. Just to name a few, mediawiki now no longer
requires deployments to switch datacenters but rather relies on a
state key in the etcd database, there is a library + cookbooks to
automate the currently automateable steps, there is work to update the
documentation and make it update to date, accurate and more usable by
more people and at shorter notices and so on. That being said, it's
never gonna be free, but the toil is significant to make us want to
reduce and that what we aim for.
On Thu, Aug 30, 2018 at 8:38 AM Jaime Crespo <jcrespo(a)wikimedia.org> wrote:
Let me explain the rationale of the bellow
request for clarification:
On Wed, Aug 29, 2018 at 11:30 PM MA <strigiwm(a)gmail.com> wrote:
>For the duration of the switchover (1 month), deployers are kindly
>requested to refrain from large db schema changes and avoid deploying
>any kind of new feature that requires creation of tables.
>There will be a train freeze in the week of Sept 10th and Oct 8th.
During the failover, some schema changes will be finalized on the current
active datacenter (plus some major server and network maintenance may be
done)- our request is mostly to refrain from quickly enabling those large
new unlocked features (e.g. the ongoing comment refactoring, actor/user
refactoring, Multi Content Revision, JADE, major wikidata or structured
comons structure changes, new extensions not ever deployed to the cluster,
etc.) at the same time than the ongoing maintenance to reduce variables of
things that can go bad- enabling those features may be unblocked during the
switchover time, but we ask you to hold until being back on the current
active datacenter. Basically, ask yourself if you are enabling a large new
core feature or want to start a heavy-write maintenance script and there is
a chance you will need DBA/system support. Sadly, we had some instances of
this happening last year and we want to explicitly discourage this during
these 2 weeks.
In own my opinion, enabling existing features on smaller projects (size
here is in amount of server resources, not that they are less important) is
equivalent to a swat change, and I am not against it happening. I would ask
contributors to use their best judgement on every case, and ask people on
the #DBA tag on phabricator or add me as reviewers on gerrit if in doubt.
My plea is to not enable major structural changes during that time may
affect thousands of edits per minute. Swat-like changes and "boring" :-)
trains are ok.
For new wiki creations I would prefer if those were delayed but CC #DBA s
on the phabricator task to check with us.
Wikitech-l mailing list
Wikitech-l mailing list
Alexandros Kosiaris <akosiaris(a)wikimedia.org>