[Labs-admin] Labsdb migration (was: Report from ops meeting)

[Jaime Crespo]

As an admin, everything you should know about the upcoming labsdb1005
reimage:

* We are ready (DBAs) to do this at any time, we just need to tell users in
advance of potential outages/degradations of service
* For 99% of the users, we will just switchover them transparently to the
slave (should not cause issues). As usual, if their application does not
retry to reconnect, there will be problems.
* For 3 users (databases), there will be full outage because they have such
a heavy usage that we cannot replicate them in real time. They were made
aware of this limitation months ago, so it should not come as a surprise:
https://phabricator.wikimedia.org/T127164 The users's databases are
documented at:
https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/templates/mariadb/tools.my.cnf.erb;f21ce599fe626e7c96010a5d0335370ebe510ca7$65
* Data will be copied away, server will be reimaged, then data will be
copied back That normally takes 3 hours, but things could go wrong...
* People could complain for a 10.0 upgrade (?). But some people actually
complained already for the lack of 5.5 -> 10 upgrade.
https://phabricator.wikimedia.org/T138517#2796682
* On switch-back, again bad-programmed application may temporarily fail,
but good ones should just switch transparently; unavailable dbs should be
available again

That should be enough background to schedule and send an email to users :-)

---------- Forwarded message ----------
From: Yuvi Panda <yuvipanda at gmail.com>
Date: Mon, Jan 30, 2017 at 7:14 PM
Subject: [Labs-admin] Report from ops meeting
To: Labs admin list for infrastructure and discussion <
labs-admin at lists.wikimedia.org>


1. Faidon talking about ip space discussions wrt asia dc discussion,
and mentioned we might / should renumber labs IP space. Not sure about
more details.
2. Ping on labsdb migration to Jessie
3. Mid-year review of annual goals coming up, need status about OGE
migration

That's it.

--
Yuvi Panda T
http://yuvi.in/blog

_______________________________________________
Labs-admin mailing list
Labs-admin at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-admin



-- 
Jaime Crespo
<http://wikimedia.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-admin/attachments/20170130/206ee1cb/attachment.html>