On Wed, Nov 1, 2017 at 9:42 AM, Bryan Davis <bd808(a)wikimedia.org> wrote:
* c1.labsdb (labsdb1001.eqiad.wmnet) is down due to hardware issues
* *.labsdb are pointing to c3.labsdb (labsdb1003.eqiad.wmnet)
Manuel (one of our awesome DBAs) was able to get the MySQL server for
c1.labsdb back up and running in read-only mode.
We do not know how much longer the failing SSD drive will survive, but
until it fails again, users who have user created databases on this
host can attempt to connect and archive them. If the data you have
stored there is something that you can recreate, you should probably
do that instead.
Please move non-reproducible data to tools.db.svc.eqiad.wmflabs.
c3.labsdb (labsdb1003.eqiad.wmnet) will be shutdown on Wednesday 13
December 2017, so moving any data to c3 will gain you less than 6
weeks. You should also be working to update your tools that need to
write data to a database to use tools.db.svc.eqiad.wmflabs. See
for more details.
Due to the failure of labsdb1001.eqiad.wmnet, the planned reboot of
labsdb1003.eqiad.wmnet on Tuesday 07 November 2017 has been cancelled.
Bryan (on behalf of the Cloud Services and DBA teams)
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA
irc: bd808 v:415.839.6885 x6855