One clarification that has to made is that toolsdb and the new servers have both drive redundancy (RAID10) and host redundancy (3 servers serving 2 services with proxies in the middle for replicas; 2 servers for 1 service for toolsdb), that is one of the many reasons why we encourage the usage of the new servers and not the fragile old ones.
On Fri, Nov 3, 2017 at 2:13 AM, Bryan Davis bd808@wikimedia.org wrote:
On Thu, Nov 2, 2017 at 6:13 PM, Maximilian Doerr maximilian.doerr@gmail.com wrote:
Can you provide a list of tools/users impacted by the drive failure? Or
is there a redundant drive covering for this?
As long as c1 stays up, https://tools.wmflabs.org/tool-db-usage/ will show the users with user-owned databases there. These users should have all also received a MassMessage spam from me on their Wikitech talk page about a week ago.
There is no drive or data redundancy for user-created tables on c1.labsdb or c3.labsdb. The tools.db.svc.eqiad.wmflabs databases however are replicated to a secondary server. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/ Database#ToolsDB_Backups_and_Replication
Bryan
Bryan Davis Wikimedia Foundation bd808@wikimedia.org [[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA irc: bd808 v:415.839.6885 x6855
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud