This last line in the conversation strikes me as dry, useless and submissive. We should not ever "neglect" a sister project (including wikimedia projects, phabricator, wikitech, tools, etc), as small as their userbase might seem. For many, weekends are the volunteering or coding time and if the website they use for it was off, such people would be frustrated. Would be interesting to see how to set up multi-server instance of FAB. http://blog.iweb.com/en/2012/02/how-to-distribute-website-load-across-multi… What I don't understand is how to decentralise the database. Happy Thanksgiving to all, of course... -- svetlana On Sun, 30 Nov 2014, at 20:13, Gerard Meijssen wrote:

Hoi, The argument about "non-working" hours is problematic. When the only thing that counts are the working hours of staff in the USA you may be right. As it is, WIkimedia Germany has staff working at other times and they are affected. Affected are the non-professionals as well.. My advise, do not go there. It is broken and it needs fixing. Thanks, GerardM On 30 November 2014 at 19:02, Brian Wolff <bawolff(a)gmail.com> wrote:

Hi, let me recycle this reply posted initially at "Determine phabricator.wikimedia.org service level" - https://phabricator.wikimedia.org/T76381 Currently Phabricator is getting the same service level that Bugzilla had. Looking at the whole Wikimedia picture, I think this is the most sensible option. I don't see any strong reason to change it. Bugzilla was down unexpectedly several times in the past years, and if Ops was able to react quicker it's just because we were luckier with the cause, timing and location of the breaks. If we would have Bugzilla instead of Phabricator in the rack that went down this weekend, the service provided by Ops would have been exactly the same. We can reopen this discussion when planning the migration of code review and (eventually) continuous integration. For now, I think we are good. This is the opinion of the Engineering Community team. If this works also for Operations and Platform Engineering, then we can resolve this task. PS: About the downtime itself, 5 hours on a weekend is clearly unfortunate, but imho nothing that should make us revise the current service level. Was anybody unable to work, arms crossed? Was any project delayed? I'm counting volunteers as much as employees. Personally I learned about the downtime only in wikitech-l, having used Phabricator on Saturday-Sunday night at 1am CET, and then on Sunday at 1pm. -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil