[Wikimedia-l] Fwd: [Wikimania-l] git.wikimedia.org dead due to wikimania ; )

[Oliver Keyes]

We have weekend support - for core services, which constitute our MediaWiki
instances. Git, however, is not a core service - as Max accurately notes,
while it makes development finicky and frustrating, it doesn't impact on
our core mission, which is to provide the sum of human knowledge, for
everybody. It being down for a short period of time is not critical -
Wikipedia being down? Well, I think we'd agree that's a pretty serious
problem.

I'd like to think Engineering do a pretty good job at uptime for core
services - when was the last time you saw Wikipedia down for any extended
period of time? - regardless of what day of the week it is. I recall a few
months ago that we had weird, inconsistent cacheing problems that were
causing trouble for users - Leslie Carr, one of our Ops engineers, worked
11 hours on Friday and (iirc) 14 hours over the weekend to get it fixed.
This isn't the only story in that vein - where there are critical problems
to critical services, they tend to be fixed as promptly as possible, and
staffers pull out all of the stops to make it happen. But it's not
realistic for ops and dev/ops to maintain 24/7 uptime not just for our core
services, but for every single one of the miscellaneous instances we host.
If we had 150 engineers in Platform and Operations, then I might be singing
a different tune, but unfortunately we don't ;).

On the other hand, if you would like to donate enough money to pay for a
team of 6 to maintain our git instance 24/7, I'm sure we could set
something up :P


On 12 August 2013 20:19, Richard Farmbrough <richard at farmbrough.co.uk>wrote:

> I maintained 24/7 support with a team of 6. WMF has 150 staff and does not
> have weekend support.  The tail is wagging the dog.
>
>
>
> ______________________________**_________________
> Wikimedia-l mailing list
> Wikimedia-l at lists.wikimedia.**org <Wikimedia-l at lists.wikimedia.org>
> Unsubscribe: https://lists.wikimedia.org/**mailman/listinfo/wikimedia-l<https://lists.wikimedia.org/mailman/listinfo/wikimedia-l>,
> <mailto:wikimedia-l-request@**lists.wikimedia.org<wikimedia-l-request at lists.wikimedia.org>
> ?subject=**unsubscribe>
>



-- 
Oliver Keyes
Community Liaison, Product Development
Wikimedia Foundation