<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Ryan,<br>
<br>
Op 6-9-2013 0:57, Ryan Lane schreef:<br>
</div>
<blockquote
cite="mid:CAC9E6crtjn_fztpO7sBDfGk7ufnNF02km7Y1zWZ2=r=x=pbzYQ@mail.gmail.com"
type="cite">
<div dir="ltr">On Fri, Sep 6, 2013 at 6:27 AM, Platonides <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:platonides@gmail.com" target="_blank">platonides@gmail.com</a>></span>
wrote:<br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 06/09/13 00:01, Ryan Lane wrote:<br>
> Outside of tools (and deployment-prep, which is
rather ephemeral) we<br>
> don't consider any project "semi-production" and
the failure model is<br>
</div>
> meant to be handled at the instance level. (...)<br>
<br>
Well, I think it's on tools: <a moz-do-not-send="true"
href="http://tools.wmflabs.org/heritage/api/."
target="_blank">http://tools.wmflabs.org/heritage/api/.</a>..
<div class="im"><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
That said, relying on labs for something like this is
legitimately<br>
insane. Have you talked with Wikimedia Foundation
about getting<br>
production level support for WLM? That's what you
actually need.<br>
<br>
What will you do if the node hosting your instance
completely dies? Is<br>
your work puppetized? Can you just bring up a new
instance to replace<br>
it? Are you doing backups?<br>
</blockquote>
<br>
</div>
I think it's just a clone of the project at the
toolserver, and the code is under version control. It
would be nice to have it puppetized, though.<br>
<br>
IMHO a hostname like <a moz-do-not-send="true"
href="http://api.wikilovesmonuments.org" target="_blank">api.wikilovesmonuments.org</a>
should have been used, for independence from toolserver,
tools, wmflabs instances...
<div class="HOEnZb">
<div class="h5">
<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Using a different hostname doesn't really do much for
independence from anything unless you're also going to
host the infrastructure as well.</div>
<div><br>
</div>
<div>
My point still stands whether this is on tools or not. If
something is important enough that it shouldn't have
downtime it shouldn't be on Labs, even in the tools
project. It should have production-level support.</div>
<div><br>
</div>
<div>Labs is not funded, staffed, or architected to handle
production-level services. Tools was created in a way that
will work around host-level failures in the Labs
infrastructure, but if the network node dies, tools will
still go down. There's a number of other SPOFs in the
architecture that we're willing to accept for a
semi-production environment that would not exist in a
fully production environment.</div>
<div><br>
</div>
<div>We are putting effort into eliminating the SPOFs where
feasible, but we'll never recommend Labs for services that
must be up, since that's what production is for...</div>
</div>
</div>
</div>
</blockquote>
You are aware that the Toolserver is being killed of and toollabs is
supposed to replace it? Toolserver has alway been production, just
not with so many 9's of availability. Are you aware of how important
the tool* projects are for our projects? Tool* is not a stupid
sandbox, no our projects depend on tool* being available regardless
of the production status you think it has. Our communities don't
have an alternative.<br>
<br>
So question to the WMF: Are you going to treat toollabs seriously
like a production environment or not?<br>
I understand it won't have the same availability requirement as our
main cluster but you can at least apply proper production practices
like having a maintenance window, change management etc.<br>
<br>
Maarten<br>
</body>
</html>