<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 12/29/15 7:01 PM, Bryan White wrote:<br>
</div>
<blockquote
cite="mid:CADtx0scVhe592mXN3DZocUnshxi6-S7oun_d26N6kRa1DnP2bg@mail.gmail.com"
type="cite">
<div dir="ltr">nothing of mine has run on the queue for ~90
minutes.
<div><br>
</div>
<div>Output of 'qstat -f'
<div>
<div>error: commlib error: got select error (Connection
refused)</div>
<div>error: unable to send message to qmaster using port
6444 on host "tools-grid-master.tools.eqiad.wmflabs": got
send error</div>
</div>
<div><br>
</div>
</div>
</div>
</blockquote>
12000 or so jobs were scheduled over the course of about 90 minutes
and the grid is overwhelmed -- we're working on untangling the mess.<br>
<br>
<br>
<blockquote
cite="mid:CADtx0scVhe592mXN3DZocUnshxi6-S7oun_d26N6kRa1DnP2bg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>Bryan</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Labs-l mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a>
<a class="moz-txt-link-freetext" href="https://lists.wikimedia.org/mailman/listinfo/labs-l">https://lists.wikimedia.org/mailman/listinfo/labs-l</a>
</pre>
</blockquote>
<br>
</body>
</html>