<div dir="ltr">For Tool Labs, the plan is as follows:<div> - tomorrow, we will disable the queue so no new tasks will be distributed to the affected hosts</div><div> - we will send an e-mail with tasks that are still running an hour later</div><div><br></div><div>Unfortunately, there is currently no host that can run jobs that take longer than a few days, because other virt* hosts will also be rebooted this week.</div><div><br></div><div>For reference, the current long-running jobs on these hosts are the following, grouped by user name:. Please take a look and consider whether the jobs are still doing something useful -- and if not, please kill them (qdel <job id>).<br></div><div><br></div><div>Merlijn</div><div><br></div><div><br></div><div><br></div><div>Columns:</div><div><br></div><div>job id name start date/time<br></div><div><br></div><div>aka<br></div><div><div>---------------</div><div>1317747 start Sat Aug 1 19:17:12 2015</div><div><br></div><div>tools.checkwiki<br></div><div>---------------</div><div>145845 eswiki-munch Thu Jun 25 05:00:13 2015</div><div>818559 arwiki-munch Sat Jul 18 05:00:16 2015</div><div><br></div><div>tools.dexbot</div><div>---------------</div><div>1236997 del Thu Jul 30 13:36:09 2015</div><div>1341699 kian_new2 Sun Aug 2 11:03:18 2015</div><div><br></div><div>tools.gpy</div><div>---------------</div><div>527733 gpy Thu Jul 9 01:14:28 2015</div><div><br></div><div>tools.luke081515bot<br></div><div>---------------</div><div>1346744 queue Sun Aug 2 14:24:31 2015</div><div><br></div><div>tools.mjbmrbot</div><div>---------------</div><div>209254 lgdcp2_1 Sat Jun 27 15:35:04 2015</div><div>273994 lgdcp2_2 Tue Jun 30 02:00:07 2015</div><div>345013 lgdcp2_3 Thu Jul 2 15:00:05 2015</div><div>807548 lsdcp2_3 Fri Jul 17 21:00:12 2015</div><div>1092477 lgdcp1_4 Sun Jul 26 14:00:07 2015</div><div>1093960 lsdcp1_4 Sun Jul 26 15:00:10 2015</div></div><div><div><br></div><div>tools.shuaib-bot</div><div>---------------</div><div>1622344 translator Mon Aug 10 02:10:09 2015</div><div><br></div><div>tools.wikidata-exports<br></div><div>---------------</div><div>694469 create_dumps Tue Jul 14 08:40:22 2015</div><div>735030 create_dumps Wed Jul 15 14:31:25 2015</div><div>768842 create_dumps Thu Jul 16 16:12:52 2015</div></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 10 August 2015 at 21:20, Andrew Bogott <span dir="ltr"><<a href="mailto:abogott@wikimedia.org" target="_blank">abogott@wikimedia.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
On Wednesday I'll be rebooting labvirt1001. This will cause
downtime for about 10% of labs instances, and this downtime may last
as long as 60 minutes (although the average downtime will be much
less.)<br>
<br>
We will do our best to juggle and reschedule ToolLabs jobs, but
persistent jobs that cannot gracefully restart may be interrupted
and require your personal attention.<br>
<br>
Here is the list of instances that will be affected by this reboot:<br>
<br>
<pre>| citoidtest | ACTIVE | - | Running | public=10.68.16.182 |
| conf | ACTIVE | - | Running | public=10.68.18.87, 208.80.155.233 |
| deployment-bastion | ACTIVE | - | Running | public=10.68.16.58, 208.80.155.191 |
| deployment-cache-text02 | ACTIVE | - | Running | public=10.68.16.16 |
| deployment-elastic08 | ACTIVE | - | Running | public=10.68.17.188 |
| deployment-memc03 | ACTIVE | - | Running | public=10.68.16.15 |
| deployment-parsoid05 | ACTIVE | - | Running | public=10.68.16.120 |
| deployment-pdf01 | ACTIVE | - | Running | public=10.68.16.73 |
| deployment-restbase01 | ACTIVE | - | Running | public=10.68.17.227 |
| deployment-salt | ACTIVE | - | Running | public=10.68.16.99 |
| deployment-urldownloader | ACTIVE | - | Running | public=10.68.16.135 |
| diffengine | ACTIVE | - | Running | public=10.68.17.127 |
| educationdashboard-i18n | SHUTOFF | - | Shutdown | public=10.68.16.235 |
| ee-flow-extra | ACTIVE | - | Running | public=10.68.16.102 |
| etcd01 | ACTIVE | - | Running | public=10.68.16.130 |
| etcd03 | ACTIVE | - | Running | public=10.68.16.132 |
| firstinstance | SHUTOFF | - | NOSTATE | public=10.68.16.212 |
| graphite-trusty | ACTIVE | - | Running | public=10.68.17.181 |
| huggle-d2 | ACTIVE | - | Running | public=10.68.17.194 |
| icinga | ACTIVE | - | Running | public=10.68.16.195 |
| integration-raita | ACTIVE | - | Running | public=10.68.16.53 |
| integration-slave-trusty-1013 | ACTIVE | - | Running | public=10.68.18.28 |
| integration-slave-trusty-1015 | ACTIVE | - | Running | public=10.68.18.30 |
| k8s-worker-02 | ACTIVE | - | Running | public=10.68.18.91 |
| kartotherian1 | ACTIVE | - | Running | public=10.68.16.117 |
| language-replag-slave | SHUTOFF | - | Shutdown | public=10.68.16.248 |
| maps-tiles2 | ACTIVE | - | Running | public=10.68.17.110 |
| mobile-browser-tests | ACTIVE | - | Running | public=10.68.16.149 |
| mwreview-proxy-test | ACTIVE | - | Running | public=10.68.16.83 |
| osmit-cruncher1 | ACTIVE | - | Running | public=10.68.17.92 |
| puppet-jmm-debdeploy-precise | ACTIVE | - | Running | public=10.68.18.106 |
| puppet-mailman | ACTIVE | - | Running | public=10.68.17.177 |
| sentry-builder | ACTIVE | - | Running | public=10.68.18.82 |
| staging-eventlogging | ACTIVE | - | Running | public=10.68.16.199 |
| staging-ms-be03 | ACTIVE | - | Running | public=10.68.17.249 |
| staging-rdb01 | ACTIVE | - | Running | public=10.68.17.193 |
| staging-tin | ACTIVE | - | Running | public=10.68.16.110 |
| stashbot-logstash | ACTIVE | - | Running | public=10.68.18.101 |
| tools-bastion-02 | ACTIVE | - | Running | public=10.68.16.44, 208.80.155.132 |
| tools-exec-1201 | ACTIVE | - | Running | public=10.68.17.49, 208.80.155.203 |
| tools-exec-1202 | ACTIVE | - | Running | public=10.68.16.57, 208.80.155.211 |
| tools-exec-1204 | ACTIVE | - | Running | public=10.68.17.88, 208.80.155.213 |
| tools-exec-1206 | ACTIVE | - | Running | public=10.68.17.105, 208.80.155.215 |
| tools-exec-1209 | ACTIVE | - | Running | public=10.68.17.129, 208.80.155.218 |
| tools-exec-1213 | ACTIVE | - | Running | public=10.68.17.252, 208.80.155.222 |
| tools-exec-1217 | ACTIVE | - | Running | public=10.68.18.20, 208.80.155.226 |
| tools-exec-1218 | ACTIVE | - | Running | public=10.68.18.19, 208.80.155.227 |
| tools-exec-1408 | ACTIVE | - | Running | public=10.68.18.14, 208.80.155.152 |
| tools-exec-cyberbot | ACTIVE | - | Running | public=10.68.16.39 |
| tools-webgrid-generic-1404 | ACTIVE | - | Running | public=10.68.18.53 |
| tools-webgrid-lighttpd-1409 | ACTIVE | - | Running | public=10.68.18.43 |
| tools-webgrid-lighttpd-1410 | ACTIVE | - | Running | public=10.68.18.44 |
| toolsbeta-exec-101 | ACTIVE | - | Running | public=10.68.16.7 |
| toolsbeta-exec-201 | ACTIVE | - | Running | public=10.68.16.250 |
| wikidata-mobile | ACTIVE | - | Running | public=10.68.18.41 |
| wikispy | ACTIVE | - | Running | public=10.68.17.119 |
| wlmjurytool2014 | ACTIVE | - | Running | public=10.68.17.134 |
| wmt-exec | ACTIVE | - | Running | public=10.68.17.236 |</pre>
<br>
</div>
<br>_______________________________________________<br>
Labs-announce mailing list<br>
<a href="mailto:Labs-announce@lists.wikimedia.org">Labs-announce@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-announce" rel="noreferrer" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-announce</a><br>
<br>_______________________________________________<br>
Labs-l mailing list<br>
<a href="mailto:Labs-l@lists.wikimedia.org">Labs-l@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/labs-l" rel="noreferrer" target="_blank">https://lists.wikimedia.org/mailman/listinfo/labs-l</a><br>
<br></blockquote></div><br></div>