[Labs-l] tools: some continuous jobs not rescheduled after reboot

[Merlijn van Deen]

Hello all,

During the past reboots, it seems not all jobs were correctly restarted.
This was caused by an issue [1] we do not completely understand, but it
causes jobs to fail while they are in the process of being rescheduled. We
are sorry for the inconvenience and extra maintenance burden this puts on
you as tool developers.

The possibly affected jobs are the following (sorted by user):

tools.abibot abibot
tools.admin toolhistory
tools.anon anon
tools.betacommand-dev stalkbotalerts
tools.betacommand-dev stalkboten
tools.bracketbot run
tools.cluebot cbng_redis_relay
tools.cluebot cbng_relay
tools.cluestuff recent_huggle
tools.cluestuff recent_huggle3
tools.coibot coibot
tools.commons-delinquent demon
tools.congressedits congresseditors
tools.defconbot defconbot
tools.dexbot ghaher69
tools.ecmabot ecmabot-wm
tools.giftbot sga
tools.hewiki-tools webServ
tools.hewiki-tools webServ
tools.loltrs loltrs
tools.lrbot lrbot
tools.morebots mostbots
tools.morebots production-logbot
tools.phetools ws_ocr_daemon
tools.ralgisbot bes-zdp
tools.ralgisbot qes-zdp
tools.ralgisbot tes-isbn
tools.ralgisbot tes-links
tools.ralgisbot tes-links
tools.ralgisbot ves-bienv
tools.ralgisbot ves-redir
tools.ralgisbot wes-isbn
tools.ralgisbot wes-isbnanx
tools.ralgisbot wes-linksan
tools.sigma sandbot3
tools.tc-rc start
tools.theoslittlebot wpstubs
tools.wikihistory dewiki_update2
tools.wpv cmd_perl
tools.yifeibot rmiw.w5

We will try to prevent this tomorrow by taking more time to reschedule the
jobs, but because we are uncertain of the underlying issue, we cannot
guarantee jobs will not fail after all. In any case, we will follow up with
a status report.

Again, sorry for the inconvenience.

Merlijn

[1] https://phabricator.wikimedia.org/T109362
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20150817/eb018477/attachment.html>