Hi Martin,
On 4 December 2016 at 18:29, Martin Urbanec <martin.urbanec(a)wikimedia.cz>
wrote:
I was running weblinkchecker.py for whole cswiki (job
was submited to the
grid at Sun, 20 Nov 2016 16:54:24 GMT) because I wished to have a list of
deadlinks. This may correspond with the UA (because I used script named
weblinkschecker.py). I trusted this script it won't do anything wrong
because this script was and still is in standard core package. I also use
3.0-dev version of pywikibot and Python 2.7.6.
It probably wasn't you, but it was indeed the standard weblinkchecker
causing this. Apparently no throttling is implemented -- just a maximum
number of parallel connections. Many parallel connections are fine... but
not to the same host. This bot was running on eswiki, and ewsiki has
thousands of links to
http://www.minorplanetcenter.net/.
I have contacted the user, and will file a bug for Pywikibot to get this
solved on that end.
Best,
Merlijn