Forwarding this as it might be a pywikibot operator subscribed to this list.
--
User:Whym
Member of Wikimedian Society of Tokyo (東京ウィキメディアン会) /
http://tokyo.wikimedia.jp
---------- Forwarded message ----------
From: Maximilian Doerr <maximilian.doerr(a)gmail.com>
Date: Sun, Dec 4, 2016 at 1:51 PM
Subject: [Labs-l] Some using a Python framework is relentlessly
hammering Harvard sites, resulting an IP range ban.
To: labs-l(a)lists.wikimedia.org
Would the user who is querying the Harvard sites for planet data, that
is carrying the UA “weblinkchecker Pywikibot/3.0-dev (g7171)
requests/2.2.1 Python/2.7.6.final.0”, please stop, or severely
throttle the GET requests. It’s making 168 requests to that site a
minute, and consequently they banned labs from accessing it, according
to the IT department there, who kindly shared with me the access log.
I would imagine it’s also not being very friendly with the bandwidth
usage of Labs itself.
Consequently, InternetArchiveBot cannot ascertain if the site is alive
or not because of this ban, and while I am working on a solution for
these cases, it’s really best if the bot can be able to make these
decisions on its own rather than deferring to a whitelist of sorts.
Cyberpower678
English Wikipedia Account Creation Team
Mailing List Moderator
Global User Renamer
_______________________________________________
Labs-l mailing list
Labs-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l