-- User:Whym Member of Wikimedian Society of Tokyo (東京ウィキメディアン会) / http://tokyo.wikimedia.jp ---------- Forwarded message ---------- From: Maximilian Doerr maximilian.doerr@gmail.com Date: Sun, Dec 4, 2016 at 1:51 PM Subject: [Labs-l] Some using a Python framework is relentlessly hammering Harvard sites, resulting an IP range ban. To: labs-l@lists.wikimedia.org Would the user who is querying the Harvard sites for planet data, that is carrying the UA “weblinkchecker Pywikibot/3.0-dev (g7171) requests/2.2.1 Python/2.7.6.final.0”, please stop, or severely throttle the GET requests. It’s making 168 requests to that site a minute, and consequently they banned labs from accessing it, according to the IT department there, who kindly shared with me the access log. I would imagine it’s also not being very friendly with the bandwidth usage of Labs itself. Consequently, InternetArchiveBot cannot ascertain if the site is alive or not because of this ban, and while I am working on a solution for these cases, it’s really best if the bot can be able to make these decisions on its own rather than deferring to a whitelist of sorts. Cyberpower678 English Wikipedia Account Creation Team Mailing List Moderator Global User Renamer _______________________________________________ Labs-l mailing list Labs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/labs-l