Hi all
I just got about 7000 hits for CatScan in the last 60 minutes - originating from zedler itself. This cause replication lag to rise nearly linearly over that time. So, whoever is doing that:
STOP IT!
Spidering tools is generally a bad idea, and if you have access to the database yourself, it's plain silly. In any case, before abusing the web interface, simply ask me to provide the data in an easy to process manner.
Regards, -- Daniel
Daniel Kinzler schrieb:
Hi all
I just got about 7000 hits for CatScan in the last 60 minutes - originating from zedler itself. This cause replication lag to rise nearly linearly over that time. So, whoever is doing that:
STOP IT!
Spidering tools is generally a bad idea, and if you have access to the database yourself, it's plain silly. In any case, before abusing the web interface, simply ask me to provide the data in an easy to process manner.
Might be me, but I'm using the bot mode. Anyway, I only get blank pages back from CatScan when running on zedler, which breaks my tools.
Ah, so I'll have to implement the CatScan functionality myself...
Magnus
Magnus Manske wrote:
Might be me, but I'm using the bot mode. Anyway, I only get blank pages back from CatScan when running on zedler, which breaks my tools.
Currently, you should be getting a 403; I shut off all access to my tools from the toolserver, except for WikiProxy. I might enable it again when the insanity has stopped.
Ah, so I'll have to implement the CatScan functionality myself...
Well, you can use my tools in bot mode, at a reasonable frequency. Not > 100 requests per minute. What are you doing anyway?
Note that if you implement it yourself and use it at such a rate, you'll kill the db.
-- Daniel
Daniel Kinzler schrieb:
Magnus Manske wrote:
Might be me, but I'm using the bot mode. Anyway, I only get blank pages back from CatScan when running on zedler, which breaks my tools.
Currently, you should be getting a 403; I shut off all access to my tools from the toolserver, except for WikiProxy. I might enable it again when the insanity has stopped.
Ah, so I'll have to implement the CatScan functionality myself...
Well, you can use my tools in bot mode, at a reasonable frequency. Not > 100 requests per minute. What are you doing anyway?
Note that if you implement it yourself and use it at such a rate, you'll kill the db.
/If/ I am the cause, this is due to the unexpected popularity of
http://tools.wikimedia.de/~magnus/articleweight.php
Manus
toolserver-l@lists.wikimedia.org