I noticed this afternoon that something at IP 144.167.21.15 was spidering the site, loading up thousands of page diffs, user contributions pages, and other slow things at a rate of several per second -- apparently as fast as it could get them in. I blocked its IP from access to the /w/ directory (so it can only access regular pages and default-view special pages via the / and /wiki/Foo paths; I put a general prohibition into robots.txt as well), and the server load has gone *dramatically* down.

It appears to be someone running 'WebStripper' trying to copy the whole site; either it doesn't have sane throttling controls or they've disabled it.

The IP is an unnamed host belonging to University of Arkansas at Little Rock; probably some college kid enjoying the wonders of uni network bandwidth.

-- brion vibber (brion @ pobox.com)