On Wed, 23 Apr 2003, Nick Reinking wrote:
Perhaps it would be best to put in an iptables rule to deny a TCP/IP connection from their crawler (assuming it comes from a reasonably limited set of IPs)?
No, see, it's a *distributed* crawler. So I don't know where the fuck they're coming from. Like I said, it's really pissing me off.
See also: http://www.grub.org/forums/viewtopic.php?t=124
-- brion vibber (brion @ pobox.com)