On Wed, 23 Apr 2003, Nick Reinking wrote:
Perhaps it would be best to put in an iptables rule to
deny a TCP/IP
connection from their crawler (assuming it comes from a reasonably
limited set of IPs)?
No, see, it's a *distributed* crawler. So I don't know where the fuck
they're coming from. Like I said, it's really pissing me off.
See also:
http://www.grub.org/forums/viewtopic.php?t=124
-- brion vibber (brion @
pobox.com)