Hi all,
We had an outage yesterday, which was likely caused by a bot requesting
about 25,000 requests in a 20 minute span during a normally heavy traffic
period. Most of the requests didn't hit the cache, so our Apache servers got
hammered, and our site went down, but our Squid server was fine. All of the
requests came from the same IP range.
Is there a common way to deal with bots like these, other than manually
identifying them (usually when it's too late) and blocking them via
iptables? Does Squid offer any throttling? Or are there any other options?
Just curious.
Thanks,
Travis