On Tue, 11 Mar 2003, Brion Vibber wrote:
Googlebot is fairly light (several seconds to 30 seconds between requests, and follows our robots.txt restrictions - its getting articles, not millions of diffs or contribs pages. It's only a fraction of total pages being served). I have written them an e-mail asking if it's possible to restrict the spidering to off-peak hours, though.
I should point out too that I tried barring googlebot (by 403 Forbidden reponses) for the last ten minutes or so. No improvement in response time, so I've restored access.
-- brion vibber (brion @ pobox.com)