With your google Webmaster tools account, you can view the graphs of number of pages fetched per day in the Tools->Set crawl rate option. On the wiki I manage, with about 2,000 pages of content, it is hit an average of 130 pages per day from google over the past four months. Certainly not a load of much importance on the WEB server. Beware of alias names that google may see your site as, such as: website.com www.website.com ... etc ...
--Hiram
Agon S. Buchholz wrote:
David A. Desrosiers wrote:
Log into Google Webmaster Tools, add your site to their system, and adjust the crawl rate to be slower using the tools provided there.
I tried this; it has no noticable effect, neither on the effective server load nor on the threads/queries I see in mytop.
Thanks anyway! -asb