Webmaster wrote:
http://en.wikipedia.org/wiki/Wikipedia:Database_download
Wikipedia limits crawling to 1 page per second in its robots.txt
Not anymore... http://en.wikipedia.org/robots.txt has:
## *at least* 1 second please. preferably more :D ## we're disabling this experimentally 11-09-2006 #Crawl-delay: 1
Matthew Flaschen