On 5/18/02 3:08 PM, "Jimmy Wales" jwales@bomis.com wrote:
The Cunctator wrote:
Again, if we surmise that spiders are causing slowdowns, we should be able to find evidence for that BEFORE we block parts of the site from them. And even then we should see if the fault lies in the site's code.
I think this is right, although blocking them from 'edit' doesn't seem harmful. Certainly, it's good for spiders to hit 'Recent Changes', and often.
Both points make perfect sense to me.