I can see how to exclude certain directories, but how can I prevent the
Recent Changes pages, and the edit and history pages of articles from being
crawled?
At 08:41 PM 6/9/05, Brion Vibber wrote:
=James Birkholz= wrote:
How can I steer the Googlebot?
http://www.robotstxt.org/wc/exclusion.html