On Tue, 16 Aug 2005, andres wrote:
Accidentally i watched my logfiles (tail -f) and noticed a weird behaviour of the google spider. It spends literally days following all links in Spezial:Recentchanges.
there is one problem with Special:Recentchanges where you can click further and further on "next pages", even if there are no more.
How can i disallow search engines from indexing all the recent changes? They are worthless to index anyway.
check out: http://en.wikipedia.org/robots.txt for an example. You could use:
User-agent: * Disallow: /wiki/Special:Randompage Disallow: /wiki/Special%3ARandompage Disallow: /wiki/Special:Recentchanges Disallow: /wiki/Special%3ARecentchanges
christof