[Mediawiki-l] googlebot
Christof Damian
christof at damian.net
Tue Aug 16 15:40:13 UTC 2005
On Tue, 16 Aug 2005, andres wrote:
> Accidentally i watched my logfiles (tail -f) and noticed a weird
> behaviour of the google spider. It spends literally days following
> all links in Spezial:Recentchanges.
there is one problem with Special:Recentchanges where you can click
further and further on "next pages", even if there are no more.
> How can i disallow search engines from indexing all the recent
> changes? They are worthless to index anyway.
check out: http://en.wikipedia.org/robots.txt for an example. You
could use:
User-agent: *
Disallow: /wiki/Special:Randompage
Disallow: /wiki/Special%3ARandompage
Disallow: /wiki/Special:Recentchanges
Disallow: /wiki/Special%3ARecentchanges
christof
--
Christof Damian
christof at damian.net
More information about the MediaWiki-l
mailing list