On 15/08/05, Juliano Ravasi Ferraz ml@juliano.info wrote:
The page itself has <meta name="robots" content="noindex,follow">, and the link itself doesn't have a rel="nofollow" attribute, meaning that google is being instructed to follow those links.
Each time the link is followed, a new page is generated with a different &from=... parameter, and many new different links for last 50, 100, 250 and 500 changes, the last 1, 3, 7, 14 and 30 days, rss and atom feeds, which are all followed also.
Is there a quick fix for this?
Aha, there is indeed - if I understand it correctly, http://en.wikipedia.org/robots.txt disallows this by telling [friendly] spiders not to look at "/w/", where the actual scripts are, but letting them into "/wiki/" where the canonical rewritten URLs live.
Thus, a vanilla view of Special:Recentchanges will be spidered at "/wiki/Special:Recentchanges", but the alternate views are all of the form "/w/index.php?title=Special:Recentchanges&..." and will be ignored.
On a side note, what about making all "diff" links on Special:Recentchanges and Special:Contributions also have a rel="nofollow" attribute?
IIRC rel="nofollow" is badly named - it doesn't stop anything from following the link, only from using it to calculate page rankings. Unless I'm wrong about that, it's irrelevant for the current discussion.
wikitech-l@lists.wikimedia.org