On 15/08/05, Juliano Ravasi Ferraz <ml(a)juliano.info> wrote:
The page itself has <meta name="robots"
content="noindex,follow">, and
the link itself doesn't have a rel="nofollow" attribute, meaning that
google is being instructed to follow those links.
Each time the link is followed, a new page is generated with a different
&from=... parameter, and many new different links for last 50, 100, 250
and 500 changes, the last 1, 3, 7, 14 and 30 days, rss and atom feeds,
which are all followed also.
Is there a quick fix for this?
Aha, there is indeed - if I understand it correctly,
http://en.wikipedia.org/robots.txt disallows this by telling
[friendly] spiders not to look at "/w/", where the actual scripts are,
but letting them into "/wiki/" where the canonical rewritten URLs
live.
Thus, a vanilla view of Special:Recentchanges will be spidered at
"/wiki/Special:Recentchanges", but the alternate views are all of the
form "/w/index.php?title=Special:Recentchanges&..." and will be
ignored.
On a side note, what about making all "diff"
links on
Special:Recentchanges and Special:Contributions also have a
rel="nofollow" attribute?
IIRC rel="nofollow" is badly named - it doesn't stop anything from
following the link, only from using it to calculate page rankings.
Unless I'm wrong about that, it's irrelevant for the current
discussion.
--
Rowan Collins BSc
[IMSoP]