[Wikipedia-l] Re: Google is indexing edit pages
Daniel Mayer
maveric149 at yahoo.com
Sat Jun 22 02:07:05 UTC 2002
On Friday 21 June 2002 09:02 am, tarquin wrote:
> might Google also be following "pages that link here", "random" and
> "history"?
> Wikipedia freezes almost every day around 3pm UK time, which is odd as
> it's during the pediod with the fewest edits. Could that be when google
> comes a-crawling?
> we had problems on UnrealWiki with downloading robots calling too many
> diff links & oveerloading the server.
>
> tarquin
Is this fixed in the new code? I thought we decided not to allow Google to
index edit links (along with older versions of articles in History, log pages
and special pages other than maybe RecentChanges)? I don't see a problem with
allowing Google to follow links through "pages that link here" though -- so
long as this doesn't cause undue hardship on the server (if anything, this
should decrease the amount of time it take Googlebot to do its thing).
maveric149
More information about the Wikipedia-l
mailing list