[Wikipedia-l] Re: Google is indexing edit pages

Daniel Mayer maveric149 at yahoo.com
Sat Jun 22 02:07:05 UTC 2002


On Friday 21 June 2002 09:02 am, tarquin wrote:
> might Google also be following "pages that link here", "random" and
> "history"?
> Wikipedia freezes almost every day around 3pm UK time, which is odd as
> it's during the pediod with the fewest edits. Could that be when google
> comes a-crawling?
> we had problems on UnrealWiki with downloading robots calling too many
> diff links & oveerloading the server.
>
> tarquin

Is this fixed in the new code? I thought we decided not to allow Google to 
index edit links (along with older versions of articles in History, log pages 
and special pages other than maybe RecentChanges)? I don't see a problem with 
allowing Google to follow links through "pages that link here" though -- so 
long as this doesn't cause undue hardship on the server (if anything, this 
should decrease the amount of time it take Googlebot to do its thing).

maveric149 



More information about the Wikipedia-l mailing list