[Mediawiki-l] Re: rel=nofollow for edit, history and diff links

[Rowan Collins]

On 29/04/05, Jeremie Bouillon <jeremie at shadowfrance.net> wrote:
> About the robot.txt, can we find the one used by large wiki (like
> Wikipedia, or others) to help Wikimedia newbie exclube spiderbot from where
> there aren't supposed to be ?

robots.txt is just a plain text file in the root of the webserver; you
can look at anybody's in a browser by typing the URL in e.g.
http://en.wikipedia.org/robots.txt
http://www.google.com/robots.txt
or even http://www.robotstxt.org/robots.txt :)

-- 
Rowan Collins BSc
[IMSoP]