[Wikipedia-l] robots.txt - why disallow everyone, everywhere?

Gerrit Holl gerrit at nl.linux.org
Fri Mar 25 19:24:26 UTC 2005


Hi,

What happened to robots.txt?
Why is this happening?

$ lynx -dump http://en.wikipedia.org/robots.txt
# Special robots.txt for nl.wikimedia.org
# No crawlers

User-agent: *
Disallow: /

No google, no Wayback Machine, no nothing... )-:

regards,
Gerrit.

-- 
Weather in Twenthe, Netherlands 25/03 19:25:
	9.0°C   wind 0.0 m/s None (57 m above NAP)
-- 
In the councils of government, we must guard against the acquisition of
unwarranted influence, whether sought or unsought, by the
military-industrial complex. The potential for the disastrous rise of
misplaced power exists and will persist.
    -Dwight David Eisenhower, January 17, 1961



More information about the Wikipedia-l mailing list