Hi,
What happened to robots.txt?
Why is this happening?
$ lynx -dump http://en.wikipedia.org/robots.txt
# Special robots.txt for nl.wikimedia.org
# No crawlers
User-agent: *
Disallow: /
No google, no Wayback Machine, no nothing... )-:
regards,
Gerrit.
--
Weather in Twenthe, Netherlands 25/03 19:25:
9.0°C wind 0.0 m/s None (57 m above NAP)
--
In the councils of government, we must guard against the acquisition of
unwarranted influence, whether sought or unsought, by the
military-industrial complex. The potential for the disastrous rise of
misplaced power exists and will persist.
-Dwight David Eisenhower, January 17, 1961