Gerrit Holl schreef:
Hi,
What happened to robots.txt?
Why is this happening?
$ lynx -dump
http://en.wikipedia.org/robots.txt
# Special robots.txt for
nl.wikimedia.org
# No crawlers
User-agent: *
Disallow: /
No google, no Wayback Machine, no nothing... )-:
regards,
Gerrit.
But better speed because a strong drop of visitors.
It looks like a mistake. I think the idea was to hide only
nl.wikimedia.org form the search engines but it is active for all wikis.
Walter