Hello,
Ryan wrote:
Right now, robots.txt excludes certain areas,
including XfD, from external
search. There are two problems with this:
1. Internal search indexes them.
2. Developers are forced to work on them on a case-by-case basis, which
takes a lot of time to work on. For example, bug 13398, regarding
well-intentioned bot reports that tend to seem like nasty condemnations of
sites as "spammers", has been open for over a month, but there are many
other important bugs open, and so this and other robots.txt bugs have not
yet been fixed.
For the former problem, it seems fairly straightforward to have logged-out
users search only mainspace pages, either by default behavior of the search
box, or by banning them entirely from searching these pages. Logged-in
users would retain their current defaults and ability to search user pages,
etc.
I think that this is a very bad idea. It is necessary for anonymous
users to be able to search the Meta and Talk namespaces / pages.
However, banning then from Google would be OK.
Regards,
Yann
For the latter problem, I think the community needs to
indicate that these
bugzilla requests are important, and should be handled much faster. I have
concerns about blocking namespaces because of the possible affects on other
languages who'd rather not block these namespaces (because robots.txt works
globally), but frankly I think that a full block of other namespaces would
be a good idea, and should be considered on meta.
--
[[User:Ral315]]
--
http://www.non-violence.org/ | Site collaboratif sur la non-violence
http://www.forget-me.net/ | Alternatives sur le Net
http://fr.wikisource.org/ | Bibliothèque libre
http://wikilivres.info | Documents libres