[Foundation-l] [WikiEN-l] Searchability of non-mainspace pages

[Yann Forget]

Hello,

Ryan wrote:
> Right now, robots.txt excludes certain areas, including XfD, from external
> search.  There are two problems with this:
> 
> 1.  Internal search indexes them.
> 2.  Developers are forced to work on them on a case-by-case basis, which
> takes a lot of time to work on.  For example, bug 13398, regarding
> well-intentioned bot reports that tend to seem like nasty condemnations of
> sites as "spammers", has been open for over a month, but there are many
> other important bugs open, and so this and other robots.txt bugs have not
> yet been fixed.
> 
> For the former problem, it seems fairly straightforward to have logged-out
> users search only mainspace pages, either by default behavior of the search
> box, or by banning them entirely from searching these pages.  Logged-in
> users would retain their current defaults and ability to search user pages,
> etc.

I think that this is a very bad idea. It is necessary for anonymous 
users to be able to search the Meta and Talk namespaces / pages. 
However, banning then from Google would be OK.

Regards,

Yann

> For the latter problem, I think the community needs to indicate that these
> bugzilla requests are important, and should be handled much faster.  I have
> concerns about blocking namespaces because of the possible affects on other
> languages who'd rather not block these namespaces (because robots.txt works
> globally), but frankly I think that a full block of other namespaces would
> be a good idea, and should be considered on meta.
> 
> --
> [[User:Ral315]]

-- 
http://www.non-violence.org/ | Site collaboratif sur la non-violence
http://www.forget-me.net/ | Alternatives sur le Net
http://fr.wikisource.org/ | Bibliothèque libre
http://wikilivres.info | Documents libres