On Fri, May 2, 2008 at 4:24 PM, Judson Dunn cohesion@sleepyhead.org wrote:
On Fri, May 2, 2008 at 11:22 AM, Chris Howie cdhowie@gmail.com wrote:
I think the closest we're going to get to a good technical solution would be to have some magic words __INDEX__ and __NOINDEX__ that will force indexing on or off for a document. The default could be set per-namespace, and I'm assuming we'd want articlespace to be the only one indexed by default.
I think everything should be indexed by default, and we can address problems as they appear (with {{NOINDEX}}) rather than hiding valuable information from the web. Obviously when something is sure to cause problems a template could be used preemptively. Better to make small organic changes than giant sweeping ones in my opinion.
I'm not particularly sold on either preemptive or as-needed yet, just tossing out ideas. There are certain areas that have already been preemptively excluded from search results (like AfD). I'm surprised that ANI hasn't received the same treatment yet.
Even if we use the preemptive approach, a rather large chunk of WP: would probably wind up in Google anyway, if we forced indexing in templates like {{policy}}, {{guideline}}, {{essay}}, etc. A lot of other stuff would probably be useful too but I think the majority of stuff people are searching for would still be there.
Oh and we'd probably also want to allow indexing to Help: by default, if we wind up going that route.