I've altered the rewrite rules for the en/en2 split so that user agents
that don't identify themselves as some variant of "Mozilla" are kept
solely on
en.wikipedia.org.
The vast majority of web browsers identify as "Mozilla/XX (something)
blah blah" for historical compatibility reasons. The vast majority of
web search engine indexing spiders, on the other hand, do not. This
should help clean up search indexes so a search limited to
'en.wikipedia.org' can turn up all articles, and internal linking will
be properly taken into account.
This will push a little extra load onto larousse, but not I think too
much.
Just a note: on an overall basis, the traffic for the English wikipedia
is about the same as all the others combined. If we could get the
processor upgrade for larousse actually in place, well that would be
great. Since that doesn't seem to be forthcoming, alternatively, we
could move some of the languages to larousse and put English on pliny
only.
This should help balance things a little more cleanly without yet the
need for IP virtualization. But, that'll require updating DNS for
wikipedia.org, which I don't have access to. Any thoughts on this plan
before we go trying to organize it?
-- brion vibber (brion @
pobox.com)