I've altered the rewrite rules for the en/en2 split so that user agents that don't identify themselves as some variant of "Mozilla" are kept solely on en.wikipedia.org.
The vast majority of web browsers identify as "Mozilla/XX (something) blah blah" for historical compatibility reasons. The vast majority of web search engine indexing spiders, on the other hand, do not. This should help clean up search indexes so a search limited to 'en.wikipedia.org' can turn up all articles, and internal linking will be properly taken into account.
This will push a little extra load onto larousse, but not I think too much.
Just a note: on an overall basis, the traffic for the English wikipedia is about the same as all the others combined. If we could get the processor upgrade for larousse actually in place, well that would be great. Since that doesn't seem to be forthcoming, alternatively, we could move some of the languages to larousse and put English on pliny only.
This should help balance things a little more cleanly without yet the need for IP virtualization. But, that'll require updating DNS for wikipedia.org, which I don't have access to. Any thoughts on this plan before we go trying to organize it?
-- brion vibber (brion @ pobox.com)
Is there some way to balance the load without having the URLs explicitly direct to en2.wikipedia.org?
It would be much better if all users were consistently directed to "en.wikipedia.org" (even though I'll reiterate my basic opposition to the redirect from www in the first place) instead of randomly between "en.wikipedia.org" or "en2.wikipedia.org".
Brion Vibber wrote:
Just a note: on an overall basis, the traffic for the English wikipedia is about the same as all the others combined. If we could get the processor upgrade for larousse actually in place, well that would be great. Since that doesn't seem to be forthcoming, alternatively, we could move some of the languages to larousse and put English on pliny only.
Actually, Jason has a new machine built which I'm giving to Wikipedia. It's apparently stable and all, and he offered to drive it down today, but I figured he could wait a bit until he has other reasons to go down (which he will, soon, since I've ordered some parts and a new server for Bomis, etc.).
So within a week or two, Wikipedia will be getting a new server as a Christmas gift from Bomis. Not a mondo powerhouse, but perfectly decent.
--Jimbo
wikitech-l@lists.wikimedia.org