Hello,
I used the last hours trying to dig in the infrastructural organization of the Wikimedia servers. My starting points where [[meta:Wikimedia_servers]] and Ganglia and my motivation was Wikipedias slowness in the last time.
In contrast to my expectations, the database servers are far away from being under high load. It even seems the pressure is so low, you can easily live without holbach and webster for days (resp. over a month). Bottlenecks are Apaches and Squids (yes, I know that's nothing new for you).
But like all other clusters too, the load is very unequally distributed over the machines. For example the Yahoo! squids showed yf1003 9.39, yf1000 7.60, yf1004 1.60, yf1002 1.44, yf1001 0.73 at noon (UTC) today and similar load values (albeit with a different distribution) at other times.
Or the Apaches in Florida: 16 Apaches with load around 15, 9 between 1.5 and 2, 8 between 1 and 1.5 and 10 less than 1.
Where does this come from, or is this wanted? Wouldn't a more balanced load be better?
Other point: The Yahoo! Squids do virtually nothing between 18:00 and 0:00 (and machines besides yf1000-yf1004 to virtually nothing around the clock). How nice would it be make them helping out the other overloaded machines in Florida and Netherlands at least in these six hours.
And no, I don't criticize anyone or know how to do it better. But available informations look strange to me - it would be great to get some explanations.
Speaking of explanations. I've three more simple questions: 1. Squids at lopar idle all the time since dns has been moved of them. What where the problems with them and will they be back soon? 2. Commons is very slow since the move from the prior "overloaded" server to the new one. Any explanation to satisfy a simple user? And what server is the new one? 3. I read about new machines srv51-70. Where do they come from? Can't see a recent order for them or they are mentioned on [[meta:Wikimedia_servers]].
Thank you in advance, Juergen