So we've been trawling around on the net looking for info on this problem. The SATA controller on the non-suda boxes seems to be an Intel ICH5 of some sort. In theory this should be supported in newer kernels either out of the box or after some tweaking. Maybe.
We tried a tweaked 2.6.6 kernel on Curly, with ata_piix compiled in instead of as a module. It booted, but still no DMA, and still with stuff on hda rather than sda as, perhaps, it ought to be using the SATA driver.
Curly crashed a few minutes later. :P It doesn't seem to be on our power manager; Jimmy or Jason, if you guys can either give it a reboot remotely or Jimmy if you can just reset it when you're next that way, that'd be great.
Googlings turned up that the BIOS generally has some sort of 'compatibility' mode setting for Parallel ATA. It either *should not* be on, or *should* be on, I can't tell which. I can't really get at the BIOS screen from California, so this'll have to be tried out in person.
The 3ware SATA RAID card of course will be a separate kettle of fish. :) It has its own driver, which we'll have to figure out someday.
-- brion vibber (brion @ pobox.com)
According to http://kerneltrap.org/node/view/1787 We may should install a debian Sarge on. If you want to do it jimmy, you can find iso image on ftp://ftp.lami.univ-evry.fr/debian-unofficial/sarge/ the first cd should be enought.
Did I say I love debian and his apt-get utility :) I use it at home for a year (sarge) and it works fine :) Very stable And with this, we dont have to care about security pbs, apt-get does the job :)
Shaihulud
On Thu, 2004-05-13 at 11:39 +0200, Camille Constans wrote:
According to http://kerneltrap.org/node/view/1787 We may should install a debian Sarge on. If you want to do it jimmy, you can find iso image on ftp://ftp.lami.univ-evry.fr/debian-unofficial/sarge/ the first cd should be enought.
Did I say I love debian and his apt-get utility :) I use it at home for a year (sarge) and it works fine :) Very stable And with this, we dont have to care about security pbs, apt-get does the job :)
Echoing this preference, something as simple as installing ploticus takes something like an hour because one has to fix all the dependencies manually (not to mention there's no working deb available). In debian, the latest version 2.20 is there, even a separate doc package. Copy&pasting url's back and forth is just so backwards ;-)
I would volunteer to install debian on the new machines if there's a concensus in favour. The current apaches would profit from a kernel upgrade sooner or later as well, once the new machines are up one could migrate the other 1U servers to debian and upgrade to 2.6 as time permits.
Gabriel Wicke
On Thu, May 13, 2004 at 08:30:13PM +0200, Gabriel Wicke wrote:
On Thu, 2004-05-13 at 11:39 +0200, Camille Constans wrote:
According to http://kerneltrap.org/node/view/1787 We may should install a debian Sarge on. If you want to do it jimmy, you can find iso image on ftp://ftp.lami.univ-evry.fr/debian-unofficial/sarge/ the first cd should be enought.
Did I say I love debian and his apt-get utility :) I use it at home for a year (sarge) and it works fine :) Very stable And with this, we dont have to care about security pbs, apt-get does the job :)
Echoing this preference, something as simple as installing ploticus takes something like an hour because one has to fix all the dependencies manually (not to mention there's no working deb available). In debian, the latest version 2.20 is there, even a separate doc package. Copy&pasting url's back and forth is just so backwards ;-)
I would volunteer to install debian on the new machines if there's a concensus in favour. The current apaches would profit from a kernel upgrade sooner or later as well, once the new machines are up one could migrate the other 1U servers to debian and upgrade to 2.6 as time permits.
That's a great idea - and please install it on all machines where texvc is running too, so we can have working Unicode in math mode.
Gabriel Wicke wrote:
Echoing this preference, something as simple as installing ploticus takes something like an hour because one has to fix all the dependencies manually (not to mention there's no working deb available). In debian, the latest version 2.20 is there, even a separate doc package. Copy&pasting url's back and forth is just so backwards ;-)
Just a simple question, why don't you use something like yum to manage all dependancies? I've been using linux for a few years (mandrake with urpmi/q/f, debian with apt-get or gentoo with emerge), and i have never had to manage dependancies manually. It seems weird to me to have to do that manually nowadays. :o) If you want to change of distribution, why not using one compiled specifically for the architecture. Isn't using binaries made for i386 just a waste of 20 years of cpu evolution? ;)
Med
Brion Vibber wrote:
The 3ware SATA RAID card of course will be a separate kettle of fish. :) It has its own driver, which we'll have to figure out someday.
I tested it here in the office on an old machine, by installing Fedora Core 1. I did not test it regarding DMA or anything fancy, all I can say is that it functions under Fedora Core 1.
When the new servers arrive, my plan is to install it with 2x250Gb and probably Fedora Core 1 (with your permission) and then stick it in the colo for ya'll to test more thoroughly. With 1 new big powerful db server and 4 new 1U boxes (to be customized according to our needs), we should have spare boxes enough to be able to run tests and such.
--Jimbo
On Thu, 13 May 2004 07:12:18 -0700 Jimmy Wales jwales@bomis.com wrote:
Brion Vibber wrote:
The 3ware SATA RAID card of course will be a separate kettle of fish. :) It has its own driver, which we'll have to figure out someday.
I tested it here in the office on an old machine, by installing Fedora Core 1. I did not test it regarding DMA or anything fancy, all I can say is that it functions under Fedora Core 1.
When the new servers arrive, my plan is to install it with 2x250Gb and probably Fedora Core 1 (with your permission) and then stick it in the colo for ya'll to test more thoroughly. With 1 new big powerful db server and 4 new 1U boxes (to be customized according to our needs), we should have spare boxes enough to be able to run tests and such.
I'm afraid that the 4 new 1U boxes will be used quickly.... As our 5 web servers are completely overloaded. And one squid, browne, is often at a load 20-30. Just an idea, for the web servers, as we use only cpu on this boxes, maybe we could buy some inexpensive dual-athlon instead of mono-P4.
Just my 2 cents.
Shaihulud
Camille Constans wrote:
I'm afraid that the 4 new 1U boxes will be used quickly.... As our 5 web servers are completely overloaded. And one squid, browne, is often at a load 20-30. Just an idea, for the web servers, as we use only cpu on this boxes, maybe we could buy some inexpensive dual-athlon instead of mono-P4.
Do others share this assessment? If so, we need to increase the visibility of our fund raising efforts, because we've now spent most of our original campaign's funds. Some money trickles in all the time, but it's been de-emphasized significantly.
I'd strongly prefer that we not wait for an emergency, and I hope that we can have an "annual campaign" again at the end of the year to raise a large block of money for 2005.
The total price of the new db server plus 4x1u mono-P4 machines was just over $20,000.
I bought more than recommended this time around because I am going to be out of town for 5 weeks. I do have someone here (Michael) who can setup more boxes while I'm gone, if absolutely needed, but obviously it'll be harder that way. If we need more 1u machines, dual athlons or whatever, we should make that move now probably.
--Jimbo
Jimmy Wales wrote:
Camille Constans wrote:
I'm afraid that the 4 new 1U boxes will be used quickly.... As our 5 web servers are completely overloaded. And one squid, browne, is often at a load 20-30. Just an idea, for the web servers, as we use only cpu on this boxes, maybe we could buy some inexpensive dual-athlon instead of mono-P4.
Do others share this assessment?
There's more efficiency to be gained in the software; we're still not saving rendered html and diffs and such in many places where we could/should.
-- brion vibber (brion @ pobox.com)
Brion Vibber wrote:
Jimmy Wales wrote:
Camille Constans wrote:
I'm afraid that the 4 new 1U boxes will be used quickly.... As our 5 web servers are completely overloaded. And one squid, browne, is often at a load 20-30. Just an idea, for the web servers, as we use only cpu on this boxes, maybe we could buy some inexpensive dual-athlon instead of mono-P4.
Do others share this assessment?
There's more efficiency to be gained in the software; we're still not saving rendered html and diffs and such in many places where we could/should.
According to Gabriel, the dominant use of CPU at the moment is rendering logged-in page views. When that is removed by better caching, we could be looking at a factor of 3 reduction in CPU load. After that, caching diffs could produce a significant further saving.
As I've said before, the apache machines currently need CPU power, and have very little use for disk or RAM.
Just for comparison, here's the cluster architecture used by my theoretical physics group. We have 23 compute nodes plus a head node, purchased over a year ago. Each node is a diskless single Pentium IV with 1 GB RAM. 10 x 2GHz, 13x2.4 GHz, head is 550 MHz with a disk. 100 Mbps ethernet. It's rumoured to have been as cheap as chips, on the order of A$10,000 (US$7,000). Now I'm not saying this architecture precisely would be right for us, I'm just saying that there are ways of getting CPU power without paying a lot of money.
-- Tim Starling
wikitech-l@lists.wikimedia.org