[Foundation-l] Re: Concern with performance issues

[Tim Starling]

Daniel Mayer wrote:
> The issue as I see it, is that we are now asking too much of too few volunteer
> developers and none of them can be on-site at the colo. Thus I think it is the
> time to consider hiring one full time or two of them part time to perform
> server maintenance tasks for us that currently are not done in a timely manor
> (esp the on-site stuff). Our budget is now so big ($US80,000 last quarter) that
> the addition of one or two employees under contract would not be too much of a
> burden. Neither would some up-front relocation money. 

We're doing this already, paying the guy who goes by the name "Baylink" 
on IRC. More servers would also be nice -- more in every category. We're 
having trouble using the hardware we have efficiently due to the lack of 
system administrators and developers, but inefficiency can be 
compensated for by buying more hardware.

The cheap way to get more hardware is to fix the ~7 machines that are 
currently broken and under warranty. Setting up new machines is a 
significant time cost to the system administrator team, it would be 
better if the existing machines were fixed and put back into service as 
soon as possible after breaking. The cluster configuration changes 
constantly, if a machine is out of action for long enough, reconfiguring 
it to match the rest of the cluster becomes as hard as adding a new machine.

There's not much any of us can do to get machines fixed besides hassle 
Jimbo.

I'm told 10 machines were ordered yesterday, that will help. Here are 
the details:

http://meta.wikimedia.org/wiki/Hardware_ordered_January_2005

I suspect any difference between the Paris squids and the Florida squids 
will disappear when those dual-H/D machines are put into service. 
However I don't think it will be long before the apache cluster needs 
attention again.

JamesDay is of the opinion that we need two more database slaves worth 
$10,000-$15,000 each. I'd add that to a large degree, available database 
hardware limits what we can consider implementing in MediaWiki. We 
typically live on the breadline, but we can always find a use for more 
database hardware.

-- Tim Starling