[Wikipedia-l] what's being done about performance?

[Neil Harris]

Fred Bauder wrote:

>At 02:29 AM 7/13/02 -0700, maveric149 wrote:
>  
>
>>Much is now being done to remedy performance problems -- so I do believe
>>    
>>
>what 
>  
>
>>you said is needlessly rude (even if there is a grain of truth to it). 
>>    
>>
>
>My experience with coders has been in MUDs. Coders like to make new areas,
>tweak races and guilds, etc. Performance issues are not nearly as
>attractive nor is success in dealing with them easily attainable or
>recognizable to others. Like a MUD we are relying on voluntary coders so,
>sure, courtesy is necessary.
>
>But so is focus. Right now Wikipedia is like some Ionesco play with a dead
>body in the room that everyone inexplicably ignores.
>
>Fred Bauder
>
>[Wikipedia-l]
>To manage your subscription to this list, please go here:
>http://www.nupedia.com/mailman/listinfo/wikipedia-l
>
>
>
>  
>
Hi Fred,

I'm not a Wikipedia coder, I mostly contribute articles. Wearing my 
contributor hat, I've been really irritated by the disastrous 
performance in the last couple of months. My style of editing is 
pointillistic, with lots of tiny edits, even breaking down large 
articles into a series of smaller commits. I can understand that the 
poor performance will have discouraged many people.

Having said that, I am helping test the new system at the moment, and I 
can say that the coders are putting a lot of effort into making the 
system fast and smooth. I think they've taken the right decision, to 
give up on maintaining the old system, and work instead on getting the 
new system up and running really well before putting it into service 
ASAP. That means testing it really hard before going live.

Here's a progress report.

The new server is a dual-Athlon multiprocessor SMP machine with 2G of 
RAM, and an ext3 journalling filesystem.

At the moment,  the new server is running a hugely expanded database 
containing over 80,000 'articles', making a total of over 100,000 
entries in the database. This consists of a snapshot of the entire 
Wikipedia, together with about 50,000 machine-generated articles. It's 
serving around 1.2 articles per second under as realistic a traffic load 
as we can generate using lots of bots running on external servers - the 
traffic is nice and bursty, and the system is regualrly peaking at 12 
articles/second or more.

77% of pages are served in less than a second: what I'd consider to be 
'instant'
89% of pages are served in less than 2 seconds: what I'd consider to be 
'quick'
98% of pages are served in less than 5 seconds, about the start of the 
threshold of irritation
99.3% of pages are served in less than 10 seconds, a significant delay

Only 0.1% of articles take longer than 20 seconds to serve: what I would 
regard as failures.
About 30% of these are very big special pages. I'm trying to do a finer 
analysis of the rest, and find the exact conditions that seem to trigger 
the hiccups.

What this means, I hope, is that when the new server comes into service 
it will perform really well under normal loads, and editing will be 
smooth and lovely again.

Neil