[Wikipedia-l] slow hardware

[Rowan Collins]

On Fri, 22 Oct 2004 19:59:40 +0300, NSK <nsk2 at wikinerds.org> wrote:
> I notice that the WP site is very slow. I think WP is a victim of its growth.
> 
> Have you thought of cooperating with some service like Coral? Or, you could
> just build your own version of it.

Various ways of distributing the server load are indeed always being
considered. If you look at the diagram on
http://meta.wikimedia.org/wiki/Wikimedia_servers you will see that
while not *geographically* distributed, the current system is not
exactly what you'd call "flat". :D

I believe further proposed enancements include the purchase of "squid"
proxy servers in different countries (I tihnk somebody was in the
process of trying this in France), with some kind of geographical
server selection. There was also talk of using something called "Super
Sparrow", or a similar solution, which is also related to this issue.

The biggest problem, though, and why we can't simply piggy-back a
general-use replication network like Coral seems to be, is that by its
very nature, Wikipedia content is not static, and it is imperative
that we can keep track of what version is being served to whom. In the
current setup, the software is able to flush out-of-date pages from
the various 'squid' caches when they are edited, so that things like
removal of vandalism won't take an arbitrary amount of time to filter
through to normal users. Any wider distribution system needs to retain
this per-article control.

Plus, a lot of the slow-down is from people using inherently
un-distributable features like editing pages, or refreshing their
watchlists. It *might* be possible to create some kind of
wikiP2Pedia[tm], where the database itself was more widely
distributed, but it would be no simple feat of software engineering...

-- 
Rowan Collins BSc
[IMSoP]