On Tue, 2005-05-10 at 00:47 +1000, Tim Starling wrote:
One important thing to note is that the parser is a
performance problem,
not a scalability problem. We really can throw more hardware at it, and
that will fix it quite nicely. In my opinion, our biggest performance
problem at the moment is not the parser, it's replication lag and
database load balancing. There's lots of work that has to be done there.
I agree, I don't think the parser's a big issue, although it would be
nice for a bit snappier response. In hindsight, storing the wikitext in
a database was a mistake. There's already a wonderful piece of software
highly optimized and scalable for storing randomly accessed variable-
sized chunks of text with lots of tools for backup, replication, and so
on; it's called a file system. Storing the wikitext itself in something
like Reiserfs would probably speed it up, and also speed up access to
the rest of the metadata in the database which would become much
smaller.
--
Lee Daniel Crocker <lee at piclab.com> <http://www.piclab.com/lee/>
<http://creativecommons.org/licenses/publicdomain/>