Brion Vibber wrote:
Lee Daniel Crocker wrote:
On Mon, 2005-05-09 at 12:43 -0700, Brion Vibber wrote:
A possible middle road is to rewrite the core wiki engine to a separate daemon, and adapt the existing PHP user interface to call into it for much of the backend work that actually touches data.
That's the approach I'd favor. The existing PHP code represents a lot of good user-interface work for which PHP is perfectly suited. The underlying stuff could easily be split up into multiple daemons (say, one for wikitext, one for images, one for equations,...) that could feed the PHP front-end.
This would also allow incremental development, since each of the daemons could be written and attached indicidually without disturbing the rest of the codebase.
<nod>
Our biggest, nastiest burden is with internal communications: the database changes too much. We have to wait on things getting sent, received, applied, and copied around, and lagging databases send everything into the toilet fast.
Yep. That's why I think the bulk of the text should be stored in a plain filesystem, where those problems are already well-known and solved, for the most part. That would reduce the database to just metadata, which would be much smaller and more efficient.
Replication's still an issue with filesystems; it seems like every network filesystem a) sucks and b) is a SPOF, and every distributed filesystem a) sucks and b) sucks.
This is one reason we've been talking about using an external object store rather than a filesystem. But a filesystem might work too; anyway that's a separate issue.
See my previous comments about Mr Torvald's lock-free "git" system. The ideas behind this look ideal as a way of running an object store for article versions, images and other BLOBs, and the whole linux-kernel list are kicking the **** out of it as real-life beta-testers as we speak. So far, no-one's found any show-stopping problems with this radical approach to large-scale version management. (or rather, version lack-of-management, since it more-or-less discards all previous wisdom about version control).
-- Neil