Hi!
Interesting, if what you say is true. I assumed that limits and indexes were
done by the primary mysql engine, not at the data container level.
Container provides different access methods to the data in it, so you can have different kinds of indexes, access methods, optimizations inside.
Say if container can allow efficient returning of ordered rows based on index (like InnoDB does), MySQL doesn't have to do sorting afterwards. Falcon, on the other hand, in some cases may provide better performance for unordered rows, but the way it reads data does not make covering index reads, or ordered index reads possible.
This
would leave the DB primarily as an index.
This is what we use core DBs now for. Though articles are stored in external instances, it is still MySQL (it is easier to handle replication that way ;-)
Something that occured to me that might be a good idea (TM) is to link up to subversion for the file repositories. This would have big space benefits, as only deltas would be stored, and give a more powerful view to the data.
Delta storage isn't that much different in terms of efficiency than compressing concatenated text. Moreover, the "view to the data" wouldn't be extended that much - we still have own version of wiki diffing, which is somewhat different from binary deltas.
With replication added (it is trivial of course in the kind of operations we're doing now), it may be more difficult to maintain Subversion-based repository (or some other kind of versioning system). Of course, it is possible.
Link
this up with file based articles and you have the potential to (relatively) easily produce a standalone wiki engine that could work remotely, much as devs do. I guess the benefits for the wikipedia is slight, so it probably wouldn't happen in MW...
MediaWiki isn't just storing articles. It is storing lots of information about relations between articles, tracking actions, changes, various metadata, etc.
I guess there're wiki engines with subversion storage, but.. we don't hear too much about them.
-- Domas http://dammit.lt/