Hello,
Speaking of which, are we going to have a report about external storage in 1.4 from JeLuF or Domas?
I did merge my ExternalStorage support from HEAD to 1.4, and JeLuF did nice work with specialized DB class, which allows having archives (in 1.4) and all text (in 1.5). Currently we have ability to simply move HistoryBlobs to external storage and place ExternalStorage locators in main database. On live site we started with building 3-node mysql cluster on top of some apache hosts and JeLuF is already moving archive data (concatenated gzips) to it.
Currently we haven't finalized the blob format in external storage, as PHP archiving API isn't that handy, and serialized content sucks.
Anyway, idea of ExternalStore is really simple. You have a external storage locator string either in 'text' or 'old' tables, with flags set to 'external', or in future we might be adding special location field to revision table (or even make use of snowflake design ;-) The locator is like:
megastore://___put_anything_here___
When framework notices such content, it fires up ExternalStorageMegastore.php (or any other, according to protocol), and then storage class parses remaining part. Remaining part can specify _anything_, and may be purely specific to ES class. In DB case, it specifies MySQL cluster name and object id inside. In future it may specify content id (in case of blob retrieval), etc. There is sample HTTP class as well, which allows putting content on some external web server.
I would not be that surprised if someone would implement ExternalStorageGmail ;-)
Anyway, keeping main relational databases as lean as possible should be one of our primary concerns. It doesn't matter that much in case of single server setups though :)
Cheers, Domas
wikitech-l@lists.wikimedia.org