Magnus Manske wrote:
A next step would be to output the "file heap" as compressed files to save disk space. Further, these compressed files could be merged into one single huge file, since the file system has a little trouble coping with that many files. That would require a separate file with title and position-in-file indices, though.
ZIP or another like archive format that allows random-access to contained files might be appropriate here.
-- brion vibber (brion @ pobox.com)