http://bugs.openzim.org/show_bug.cgi?id=21
--- Comment #1 from Tommi Mäkitalo tommi@tntnet.org 2010-09-26 22:01:26 CET --- The data is always fully uncompressed. There is no way to prevent that. LZMA2 uses 1MB chunks internally and it will always uncompress the whole chunk. If I try to read only some bytes, LZMA2 will still uncompress the 1MB data and just return part of it. So it won't save any space nor time if we read only a few bytes.
The zimlib is designed to prevent unnecessary copies. So if you request an article, the data is uncompressed and the article point directly to the uncompressed data. This is one of the reasons, the data is not necessarily zero terminated. Mostly after the last byte of an article you can find the first byte of the next article.
The only situation, where it really may save some time is really for very large articles. If a article data is much larger than 2MB it may take multiple LZMA2 chunks and if you really need only the first MB, you don't really need to uncompress the whole chunk.
Zim is optimized for many small articles. And I feel, that this matches our target. I don't think we really need to do optimization for something else.