On Thu, Mar 26, 2009 at 8:51 PM, ERSEK Laszlo <lacos(a)elte.hu> wrote:
On 03/27/09 01:14, Brion Vibber wrote:
LZMA is nice and fast to decompress... but
*insanely* slower to
compress, and doesn't seem as parallelizable. :(
The xz file format should allow for "easy" parallelization, both when
compressing and decompressing; see
http://tukaani.org/xz/xz-file-format.txt
3. Block
3.1. Block Header
3.1.1. Block Header Size
3.1.3. Compressed Size
3.1.4. Uncompressed Size
3.1.6. Header Padding
3.3. Block Padding
At least in theory, this "length-prefixing" should make it fairly
straightforward to write a multi-threaded decompressor with a splitter
that can work from a pipe and is input-bound. I reckon the xz structure
will eventually prove useful even for distributed
compression/decompression.
lacos
It includes an index for random access too. Cool. I wonder what kind of
block size you'd need to get a compression ratio approaching that of 7z.