Hi,
I apreciate to see this enthusiastic discussion.
My suggestion is:
* we have lzma and bzip2 support in zimlib
* the zimreader can use both algorithms
* as long as we didn't approve lzma to work on all platforms and systems, the
zimwriter uses bzip2 by default
* as soon as we can approve lzma we switch the zimwriter's default to lzma
* for experiments, development, porting etc. someone who knows what he does he can use
lzma anyway
* after the approval of lzma we will wait for some time until we are sure that no more
bzip2-compressed are being made, we drop bzip2-support completely
This way we can make the adoption of lzma smooth. The reader will still be backwards
compatible during the transition period.
Have a nice Christmas,
/Manuel
-- Urspr. Mitt. --
Betreff: Re: [openZIM dev-l] Update zim file format in trunk
Von: Tommi Mäkitalo <tommi(a)tntnet.org>
Datum: 26.12.2009 19:35
Am Samstag, 26. Dezember 2009 18:49:20 schrieb Emmanuel Engelhart:
Tommi Mäkitalo a écrit :
Am Samstag, 26. Dezember 2009 14:19:26 schrieb
Emmanuel Engelhart:
Tommi Mäkitalo a écrit :
...
The next
step is to remove support for zlib and bzip2.
Currently I'm working on porting the new file format to the nano note.
Unfortunately I'm stuck with porting xz (the lzma-library) to openwrt.
There are some difficulties there, but I'm sure, they can be resolved.
I have already asked for help at the openwrt developer list but have
not yet received any answers.
That's why I asked to be careful by removing bzip2 and/or zlib.
It's essential to have at least at any moment a working solution for
every arch/system. I have for example not checked if xz works good with
Windows.
Emmanuel
We want to define a standard format. And if you create zim files with
lzma compression we need to make sure, we can read the file on every
system, we support. It makes no sense to have some zim files with bzip2
compression, which work on windows and other zim files with lzma
compresion which don't. We have to commit us to one compression method to
prevent fragmentation. Or do you want to offer zim files for ubuntu and
zim files for windows and zim files for fedora and zim files for freebsd?
I prefer having *now* a solution with 3 compression algorithms and
knowing one of them does not always work than having a format which is
totally unusable in a few use cases.
So, we will try to have lzma working everywhere... but as long as this
is not sure, it would be preferable IMO to be able to say "look you can
use bzip2" than "sorry zimlib is not for you, it does not work currently
for your use case".
We have to choice our disadvantage:
* Remove bzip2 and having the risk to discover portability issues
* Keeping the bzip2/gzip dependence a little bit longer
... in any case this is not so critical... in the worth case we will
have to work hard to fix the portability issues... and I hope we will
because we want the zimlib being portable.
Emmanuel
How do you want to create your zim files? With lzma? With bzip2? With zlib? If
you sometimes use one and sometimes the other your zim files will work
sometimes on one platform sometimes on the other. Either we use lzma or we
don't. Otherwise we create lzma compresed zim files, which do not work
everywhere.
If you offer lzma compressed zim files for download, not dropping bzip2 support
do not solve any portability issue. We just get a zimlib which does not work
with your zim files. And that's really bad.
We should work on the portability of xz. As I read xz works on windows. It can
be compiled using mingw. So this should not really be an issue.
Tommi
_______________________________________________
dev-l mailing list
dev-l(a)openzim.org
https://intern.openzim.org/mailman/listinfo/dev-l