http://bugs.openzim.org/show_bug.cgi?id=18
Summary: debian needs an init.d script
Product: openZIM
Version: unspecified
Platform: PC
OS/Version: Linux
Status: NEW
Severity: enhancement
Priority: P5
Component: zimreader
AssignedTo: tommi(a)tntnet.org
ReportedBy: andyr(a)wizzy.com
CC: dev-l(a)openzim.org
Estimated Hours: 0.0
One attached.
--
Configure bugmail: http://bugs.openzim.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
http://bugs.openzim.org/show_bug.cgi?id=26
Summary: CRC check
Product: openZIM
Version: unspecified
Platform: All
OS/Version: All
Status: NEW
Severity: enhancement
Priority: P3
Component: zimlib
AssignedTo: tommi(a)tntnet.org
ReportedBy: emmanuel(a)engelhart.org
CC: dev-l(a)openzim.org
Estimated Hours: 0.0
Downloaded ZIM files are unfortunately not always fully valid at the end of the
download stage. Consequently users may have a bad user experience without
having a chance to easily check if the file is valid.
It would be great to provide in the zimwriter/zimlib a way to check the zim
file integrity easily.
A way to achieve that could be:
* At the end of the ZIM file creation process (file is created) compute and
append at the end of the file a CRC (md5sum, sha1)
* In the zimlib add a method checkIntegrity() or something similar able to
compute the CRC from the file (excepting the CRC hash at the end) and make the
comparison.
Hash algorithm should be fast and reliable (md5?).
--
Configure bugmail: http://bugs.openzim.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
http://bugs.openzim.org/show_bug.cgi?id=29
Summary: SVN r333 does not always compile
Product: openZIM
Version: unspecified
Platform: PC
OS/Version: Linux
Status: NEW
Severity: major
Priority: P5
Component: zimlib
AssignedTo: tommi(a)tntnet.org
ReportedBy: emmanuel(a)engelhart.org
CC: dev-l(a)openzim.org
Estimated Hours: 0.0
On one of my computer (old Ubuntu Hardy x86_32) the zimlib trunk does not
compile anymore. In fact the configure script fails:
configure: creating ./config.status
config.status: error: cannot find input file: Makefile.in
See the log:
http://www.kiwix.org/tmp/config.log
The problem was introduced with r333, with r332 it works.
--
Configure bugmail: http://bugs.openzim.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
http://bugs.openzim.org/show_bug.cgi?id=30
Summary: zimdump -d fails on x86_32 with big ZIM files
Product: openZIM
Version: unspecified
Platform: PC
OS/Version: Linux
Status: NEW
Severity: critical
Priority: P5
Component: zimlib
AssignedTo: tommi(a)tntnet.org
ReportedBy: emmanuel(a)engelhart.org
CC: dev-l(a)openzim.org
Estimated Hours: 0.0
Using zimdump SVN r328 on Ubuntu Hardy x86_32.
Using the following file:
http://tmp.kiwix.org/zim/0.9/wikipedia_es_all_09_2010_beta1.zim
If get the following error:
$zimdump -D html wikipedia_es_all_09_2010_beta1.zim
error 22 seeking to -2147469848 in file wikipedia_es_all_09_2010_beta1.zim:
Invalid argument
$ echo $?
254
This error does not occurs under x86_64.
--
Configure bugmail: http://bugs.openzim.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
Hi,
I'd like to forward the original mail to you - it is not said in the
mail, but the supplied link points it out:
There is a project in Kenya using openZIM with Kiwix!
/Manuel
---------- Forwarded message ----------
From: *Abbas Mahmoud* <abbasjnr(a)hotmail.com <mailto:abbasjnr@hotmail.com>>
Date: Sat, Oct 23, 2010 at 4:00 PM
Subject: [Foundation-l] Proposed Wikimedia Project in Kenya
Hi folks,
As some of you may know, there exists a small bunch of Wikipedians in
Kenya. In the last couple of months, we have been discussing ways in
which we might increase Wikip\media awareness within Kenya. We then
decided to experiment by starting by using offline Wikipedia in primary
& secondary schools.
We still are at a very early stage: the framework/proposal is still
sketchy. Please check it out at
http://meta.wikimedia.org/wiki/Wikimedia_Kenya/Project_for_Kenyan_Schools and
give us your feedback. Feel free to edit, redaft or whatever you may
call it so that we can come up with a more concrete proposal.
Looking forward to getting your collaboration.
Yours,
m|Abbas.
http://bugs.openzim.org/show_bug.cgi?id=31
Summary: -s argument seems to be (newly) invalid
Product: openZIM
Version: unspecified
Platform: PC
OS/Version: Linux
Status: NEW
Severity: trivial
Priority: P5
Component: zimwriter
AssignedTo: tommi(a)tntnet.org
ReportedBy: emmanuel(a)engelhart.org
CC: dev-l(a)openzim.org
Estimated Hours: 0.0
In the trunk (r335) if you call zimwriterdb with -s 1024 (for example) you get
the usage(): no zim file with be generated.
$/usr/local/bin/zimwriterdb -s 1024 --db "postgresql:dbname=wiki2b user=kiwix
port=5432" titi
create zim file from db
usage: /usr/local/bin/zimwriterdb [options] output-filename
".zim" is appended to the output filename if not already given.
options:
-s <number> specify chunk size for compression in kB (default 1024)
--db <dburl> specify a db source (default: postgresql:dbname=zim,
tntdb is used here)
-Z <articlefile> create a fulltext index for specified article
-S <words> search in zim file for articles
-I <articlefile> article file for search
-X <indexfile> use indexfile as full text search index
-D <directory> create zim file from directory content
additional options for full text indexer:
-T <file> trivial words file for full text index (a text file with
words, which are not indexed)
-M <number> memory factor (default 64, smaller factors reduce memory
usage but makes indexer slower,
try smaller values when you run out of memory)
-t <filename> temporary file name (default zimwriter.tmp)
But if you call the same command line but without "-s 1024"... it works.
$ /usr/local/bin/zimwriterdb --db "postgresql:dbname=wiki2b user=kiwix
port=5432" titi
create zim file from db
create directory entries
collect articles
sort 6 directory entries (aid)
remove invalid redirects from 6 directory entries
sort 6 directory entries (url)
set index
translate redirect aid to index
6 directory entries created
create title index
6 title index created
create clusters
0% ready
10% ready
20% ready
30% ready
40% ready
50% ready
60% ready
70% ready
80% ready
1 clusters created
fill header
write zimfile
ready
zim file ready
--
Configure bugmail: http://bugs.openzim.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
A heads-up from Jimmy Wales:
-------- Original-Nachricht --------
Betreff: Re: [Offline-l] openZIM Developers Meeting 2010 in Strasbourg
Datum: Thu, 21 Oct 2010 06:53:28 +0100
Von: Jimmy Wales <jwales(a)wikia-inc.com>
Hi everyone!
My original dream for Wikipedia was to create and distribute a free
encyclopedia to every single person on the planet, in their own
language. This is the dream that has inspired hundreds of millions of
readers, millions of editors, and thousands of developers.
Your work here is among the most important and most exciting things
going on in the wiki world, for multiple reasons. First the obvious:
this work will make it possible for the next billion people to see
Wikipedia on offline devices and mobile devices much sooner than they
would see it if they had to wait for Internet access.
But there is a more subtle effect - one thing we know is that writers
love to have readers. Many people who speak languages of the developing
world and have access to the Internet may not think to write in their
home language, because they don't think many people will see it. As
more people get access to read, there will be more motivations to write.
To sum up: you all have my heartfelt personal thinks. Your work is
important, and I am honored to work by your side.
--Jimbo
On 10/19/10 7:42 PM, Manuel Schneider wrote:
> Thanks, we are all looking forward seeing this happen ;-)
>
> Am 19.10.2010 19:45, schrieb Erik Moeller:
>> congratulations to a successful meeting. We'll be talking about the
>> PediaPress project later this week and will post an update here when
>> there are things to report. :-)
_______________________________________________
Offline-l mailing list
Offline-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/offline-l
Dear all,
last weekend the openZIM team held its Developers Meeting for 2010. We
planned to have two Developers Meeting per year, one in spring, one in
fall. But as there were too many events in and around spring and all of
us are volunteers we decided to better spend this weekend with our
families. So this was our first and only Developers Meeting in 2010.
Participants:
* Emmanuel Engelhard (Kiwix maintainer and organizer of this meeting)
* Tommi Mäkitalo (openZIM main developer)
* Christian Pühringer (WikiOnBoard maintainer)
* Ralf Schmitt (developer of Pediapress)
* Manuel Schneider
== Collections Extension for MediaWiki / Pediapress ==
Ralf from Pediapress visited us on Saturday to discuss with us the
integration of the ZIM format into their Collections extension which is
used on Wikimedia wikis to create books (printed on demand or exported
as PDF or ODF). This idea was born on the Wikimedia Conference in April
2010 when we had a Wikipedia Offline meeting.
Tommi and Ralf exchanged thoughts about the implementation, Tommi made
some changes to the zimwriter to make it more modular while Ralf wrote a
Python wrapper for zimlib. In a test program Ralf was able to create his
first ZIM file using Python and the new modularized zimwriter.
While Pediapress is in touch with the Wikimedia Foundation in order to
make a project with them for the integration of ZIM into their software
Ralf agreed to at least finish the test program in Python which can be
downloaded here:
http://github.com/schmir/pyzim
We hope that WMF and Pediapress come to an agreement so the actual
integration can happen soon.
== Documentation of the ZIM File Format ==
As we made a lot of changes last year on the ZIM file format the
documentation was partly outdated and partly not detailed enough. We
spent roughly one day to improve the documentation, its size has doubled
during this weekend.
A few parts of the format, notably the URL scheme and encoding, native
category handling, per-article meta data and the HTML layout was not
properly specified, so we spend some time to discuss the details and fix
these issues. They need to be implemented now, maybe the last major
changes to the ZIM format, but these changes don't break compatibility
as they were already prepared in the past.
Another addition to the format - a new idea, but also
backward-compatible - was the checksum. Newer ZIM files can now be
checked by the new zimlib if they are corrupted. This was a request from
users in Peru which have a hard time to download ZIM files and as they
often have to resume transfers the files are likely to get corrupted. As
they still work in most parts but cause errors in specific situations
only it was hard to tell what the problem was.
http://openzim.org/ZIM_File_Format
== Reader Applications ==
News on specific readers, this is not a list of all available reader
applications.
=== Windows / Linux ===
Kiwix is already wellknown and works perfectly on both GNU/Linux and
Windows. It is a GUI application - a browser - to view ZIM files and
also features a webserver to view ZIM in a network using a standard browser.
Emmanuel's roadmap is to create big ZIM files in the next months
including all images for the selected articles. The next WP 1.0 release
will be produced soon, the selection process is almost done. He bought a
new server to do that, partly sponsored by Wikimedia Italia. The biggest
effort is to create and optimize all the thumbnails of the images, so he
is now trying to get a server at the datacenter of the Wikimedia
Foundation where he can read the image files directly from the repositories.
http://www.kiwix.org/index.php/Main_Page
=== Symbian / Android ===
This year Christian published WikiOnBoard, a Symbian application to view
ZIM files. It works very well on recent Nokia phones. On Sunday
Christian presented us WikiOnBoard on Android - after he was able to get
Qt installed on Android WikiOnBoard can be compiled on Android without
any porting.
The plan is to implement a touch screen GUI to WikiOnBoard, so it can be
used on never Nokia phones and on Android devices. The current version
of WikiOnBoard only works with a keyboard which becomes less and less
common. Once this is done WikiOnBoard will be published in the Nokia Ovi
Store and Android market.
Christian asked for sponsorship for a new touch screen phone (about 150
EUR plus the help to get it via Germany as it is not available yet in
many other countries). As the openZIM budget offers enough room for this
it only depends on the team to approve this sponsorship.
http://wiki.github.com/cip/WikiOnBoard/
== MediaWiki API interface for zimwriter ==
Emmanuel and Tommi want to write a proof-of-concept of a zimwriter
interface which uses the MediaWiki API directly to get all articles out
of a wiki.
== ZIM File Library ==
We want to start working on our ZIM File library to make it easier for
users to fine ZIM files.
This involves finding mirror sites and writing a download page which
reads ZIM meta data, shows a searchable list of files and redirects
downloads to the mirror servers.
== Events in 2011 ==
=== Developers Meeting ===
August 2nd / 3rd
We decided to have the next and only openZIM meeting in 2011 during the
Hacking Days which are prior Wikimania 2011 in Haifa, Israel.
Asuming that we will be able to get the same sponsorship as this year we
agreed that in this case will reimburse travel costs for this meeting in
Israel.
Registration to Wikimania will start in January, Manuel will send a
notification to the openZIM mailing list. Everyone interested is urged
to register and pay as soon as possible, as there will be a decent early
bird discount and accommodation is part of the registration. Then
everyone should book a flight. The receipts and invoices for
registration and the flights will then go to Manuel who will take care
of the reimbursement.
=== LinuxTag ===
May 11th - 14th
As always LinuxTag will take place in Berlin and we will be part of it.
We plan to have one or two talks about openZIM, Tommi will do again a
live-coding session to show how to use zimlib while Emmanuel offered a
talk about the steps of how to create an offline version of Wikipedia,
explaining selection, code cleanups, link rewriting etc.
== End ==
Ok, that's it. Quite a long report, but we actually did a lot of work
during these two days. Additionally it is always very valuable to speak
face-to-face and get to know new team members.
This last year was not as active concerning features and development as
last year, but I consider it being a great progress nevertheless - this
year we were able to stabilize the format, get the implementation and
documentation right and reach out for new users such as Pediapress or
WikiOnBoard.
The next year we will have to
* finish the last open items concerning implementation
* get the mobile readers released to the official markets / app stores
* get the toolchains between MediaWiki and ZIM running, such as
Collections extension and the MW API interface
Pictures: See here http://openzim.org/Developer_Meetings/2010-2
Thanks for the participants to be there and help getting the work done
and to Wikimedia CH as our generous sponsor to make this possible.
Thank you for your attention,
Manuel Schneider
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
Hi Manuel,
Thanks for the prompt reply.
I know there's an unofficial Qt port for Android on Google Code, but I don't
think there will be an official port for Qt on the Android platform.
If the Qt port could support to interact with the Android UI system, then it
would be great. But I'm afraid it might be not due to the limitation of the
Android system.
I would be interested to know how it works in WikiOnBoard for Android since
I have past experiences on the Android system.
Regards,
Ted Chien
-- Sent from my Nexus One.
2010-10-20 上午2:31 於 "Manuel Schneider" <manuel.schneider(a)wikimedia.ch> 寫道：
Dear all,
last weekend the openZIM team held its Developers Meeting for 2010. We
planned to have two Developers Meeting per year, one in spring, one in
fall. But as there were too many events in and around spring and all of
us are volunteers we decided to better spend this weekend with our
families. So this was our first and only Developers Meeting in 2010.
Participants:
* Emmanuel Engelhard (Kiwix maintainer and organizer of this meeting)
* Tommi Mäkitalo (openZIM main developer)
* Christian Pühringer (WikiOnBoard maintainer)
* Ralf Schmitt (developer of Pediapress)
* Manuel Schneider
== Collections Extension for MediaWiki / Pediapress ==
Ralf from Pediapress visited us on Saturday to discuss with us the
integration of the ZIM format into their Collections extension which is
used on Wikimedia wikis to create books (printed on demand or exported
as PDF or ODF). This idea was born on the Wikimedia Conference in April
2010 when we had a Wikipedia Offline meeting.
Tommi and Ralf exchanged thoughts about the implementation, Tommi made
some changes to the zimwriter to make it more modular while Ralf wrote a
Python wrapper for zimlib. In a test program Ralf was able to create his
first ZIM file using Python and the new modularized zimwriter.
While Pediapress is in touch with the Wikimedia Foundation in order to
make a project with them for the integration of ZIM into their software
Ralf agreed to at least finish the test program in Python which can be
downloaded here:
http://github.com/schmir/pyzim
We hope that WMF and Pediapress come to an agreement so the actual
integration can happen soon.
== Documentation of the ZIM File Format ==
As we made a lot of changes last year on the ZIM file format the
documentation was partly outdated and partly not detailed enough. We
spent roughly one day to improve the documentation, its size has doubled
during this weekend.
A few parts of the format, notably the URL scheme and encoding, native
category handling, per-article meta data and the HTML layout was not
properly specified, so we spend some time to discuss the details and fix
these issues. They need to be implemented now, maybe the last major
changes to the ZIM format, but these changes don't break compatibility
as they were already prepared in the past.
Another addition to the format - a new idea, but also
backward-compatible - was the checksum. Newer ZIM files can now be
checked by the new zimlib if they are corrupted. This was a request from
users in Peru which have a hard time to download ZIM files and as they
often have to resume transfers the files are likely to get corrupted. As
they still work in most parts but cause errors in specific situations
only it was hard to tell what the problem was.
http://openzim.org/ZIM_File_Format
== Reader Applications ==
News on specific readers, this is not a list of all available reader
applications.
=== Windows / Linux ===
Kiwix is already wellknown and works perfectly on both GNU/Linux and
Windows. It is a GUI application - a browser - to view ZIM files and
also features a webserver to view ZIM in a network using a standard browser.
Emmanuel's roadmap is to create big ZIM files in the next months
including all images for the selected articles. The next WP 1.0 release
will be produced soon, the selection process is almost done. He bought a
new server to do that, partly sponsored by Wikimedia Italia. The biggest
effort is to create and optimize all the thumbnails of the images, so he
is now trying to get a server at the datacenter of the Wikimedia
Foundation where he can read the image files directly from the repositories.
http://www.kiwix.org/index.php/Main_Page
=== Symbian / Android ===
This year Christian published WikiOnBoard, a Symbian application to view
ZIM files. It works very well on recent Nokia phones. On Sunday
Christian presented us WikiOnBoard on Android - after he was able to get
Qt installed on Android WikiOnBoard can be compiled on Android without
any porting.
The plan is to implement a touch screen GUI to WikiOnBoard, so it can be
used on never Nokia phones and on Android devices. The current version
of WikiOnBoard only works with a keyboard which becomes less and less
common. Once this is done WikiOnBoard will be published in the Nokia Ovi
Store and Android market.
Christian asked for sponsorship for a new touch screen phone (about 150
EUR plus the help to get it via Germany as it is not available yet in
many other countries). As the openZIM budget offers enough room for this
it only depends on the team to approve this sponsorship.
http://wiki.github.com/cip/WikiOnBoard/
== MediaWiki API interface for zimwriter ==
Emmanuel and Tommi want to write a proof-of-concept of a zimwriter
interface which uses the MediaWiki API directly to get all articles out
of a wiki.
== ZIM File Library ==
We want to start working on our ZIM File library to make it easier for
users to fine ZIM files.
This involves finding mirror sites and writing a download page which
reads ZIM meta data, shows a searchable list of files and redirects
downloads to the mirror servers.
== Events in 2011 ==
=== Developers Meeting ===
August 2nd / 3rd
We decided to have the next and only openZIM meeting in 2011 during the
Hacking Days which are prior Wikimania 2011 in Haifa, Israel.
Asuming that we will be able to get the same sponsorship as this year we
agreed that in this case will reimburse travel costs for this meeting in
Israel.
Registration to Wikimania will start in January, Manuel will send a
notification to the openZIM mailing list. Everyone interested is urged
to register and pay as soon as possible, as there will be a decent early
bird discount and accommodation is part of the registration. Then
everyone should book a flight. The receipts and invoices for
registration and the flights will then go to Manuel who will take care
of the reimbursement.
=== LinuxTag ===
May 11th - 14th
As always LinuxTag will take place in Berlin and we will be part of it.
We plan to have one or two talks about openZIM, Tommi will do again a
live-coding session to show how to use zimlib while Emmanuel offered a
talk about the steps of how to create an offline version of Wikipedia,
explaining selection, code cleanups, link rewriting etc.
== End ==
Ok, that's it. Quite a long report, but we actually did a lot of work
during these two days. Additionally it is always very valuable to speak
face-to-face and get to know new team members.
This last year was not as active concerning features and development as
last year, but I consider it being a great progress nevertheless - this
year we were able to stabilize the format, get the implementation and
documentation right and reach out for new users such as Pediapress or
WikiOnBoard.
The next year we will have to
* finish the last open items concerning implementation
* get the mobile readers released to the official markets / app stores
* get the toolchains between MediaWiki and ZIM running, such as
Collections extension and the MW API interface
Pictures: See here http://openzim.org/Developer_Meetings/2010-2
Thanks for the participants to be there and help getting the work done
and to Wikimedia CH as our generous sponsor to make this possible.
Thank you for your attention,
Manuel Schneider
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch