Dear all,
last weekend the openZIM team held its Developers Meeting for 2010. We planned to have two Developers Meeting per year, one in spring, one in fall. But as there were too many events in and around spring and all of us are volunteers we decided to better spend this weekend with our families. So this was our first and only Developers Meeting in 2010.
Participants: * Emmanuel Engelhard (Kiwix maintainer and organizer of this meeting) * Tommi Mäkitalo (openZIM main developer) * Christian Pühringer (WikiOnBoard maintainer) * Ralf Schmitt (developer of Pediapress) * Manuel Schneider
== Collections Extension for MediaWiki / Pediapress == Ralf from Pediapress visited us on Saturday to discuss with us the integration of the ZIM format into their Collections extension which is used on Wikimedia wikis to create books (printed on demand or exported as PDF or ODF). This idea was born on the Wikimedia Conference in April 2010 when we had a Wikipedia Offline meeting.
Tommi and Ralf exchanged thoughts about the implementation, Tommi made some changes to the zimwriter to make it more modular while Ralf wrote a Python wrapper for zimlib. In a test program Ralf was able to create his first ZIM file using Python and the new modularized zimwriter.
While Pediapress is in touch with the Wikimedia Foundation in order to make a project with them for the integration of ZIM into their software Ralf agreed to at least finish the test program in Python which can be downloaded here: http://github.com/schmir/pyzim
We hope that WMF and Pediapress come to an agreement so the actual integration can happen soon.
== Documentation of the ZIM File Format == As we made a lot of changes last year on the ZIM file format the documentation was partly outdated and partly not detailed enough. We spent roughly one day to improve the documentation, its size has doubled during this weekend.
A few parts of the format, notably the URL scheme and encoding, native category handling, per-article meta data and the HTML layout was not properly specified, so we spend some time to discuss the details and fix these issues. They need to be implemented now, maybe the last major changes to the ZIM format, but these changes don't break compatibility as they were already prepared in the past.
Another addition to the format - a new idea, but also backward-compatible - was the checksum. Newer ZIM files can now be checked by the new zimlib if they are corrupted. This was a request from users in Peru which have a hard time to download ZIM files and as they often have to resume transfers the files are likely to get corrupted. As they still work in most parts but cause errors in specific situations only it was hard to tell what the problem was.
http://openzim.org/ZIM_File_Format
== Reader Applications == News on specific readers, this is not a list of all available reader applications.
=== Windows / Linux === Kiwix is already wellknown and works perfectly on both GNU/Linux and Windows. It is a GUI application - a browser - to view ZIM files and also features a webserver to view ZIM in a network using a standard browser.
Emmanuel's roadmap is to create big ZIM files in the next months including all images for the selected articles. The next WP 1.0 release will be produced soon, the selection process is almost done. He bought a new server to do that, partly sponsored by Wikimedia Italia. The biggest effort is to create and optimize all the thumbnails of the images, so he is now trying to get a server at the datacenter of the Wikimedia Foundation where he can read the image files directly from the repositories.
http://www.kiwix.org/index.php/Main_Page
=== Symbian / Android === This year Christian published WikiOnBoard, a Symbian application to view ZIM files. It works very well on recent Nokia phones. On Sunday Christian presented us WikiOnBoard on Android - after he was able to get Qt installed on Android WikiOnBoard can be compiled on Android without any porting.
The plan is to implement a touch screen GUI to WikiOnBoard, so it can be used on never Nokia phones and on Android devices. The current version of WikiOnBoard only works with a keyboard which becomes less and less common. Once this is done WikiOnBoard will be published in the Nokia Ovi Store and Android market.
Christian asked for sponsorship for a new touch screen phone (about 150 EUR plus the help to get it via Germany as it is not available yet in many other countries). As the openZIM budget offers enough room for this it only depends on the team to approve this sponsorship.
http://wiki.github.com/cip/WikiOnBoard/
== MediaWiki API interface for zimwriter == Emmanuel and Tommi want to write a proof-of-concept of a zimwriter interface which uses the MediaWiki API directly to get all articles out of a wiki.
== ZIM File Library == We want to start working on our ZIM File library to make it easier for users to fine ZIM files.
This involves finding mirror sites and writing a download page which reads ZIM meta data, shows a searchable list of files and redirects downloads to the mirror servers.
== Events in 2011 == === Developers Meeting === August 2nd / 3rd
We decided to have the next and only openZIM meeting in 2011 during the Hacking Days which are prior Wikimania 2011 in Haifa, Israel.
Asuming that we will be able to get the same sponsorship as this year we agreed that in this case will reimburse travel costs for this meeting in Israel.
Registration to Wikimania will start in January, Manuel will send a notification to the openZIM mailing list. Everyone interested is urged to register and pay as soon as possible, as there will be a decent early bird discount and accommodation is part of the registration. Then everyone should book a flight. The receipts and invoices for registration and the flights will then go to Manuel who will take care of the reimbursement.
=== LinuxTag === May 11th - 14th
As always LinuxTag will take place in Berlin and we will be part of it. We plan to have one or two talks about openZIM, Tommi will do again a live-coding session to show how to use zimlib while Emmanuel offered a talk about the steps of how to create an offline version of Wikipedia, explaining selection, code cleanups, link rewriting etc.
== End == Ok, that's it. Quite a long report, but we actually did a lot of work during these two days. Additionally it is always very valuable to speak face-to-face and get to know new team members.
This last year was not as active concerning features and development as last year, but I consider it being a great progress nevertheless - this year we were able to stabilize the format, get the implementation and documentation right and reach out for new users such as Pediapress or WikiOnBoard.
The next year we will have to * finish the last open items concerning implementation * get the mobile readers released to the official markets / app stores * get the toolchains between MediaWiki and ZIM running, such as Collections extension and the MW API interface
Pictures: See here http://openzim.org/Developer_Meetings/2010-2
Thanks for the participants to be there and help getting the work done and to Wikimedia CH as our generous sponsor to make this possible.
Thank you for your attention,
Manuel Schneider
Manuel,
congratulations to a successful meeting. We'll be talking about the PediaPress project later this week and will post an update here when there are things to report. :-)
Cheers, Erik
Thanks, we are all looking forward seeing this happen ;-)
Am 19.10.2010 19:45, schrieb Erik Moeller:
congratulations to a successful meeting. We'll be talking about the PediaPress project later this week and will post an update here when there are things to report. :-)
Hi everyone!
My original dream for Wikipedia was to create and distribute a free encyclopedia to every single person on the planet, in their own language. This is the dream that has inspired hundreds of millions of readers, millions of editors, and thousands of developers.
Your work here is among the most important and most exciting things going on in the wiki world, for multiple reasons. First the obvious: this work will make it possible for the next billion people to see Wikipedia on offline devices and mobile devices much sooner than they would see it if they had to wait for Internet access.
But there is a more subtle effect - one thing we know is that writers love to have readers. Many people who speak languages of the developing world and have access to the Internet may not think to write in their home language, because they don't think many people will see it. As more people get access to read, there will be more motivations to write.
To sum up: you all have my heartfelt personal thinks. Your work is important, and I am honored to work by your side.
--Jimbo
On 10/19/10 7:42 PM, Manuel Schneider wrote:
Thanks, we are all looking forward seeing this happen ;-)
Am 19.10.2010 19:45, schrieb Erik Moeller:
congratulations to a successful meeting. We'll be talking about the PediaPress project later this week and will post an update here when there are things to report. :-)
Hi, everyone.
Great news and thanks for the detailed report! I'm particular happy about the format improvements and the toolchain progress. Sorry I couldn't make this meeting; I'll _definitely_ make the next one... :)
Cheers,
Asaf
On Tue, Oct 19, 2010 at 6:24 PM, Manuel Schneider < manuel.schneider@wikimedia.ch> wrote:
Dear all,
last weekend the openZIM team held its Developers Meeting for 2010. We planned to have two Developers Meeting per year, one in spring, one in fall. But as there were too many events in and around spring and all of us are volunteers we decided to better spend this weekend with our families. So this was our first and only Developers Meeting in 2010.
Participants:
- Emmanuel Engelhard (Kiwix maintainer and organizer of this meeting)
- Tommi Mäkitalo (openZIM main developer)
- Christian Pühringer (WikiOnBoard maintainer)
- Ralf Schmitt (developer of Pediapress)
- Manuel Schneider
== Collections Extension for MediaWiki / Pediapress == Ralf from Pediapress visited us on Saturday to discuss with us the integration of the ZIM format into their Collections extension which is used on Wikimedia wikis to create books (printed on demand or exported as PDF or ODF). This idea was born on the Wikimedia Conference in April 2010 when we had a Wikipedia Offline meeting.
Tommi and Ralf exchanged thoughts about the implementation, Tommi made some changes to the zimwriter to make it more modular while Ralf wrote a Python wrapper for zimlib. In a test program Ralf was able to create his first ZIM file using Python and the new modularized zimwriter.
While Pediapress is in touch with the Wikimedia Foundation in order to make a project with them for the integration of ZIM into their software Ralf agreed to at least finish the test program in Python which can be downloaded here: http://github.com/schmir/pyzim
We hope that WMF and Pediapress come to an agreement so the actual integration can happen soon.
== Documentation of the ZIM File Format == As we made a lot of changes last year on the ZIM file format the documentation was partly outdated and partly not detailed enough. We spent roughly one day to improve the documentation, its size has doubled during this weekend.
A few parts of the format, notably the URL scheme and encoding, native category handling, per-article meta data and the HTML layout was not properly specified, so we spend some time to discuss the details and fix these issues. They need to be implemented now, maybe the last major changes to the ZIM format, but these changes don't break compatibility as they were already prepared in the past.
Another addition to the format - a new idea, but also backward-compatible - was the checksum. Newer ZIM files can now be checked by the new zimlib if they are corrupted. This was a request from users in Peru which have a hard time to download ZIM files and as they often have to resume transfers the files are likely to get corrupted. As they still work in most parts but cause errors in specific situations only it was hard to tell what the problem was.
http://openzim.org/ZIM_File_Format
== Reader Applications == News on specific readers, this is not a list of all available reader applications.
=== Windows / Linux === Kiwix is already wellknown and works perfectly on both GNU/Linux and Windows. It is a GUI application - a browser - to view ZIM files and also features a webserver to view ZIM in a network using a standard browser.
Emmanuel's roadmap is to create big ZIM files in the next months including all images for the selected articles. The next WP 1.0 release will be produced soon, the selection process is almost done. He bought a new server to do that, partly sponsored by Wikimedia Italia. The biggest effort is to create and optimize all the thumbnails of the images, so he is now trying to get a server at the datacenter of the Wikimedia Foundation where he can read the image files directly from the repositories.
http://www.kiwix.org/index.php/Main_Page
=== Symbian / Android === This year Christian published WikiOnBoard, a Symbian application to view ZIM files. It works very well on recent Nokia phones. On Sunday Christian presented us WikiOnBoard on Android - after he was able to get Qt installed on Android WikiOnBoard can be compiled on Android without any porting.
The plan is to implement a touch screen GUI to WikiOnBoard, so it can be used on never Nokia phones and on Android devices. The current version of WikiOnBoard only works with a keyboard which becomes less and less common. Once this is done WikiOnBoard will be published in the Nokia Ovi Store and Android market.
Christian asked for sponsorship for a new touch screen phone (about 150 EUR plus the help to get it via Germany as it is not available yet in many other countries). As the openZIM budget offers enough room for this it only depends on the team to approve this sponsorship.
http://wiki.github.com/cip/WikiOnBoard/
== MediaWiki API interface for zimwriter == Emmanuel and Tommi want to write a proof-of-concept of a zimwriter interface which uses the MediaWiki API directly to get all articles out of a wiki.
== ZIM File Library == We want to start working on our ZIM File library to make it easier for users to fine ZIM files.
This involves finding mirror sites and writing a download page which reads ZIM meta data, shows a searchable list of files and redirects downloads to the mirror servers.
== Events in 2011 == === Developers Meeting === August 2nd / 3rd
We decided to have the next and only openZIM meeting in 2011 during the Hacking Days which are prior Wikimania 2011 in Haifa, Israel.
Asuming that we will be able to get the same sponsorship as this year we agreed that in this case will reimburse travel costs for this meeting in Israel.
Registration to Wikimania will start in January, Manuel will send a notification to the openZIM mailing list. Everyone interested is urged to register and pay as soon as possible, as there will be a decent early bird discount and accommodation is part of the registration. Then everyone should book a flight. The receipts and invoices for registration and the flights will then go to Manuel who will take care of the reimbursement.
=== LinuxTag === May 11th - 14th
As always LinuxTag will take place in Berlin and we will be part of it. We plan to have one or two talks about openZIM, Tommi will do again a live-coding session to show how to use zimlib while Emmanuel offered a talk about the steps of how to create an offline version of Wikipedia, explaining selection, code cleanups, link rewriting etc.
== End == Ok, that's it. Quite a long report, but we actually did a lot of work during these two days. Additionally it is always very valuable to speak face-to-face and get to know new team members.
This last year was not as active concerning features and development as last year, but I consider it being a great progress nevertheless - this year we were able to stabilize the format, get the implementation and documentation right and reach out for new users such as Pediapress or WikiOnBoard.
The next year we will have to
- finish the last open items concerning implementation
- get the mobile readers released to the official markets / app stores
- get the toolchains between MediaWiki and ZIM running, such as
Collections extension and the MW API interface
Pictures: See here http://openzim.org/Developer_Meetings/2010-2
Thanks for the participants to be there and help getting the work done and to Wikimedia CH as our generous sponsor to make this possible.
Thank you for your attention,
Manuel Schneider
-- Regards Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens Wikimedia CH - Association for the advancement of free knowledge www.wikimedia.ch
Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l