Hi
Last month a dozen of Kiwix developers have attend to our Spring 2017 hackathon in Lyon, France. All these developers represent the broad spectrum of Kiwix/openZIM activities, from the scrapping to the mobile apps, through core lib development.
During a whole week we all have lived and worked together to help the whole project to make a significant step forward. Here is the essence of what we have achieved to do:
*First Chrome extension*: this is a pure Web tech Kiwix reader. Still limited in features but working and already published on the Chrome Store https://chrome.google.com/webstore/detail/kiwix/donaljnlmapmngakoipdmehbfcio.... Give it a try and report any problem here: https://github.com/kiwix/kiwix-html5/issues
*First Firefox extension*: sharing the same code base as the Chrome extension, it is still in validation but should be publicly available soon. Impatient users might install a "nightly" from here http://download.kiwix.org/nightly/. Both Firefox/Chrome extensions have been initiated by the project Evopedia which has merged with Kiwix last year.
*Gutenberg scraper* improved: Developed 3 years ago, the original scrapper has been polished and the (big) list of tickets has been cleared. The code has been published in the Python repository https://pypi.python.org/pypi/gutenberg2zim and a Docker image has been created https://hub.docker.com/r/openzim/gutenberg/. New ZIM files are currently building and will be published soon.
*First release of Sotoki*: Sotoki is our StackExchange scrapper, which means that we can now propose ZIM files of all the SE web sites https://stackexchange.com/sites. The most famous of them being StackOverflow. The script has been published in Python https://pypi.python.org/pypi/sotoki and a Docker image created https://hub.docker.com/r/openzim/sotoki/. New ZIM files are currently building and will be published soon.
*Youtube scrapper has been improved*: Our Youtube scrapper has been a bit improved and published in Python repository and a Docker image has been created https://hub.docker.com/r/openzim/youtube/.
*openedx scraper stub*: First step have been done, we currently search a sponsor to work on this next Summer and release a first version at Fall.
*Phet scrapper UI improved*: The UI of the Phet ZIM file has been really improved. You can check it with the last versions of the files http://download.kiwix.org/zim/phet/. A docker image has been created https://hub.docker.com/r/openzim/phet/. Phets Android app will be updated in the next weeks https://play.google.com/store/apps/details?id=org.kiwix.kiwixcustomphet
*Android apps*: many improvements have been brought to the apps, the most visible concerning the continuous integration and the testing. But other massive changes have been done to have a better MVC approach and bookmarking system. Many of these improvements should be visible in Kiwix for Android 2.3 to be released in June.
*Fist Kiwix Apache module* has been created as an alternative of running kiwix-serve behind a reverse proxy. So far it does not provide as much features as kiwix-serve but is definitely easier to configure and really promising. Have a look to the git repository here https://github.com/kiwix/kiwix-apache
*MWoffliner improved*: it benefits now from an option to scrape remote Mediawiki, even if they do not have Parsoid installed. This is an opportunity for creating many new ZIM files - and we will. MWoffliner (as a script and a library) is now also available as a node.js package https://www.npmjs.com/package/mwoffliner and we have created a Docker image https://hub.docker.com/r/openzim/mwoffliner/. We have also created a Docker image for zimwriterfs https://hub.docker.com/r/openzim/zimwriterfs/.
*pibox-installer*: our intern, working on our future solution to allow everyone (also non-tech) to build there own customed offline wifi hotspot, was also part of the hackathon. He made a few progresses on his project during that week. Follow his daily work at https://framagit.org/ideascube/pibox-installer
*Khan Academy video scrapper almost finished*: This effort had been started in January last year. It has been almost finished during the hackathon. A few commits more and we will be able to generate ZIM files for Khan academy videos. Please be a bit patient if you need the ZIM files, here is the git repo https://github.com/openzim/kalite
*libzim improvements*: we have moved the Xapian fulltext engine technology from the Kiwix-lib to the libzim (as an option). At this occasion many improvements have been done regarding the search and the whole integration between the libzim, libkiwix and the multiple ports.
*First release of zip2zim* which is a small service able to convert a custom ZIP (with HTML/JS/CSS/pictures inside) in the corresponding ZIM file. The code is available here https://github.com/openzim/zip2zim and we have already created a Docker image https://hub.docker.com/r/openzim/zip2zim/. We plan to setup an online service soon.
Last but now least, the hackathon has concluded ~ 6 months long effort to reorganize all our git repositories. All the code we have ever produced and still produce is now available on GitHub in two organisations: * openZIM for the low level ZIM related readers and scrappers https://github.com/openzim/ * Kiwix for all the "high-level" Kiwix specific ports and solutions https://github.com/kiwix
This was a long list, thank you for reading it to the end, but a dozen of talented and committed developers can really to a lot of work during a whole week. If you want to get a bit more details about the hackathon have a look to that page: wiki.kiwix.org/wiki/Hackathon_Spring_2017
Next one will be this summer @Wikimania in Canada/US, here is the organisation page:http://wiki.kiwix.org/wiki/Hackathon_Wikimania_2017
Regards Emmanuel