Cross post.
---------- Forwarded message ---------- From: Emmanuel Engelhart kelson@kiwix.org Date: Tue, Jun 27, 2017 at 3:26 PM Subject: [Offline-l] [INFO] Few good news about mwoffliner To: Using Wikimedia projects and MediaWiki offline < offline-l@lists.wikimedia.org> Cc: "kiwix-developer@lists.sourceforge.net" < kiwix-developer@lists.sourceforge.net>
Hi
We rarely write here specifically about mwoffliner, even if this tool is time to time named by-the-way in threads. But these last months we have done many interesting improvements to this important tool and I thought it might be valuable to report quickly about them.
As a reminder, mwoffliner is a script which is thought to build a ZIM file from any (recent) online Mediawiki. It scraps a snapshot of the online wiki (HTML/JS/pictures/...) on your local disk.
Here is the list of recent improvements:
* We have introduce Parsoid as a local dependence, which means that even if a Mediawiki does not have Parsoid/Visual Editor installed, mwoffliner should have a chance now to build the ZIM file of it by running Parsoid locally.
* We have introduced the Parsoid mobile layout suppport which allows to build ZIM file with a similar layout as Wikipedia Mobile version. This is pretty much in beta and we plan first to use it only for Wikipedia.org.
* We have introduced the support of audio/video which means that now, like the pictures, they are mirrored too. Our first tests show that for Wikipedia it tends to multiply the size of the ZIM file by a factor four. As a consequence we won't use it directly everywhere. That said the feature is there and we will step-by-step introduce video in the ZIM files we are generating with mwoffliner.
* We have published mwoffliner (and mwmatrixoffliner) to the npmjs repository: https://www.npmjs.com/package/mwoffliner. Now everybody can install it easily (but you still need to take care about the dependences).
* We have made the script a bit more modular: you can call it like any other program but now you can also use it as a library in your own Javascript/Node.js scripts.
* We have moved the git repository to the openZIM organization on Github: https://github.com/openzim/mwoffliner. By moving all our scraper to the openZIM organization we hope to bring a bit of clarity between Kiwix and openZIM respective duties. Have a look to all other scrapers we have migrated to openZIM: https://github.com/openzim
mwoffliner is not a tool for everybody but it is really important to continue to improve it to provide quality ZIM files of Wikipedia, Wiktionary, ... So if you have Javascript skills please come to help us to prepare the next big steps forward https://github.com/openzim/mwoffliner/issues
Regards Emmanuel
-- Kiwix - Wikipedia Offline & more * Web: http://www.kiwix.org * Twitter: https://twitter.com/KiwixOffline * more: http://www.kiwix.org/wiki/Communication
_______________________________________________ Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l