Hi
We have published a new version of MWoffliner: the MediaWiki scraper. Version 1.9 is - like always - available here: https://www.npmjs.com/package/mwoffliner
This new release contains again big improvements in term of performance. MWoffliner 1.9 does require a lot less memory than in earlier version thanks to the release of new libzim 5.0.0 which happens a few days ago. Beside this a new experimental feature as been introduced: the scraping of categories.
Here is the detailed changelog: * NEW: Implement basic category scraping (#15) * UPDATE: Update node-libzim (and libzim) to improve memory usage (#706) * UPDATE: Increase default scrape "speed" (#744) * UPDATE: Report on failed article downloads () * UPDATE: Retry all failed media downloads at the end of scrape (#723) * UPDATE: Re-introduce --useCache option (#700) * UPDATE: Reduce system dependencies (#681) * UPDATE: Use native <details> element for collapsible sections in ZIM files (#677) * FIX: Update article title logic (#605) * FIX: Logging format improvements (#662) * FIX: Redis clearing improved (#732, #731, #448) * FIX: Text only article list main page background colour fixed (#728) * FIX: Resource relative url improvement (#726)
This is the fourth milestone of a few we have planned with the support of the WMF.
Like always, PR and bug reports are welcome at: https://github.com/openzim/mwoffliner
Regards Emmanuel
Unsubscribe bpowers@tech-serve.com
Sent from my iPhone
On Jun 5, 2019, at 10:25, Emmanuel Engelhart kelson@kiwix.org wrote:
Hi
We have published a new version of MWoffliner: the MediaWiki scraper. Version 1.9 is - like always - available here: https://www.npmjs.com/package/mwoffliner
This new release contains again big improvements in term of performance. MWoffliner 1.9 does require a lot less memory than in earlier version thanks to the release of new libzim 5.0.0 which happens a few days ago. Beside this a new experimental feature as been introduced: the scraping of categories.
Here is the detailed changelog:
- NEW: Implement basic category scraping (#15)
- UPDATE: Update node-libzim (and libzim) to improve memory usage (#706)
- UPDATE: Increase default scrape "speed" (#744)
- UPDATE: Report on failed article downloads ()
- UPDATE: Retry all failed media downloads at the end of scrape (#723)
- UPDATE: Re-introduce --useCache option (#700)
- UPDATE: Reduce system dependencies (#681)
- UPDATE: Use native <details> element for collapsible sections in ZIM
files (#677)
- FIX: Update article title logic (#605)
- FIX: Logging format improvements (#662)
- FIX: Redis clearing improved (#732, #731, #448)
- FIX: Text only article list main page background colour fixed (#728)
- FIX: Resource relative url improvement (#726)
This is the fourth milestone of a few we have planned with the support of the WMF.
Like always, PR and bug reports are welcome at: https://github.com/openzim/mwoffliner
Regards Emmanuel
-- Kiwix - Wikipedia Offline & more
- Web: http://www.kiwix.org
- Twitter: https://twitter.com/KiwixOffline
- more: http://www.kiwix.org/wiki/Communication
MediaWiki-l mailing list To unsubscribe, go to: https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
mediawiki-l@lists.wikimedia.org