Offline-l October 2009

offline-l@lists.wikimedia.org

10 participants
14 discussions

[openZIM dev-l] Compression flag
by emmanuel＠engelhart.org 20 Oct '09

20 Oct '09

Hi, I help currently someone wanting to build his own ZIM parser. I know this is not necessary... but he wants to do it and I find this is interesting to have someone trying to do that. I have remarked that on the wiki nothing is written about the compression: * How do we know if a cluster is compressed or not ? * What are the possible value (for different compression methods) ? Regards Emmanuel

4 4

[openZIM dev-l] Kiwix issues
by Asaf Bartov 11 Oct '09

11 Oct '09

Hello again. Two Kiwix issues we see with the Hebrew ZIM (available at http://benyehuda.org/zim/hewiki_sep2009.zim) -- 1. There is no caption for images when hovering with the mouse over them. 2. There is no licensing information for images -- this seems to us to be violating the GFDL license, and is therefore a problem in distributing ZIM files. (I'll open a Bugzilla issue for each if they are acknowledged as issues.) We note that all this seems to be working fine in the Arabic ZIM you guys prepared some time ago. Perhaps we're missing something? Perhaps you'd be willing to try to create a Hebrew Wikipedia ZIM for us, just to see if it's something in our process or something in the handling of Hebrew? Many thanks in advance, Asaf Bartov Wikimedia Israel -- -- Asaf Bartov <asaf(a)forum2.org>

2 1

[openZIM dev-l] Memory consumption of mirroring scripts
by Asaf Bartov 11 Oct '09

11 Oct '09

Hello, everyone. While trying to add images to our Hebrew Wikipedia ZIM file, we ran into memory-consumption problems with the Kiwix mirroring scripts. Despite using a quad-core Linux machine with 4GB of physical memory, both * listAllImages.pl* and *mirrorMediawikiPages.pl* ran for a very long time but eventually crashed the perl process with an *out of memory* error, and indeed, while running, they consumed ever more memory, until they actually consumed the entire physical memory. Emmanuel said this shouldn't happen, but it does, and we'd like your advice on whether there's any point in attempting this again with a larger machine (perhaps an Amazon EC2 machine with more physical memory and faster CPUs) or whether we should try to find the bug in the scripts. Thanks in advance, Asaf Bartov Wikimedia Israel -- Asaf Bartov <asaf(a)forum2.org>

2 1

[openZIM dev-l] OpenZIM - reduce memory consumption?
by Mirko Vogt 02 Oct '09

02 Oct '09

Hey, I ported OpenZIM and it's dependencies to OpenWrt (not committed yet) - mainly, to get it running on the embedded device "Ben NanoNote" of qi-hardware. Unfortunately OpenZIM seems to consume an amount of memory the Ben isn't able to serve (it has 32MB of RAM). Even with a really minimalistic system (lightweight kernelconfig + busybox-based rootfs, having just a shell) and accessing the zimreader via a webbrowser running on another machine, OpenZIM is getting killed by the kernels out-of-memory killer after a few clicks. Is there a way of configuring / adjusting the zimreader to reduce memory consumption to get it running smoothly and stable on device such the ben NanoNote? Thanks a lot in advance - great work! Have a nice weekend, mirko -- This email address is used for mailinglist purposes only. Non-mailinglist emails will be dropped automatically. If you want to get in contact with me personally, please mail to: mirko.vogt <at> nanl <dot> de

4 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Offline-l October 2009