Hi,
sorry for my late answer. I was on holiday.
we are very interested in adopting openzim wherever possible. A nice handheld
device or phone would be a great showcase and of.
The ZimReader actually was once compiled on a openmoko device and it worked.
How can we help?
Tommi
On Mittwoch, 22. Juli 2009 17:49:42 you wrote:
> Hi Tommi,
>
> Sorry for using your tntnet address for an OpenZim related mail :)
>
> I am Mirko and part of the Qi Hardware Team.
>
> This Monday we launched our company [1] and announced our first product
> "Ben NanoNote", a fully opened multifunction device [3].
>
> As you can see on the product page the Ben NanoNote does not have a
> build-in RF chip. We therefore think it would be ideal for a project such
> as OpenZim and as a device for an offline version of Wikipedia.
>
> We are very interested to hear what you thoughts are. We have a mailing
> list dedicated for development [3].
>
> We see an OpenZim client as a possibility for one of the standard
> applications shipping in our image, but would need guidance. If you think
> this would be doable and something you would be interested in, let me know.
>
> Regards,
> /Mirko
>
>
> [1]
> http://linux.com/news/embedded-mobile/mids/29263-openmoko-layoffs-lead-to-n
>ew-open-hardware-venture [2]
> http://www.qi-hardware.com/products/ben-nanonote/
> [3] http://lists.qi-hardware.com/cgi-bin/mailman/listinfo/developer
Hi all!
Since an offline encyclopedia is most valuable on mobile
devices, I worked on building ZimReader for the Openmoko
Neo.
As a start I did a native build on a Debian (hackable:1 [3])
driven Openmoko Freerunner. After fixing some minor build
issue ([1], [2]), I can now browse Wikipedia offline on the
phone.
ZimReader performs amazingly well on the limited resources
of the device. I use it in conjunction with woosh, a browser
that comes with hackable:1. The content resides on SD-card.
A full-text search takes around 5s, mostly even less.
Since I'm not too much into packaging, I wrapped up a tar
ball for those interested in binaries [4]. Please see the
Readme for installation instructions.
Note: The binaries will most likely _not_ run on OE-based
distributions (OM2009, SHR). They have only been tested
on hackable1. I'll try to supply packages for other distros, as
soon as I find the time.
Thank's a lot to the openzim team for their great work!
Cheers, Marc
[1] http://bugs.openzim.org/show_bug.cgi?id=6
[2] http://bugs.openzim.org/show_bug.cgi?id=5
[3] http://www.hackable1.org/wiki/Main_Page
[4] http://www.gut-informierte-kreise.de/openmoko/openzim/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I have achieved to compile a first version of a zimlib.lib (static
library under Windows). I do not really know if I can use it... but at
least it compiles.
I want to give you here the modifications I have done to achieve that.
Tommi, it would be great to include them (or others with the same
purpose) into the SVN.
Index: include/zim/zim.h
===================================================================
- --- include/zim/zim.h (revision 236)
+++ include/zim/zim.h (working copy)
@@ -22,6 +22,12 @@
#include <stdint.h>
+#ifdef _WIN32
+typedef unsigned __int64 uint64_t;
+typedef signed __int8 int8_t;
+typedef signed __int32 int32_t;
+#endif
+
namespace zim
{
typedef uint32_t size_type;
Index: src/search.cpp
===================================================================
- --- src/search.cpp (revision 236)
+++ src/search.cpp (working copy)
@@ -106,7 +106,7 @@
// weight position of words in the document
if (Search::getWeightPos())
for (itp = posList.begin(); itp != posList.end(); ++itp)
- - priority += Search::getWeightPos() / pow(1.01, itp->first);
+ priority += Search::getWeightPos() / pow((double)1.01,
(double)itp->first); // here is a ambiguity between to pow() functions
(pow(double, int) or pow(double, double))
if (Search::getWeightPosRel())
for (itp = posList.begin(); itp != posList.end(); ++itp)
Index: src/bunzip2stream.cpp
===================================================================
- --- src/bunzip2stream.cpp (revision 236)
+++ src/bunzip2stream.cpp (working copy)
@@ -113,7 +113,8 @@
// there is data already available
// read compressed data from source into ibuffer
log_debug("in_avail=" << sinksource->in_avail());
- - stream.avail_in = sinksource->sgetn(ibuffer(),
std::min(sinksource->in_avail(), ibuffer_size()));
+
+ stream.avail_in = sinksource->sgetn(ibuffer(),
min(sinksource->in_avail(), ibuffer_size())); // compile genereates an
error if you have the "std::", do not know exactly why.
}
else
{
Index: src/uuid.cpp
===================================================================
- --- src/uuid.cpp (revision 236)
+++ src/uuid.cpp (working copy)
@@ -20,12 +20,31 @@
#include <zim/uuid.h>
#include <iostream>
#include <time.h>
- -#include <sys/time.h>
+#include <zim/zim.h> // necessary to have the new types
#include "log.h"
#ifdef WITH_CXXTOOLS
#include <cxxtools/md5stream.h>
#endif
+#if defined(_MSC_VER) || defined(__MINGW32__)
+# include <time.h>
+# include <Windows.h>
+#else
+# include <sys/time.h>
+#endif
+
+#if defined(_MSC_VER) || defined(__MINGW32__)
+int gettimeofday(struct timeval* tp, void* tzp) {
+ DWORD t;
+ t = timeGetTime();
+ tp->tv_sec = t / 1000;
+ tp->tv_usec = t % 1000;
+ /* 0 indicates that the call succeeded. */
+ return 0;
+}
+#endif // time related function/headers are differents und win32
+
+
log_define("zim.uuid")
namespace zim
@@ -62,7 +81,12 @@
*reinterpret_cast<int32_t*>(ret.data) =
reinterpret_cast<int32_t>(&ret);
*reinterpret_cast<int32_t*>(ret.data + 4) =
static_cast<int32_t>(tv.tv_sec);
*reinterpret_cast<int32_t*>(ret.data + 8) =
static_cast<int32_t>(tv.tv_usec);
+
+ #if defined(_MSC_VER)
+ *reinterpret_cast<int32_t*>(ret.data + 12) =
static_cast<int32_t>(GetCurrentProcessId());
+ #else
*reinterpret_cast<int32_t*>(ret.data + 12) =
static_cast<int32_t>(getpid()); // getpid() is GetCurrentProcessId()
under win32
+ #endif
#endif
Index: src/qunicode.cpp
===================================================================
- --- src/qunicode.cpp (revision 236)
+++ src/qunicode.cpp (working copy)
@@ -18,6 +18,7 @@
*/
#include <zim/qunicode.h>
+#include <zim/zim.h> // for the types compatibilty
#include "log.h"
#include <iostream>
#include <sstream>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkpwmusACgkQn3IpJRpNWtN4YQCfaYy9rGS5nrWyYqb5Nk8bs6eo
hgsAoKTPFo/75jY1GhUZM+iWGbGDP2dg
=goMa
-----END PGP SIGNATURE-----
Dear developers team,
I am happy to forward you some mails concerning new bugs we have received. It
lloks like we have got some more developers looking into the software.
Please see the attachments.
Cheers,
Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
a few weeks ago, Frederico reported me a small bug in a special HTML
page in a test ZIM file in Italian:
https://sourceforge.net/tracker/?func=detail&aid=2798771&group_id=175508&at…
The symptom is that references (<references/>) are not always rendered.
After auditing a little bit the Mediawiki code, it seems that this is a
Mediawiki bug. I have reported it there:
https://bugzilla.wikimedia.org/show_bug.cgi?id=19807
I can not fix it, this is too complicated for me... but maybe Mediawiki
hackers listening there have ideas ;)
Regards
Emmanuel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkphxUEACgkQn3IpJRpNWtPX+ACgtP1QDNuRlZxz8FV2Dh/YChHK
FI8An0clv19tJo8leLNQkGSBd2vAIf2V
=t4UO
-----END PGP SIGNATURE-----
Hi,
I have this week end prepared a ZIM file with the last Wikipedia for Schools selection (from SOS children's Villages UK).
http://tmp.kiwix.org/zim/schools-wikipedia-full-20081023-rc1.zim
Content should be exactly the same as before but the size is 30% smaller.
After a few minutes indexing with Kiwix the whole content are available with the search engine.
At this occasion, I have implemented the ignore feature for HTML content with the NOINDEX meta tag. This is important to avoid indexing index pages, it would be maybe good to do also that with the zimindexer (if not already done).
I have maybe someone who is ready to pay a little bit to help to port Kiwix and libzim under Windows... nothing sure but I hope it will help to find someone to do this job quickly as we ;)
I have also made a feature request to add a few fields in the header, please have a look at it:
http://bugs.openzim.org/show_bug.cgi?id=4
Regards
Emmanuel
Clarification:
This last message was by Rotem, a fellow WM-IL member helping me with the
embedding of the Hebrew Wikipedia in the One Computer Per Child project.
He is reporting issues with Kiwix and the ZIM file I created last week.
Regarding size: Size is important, because we intend to add images (the
300MB ZIM file is the complete Hebrew Wikipedia text, but no pictures). We
are hoping to have at least 5GB reserved for us in those One Computer Per
Child machines we are to install on, but we may be forced to make do with
3GB. So every MB saved from the index, is another MB available for
images...
Asaf Bartov
Wikimedia Israel
On Mon, Jul 6, 2009 at 3:58 PM, Rotem Simha <hidroo(a)gmail.com> wrote:
> * there are some errors in links of files and special pages
> examples
> קובץ:Nuvola_apps_important.svg<http://commons.wikimedia.org/wiki/File:Nuvola_apps_important.svg> link
> to ויקיפדיה:מיזמי ויקיפדיה/מיזם ערכים ללא תמונות/קטגוריות/ספורטאים איטלקים(wikipedia:wikipedia projects\ articles without images\categories\Sports
> people from Italy)
> מיוחד:אקראי (Special:Random) > 15 במאי (may 15)
> מיוחד:שינויים אחרונים (Special:RecentChanges) > 10_באוגוסט
>
> * size is important because we intend to add images
>
> 2009/7/6 <dev-l-request(a)openzim.org>
>
>> Send dev-l mailing list submissions to
>> dev-l(a)openzim.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> https://intern.openzim.org/mailman/listinfo/dev-l
>> or, via email, send a message with subject or body 'help' to
>> dev-l-request(a)openzim.org
>>
>> You can reach the person managing the list at
>> dev-l-owner(a)openzim.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of dev-l digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Kiwix index size (Asaf Bartov)
>> 2. Re: Kiwix index size (Manuel Schneider)
>> 3. Re: Kiwix index size (Emmanuel Engelhart)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Sun, 5 Jul 2009 19:18:57 +0300
>> From: Asaf Bartov <asaf.bartov(a)gmail.com>
>> Subject: [openZIM dev-l] Kiwix index size
>> To: dev-l(a)openzim.org
>> Message-ID:
>> <50a20d900907050918r3fcff23l275c67690ed7fc20(a)mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Hi, everyone.
>>
>> When running Kiwix's indexer on the ZIM file I had created from the Hebrew
>> Wikipedia last week, the Kiwix data directory ran up to a total of 31
>> items,
>> totalling 2.3 GB. The ZIM file itself is ~300MB. Does this proportion
>> make
>> sense?
>>
>> Detailed ls output attached.
>>
>> Thanks in advance,
>>
>> Asaf Bartov
>> --
>> Asaf Bartov <asaf(a)forum2.org>
>>
Hello, Pascal.
Thanks for the pointer about OkaWix!
I've downloaded it and tried it with the Hebrew Wikipedia. Initial
findings:
- excellent rendering, including directionality.
- categories don't seem to work
I'm downloading the pictures now...
I note it's using Zeno files. Is it planned to use ZIM files in the future?
A.
On Mon, Jul 6, 2009 at 7:15 PM, Pascal Martin <pmartin(a)linterweb.fr> wrote:
> So do you try okawix with the hebrew contents ?
>
>
About the dead links, few thinks:
* Are you sure the problem is not at the source (HTML files)
* the zimwriter does not check if all links in all HTML pages are OK
* it seems that the libzim returns a bad content if the content does not exist (Tommi can you confirm?). Should returns nothing or an error code IMO.
Emmanuel
Le lun 06/07/09 14:58, "Rotem Simha" hidroo(a)gmail.com a écrit:
> * there are some errors in links of files and special pages
> examples
> קובץ:Nuvola_apps_important.svg [1] link to
> ויקיפדיה:מיזמי ויקיפדיה/מיזם ערכים
> ללא תמונות/קטגוריות/ספורטאים איטלקים
> (wikipedia:wikipedia projects articles without imagescategoriesSports
> people from Italy)
> מיוחד:אקראי (Special:Random) > 15 במאי (may 15)
> מיוחד:שינויים אחרונים (Special:RecentChanges) >
> 10_באוגוסט
>
> * size is important because we intend to add images
>
> 2009/7/6
> Send dev-l mailing list submissions to
> dev-l(a)openzim.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://intern.openzim.org/mailman/listinfo/dev-l [2]
> or, via email, send a message with subject or body help to
> dev-l-request(a)openzim.org
>
> You can reach the person managing the list at
> dev-l-owner(a)openzim.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of dev-l digest..."
>
> Todays Topics:
>
> 1. Kiwix index size (Asaf Bartov)
> 2. Re: Kiwix index size (Manuel Schneider)
> 3. Re: Kiwix index size (Emmanuel Engelhart)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 5 Jul 2009 19:18:57 +0300
> From: Asaf Bartov
> Subject: [openZIM dev-l] Kiwix index size
> To: dev-l(a)openzim.org
> Message-ID:
>
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi, everyone.
>
> When running Kiwixs indexer on the ZIM file I had created from the
> Hebrew
> Wikipedia last week, the Kiwix data directory ran up to a total of
> 31 items,
> totalling 2.3 GB. The ZIM file itself is ~300MB. Does this
> proportion make
> sense?
>
> Detailed ls output attached.
>
> Thanks in advance,
>
> Asaf Bartov
> --
> Asaf Bartov
>
I have answered to Rotem about the links. I have also open a bug on the Kiwix side:
https://sourceforge.net/tracker/?func=detail&aid=2817440&group_id=175508&at…
For the search engine index size, we have to search a solution with a smaller index.
Starting with the openzim solution should be good.
I will have a look during this week.
Emmanuel
Le lun 06/07/09 15:03, "Asaf Bartov" asaf.bartov(a)gmail.com a écrit:
> Clarification:
>
> This last message was by Rotem, a fellow WM-IL member helping me with
> the embedding of the Hebrew Wikipedia in the One Computer Per Child
> project.
>
> He is reporting issues with Kiwix and the ZIM file I created last
> week.
>
> Regarding size: Size is important, because we intend to add images
> (the 300MB ZIM file is the complete Hebrew Wikipedia text, but no
> pictures). We are hoping to have at least 5GB reserved for us in
> those One Computer Per Child machines we are to install on, but we may
> be forced to make do with 3GB. So every MB saved from the index, is
> another MB available for images...
>
> Asaf Bartov
> Wikimedia Israel
>
> On Mon, Jul 6, 2009 at 3:58 PM, Rotem Simha wrote:
> * there are some errors in links of files and special pages
> examples
> קובץ:Nuvola_apps_important.svg [1] link to
> ויקיפדיה:מיזמי ויקיפדיה/מיזם ערכים
> ללא תמונות/קטגוריות/ספורטאים איטלקים
> (wikipedia:wikipedia projects articles without imagescategoriesSports
> people from Italy)
> מיוחד:אקראי (Special:Random) > 15 במאי (may 15)
> מיוחד:שינויים אחרונים (Special:RecentChanges) >
> 10_באוגוסט
>
> * size is important because we intend to add images
>
> 2009/7/6
> Send dev-l mailing list submissions to
> dev-l(a)openzim.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://intern.openzim.org/mailman/listinfo/dev-l [2]
> or, via email, send a message with subject or body help to
> dev-l-request(a)openzim.org
>
> You can reach the person managing the list at
> dev-l-owner(a)openzim.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of dev-l digest..."
>
> Todays Topics:
>
> 1. Kiwix index size (Asaf Bartov)
> 2. Re: Kiwix index size (Manuel Schneider)
> 3. Re: Kiwix index size (Emmanuel Engelhart)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 5 Jul 2009 19:18:57 +0300
> From: Asaf Bartov
> Subject: [openZIM dev-l] Kiwix index size
> To: dev-l(a)openzim.org
> Message-ID:
>
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi, everyone.
>
> When running Kiwixs indexer on the ZIM file I had created from the
> Hebrew
> Wikipedia last week, the Kiwix data directory ran up to a total of
> 31 items,
> totalling 2.3 GB. The ZIM file itself is ~300MB. Does this
> proportion make
> sense?
>
> Detailed ls output attached.
>
> Thanks in advance,
>
> Asaf Bartov
> --
> Asaf Bartov
>
