OK for me.
Le lun 31/08/09 14:41, "Manuel Schneider" manuel.schneider(a)wikimedia.ch a écrit:
> I haven't heard any objections, so November 22nd is fixed?
>
> Emmanuel, Tomasz?
>
> /Manuel
>
> Am Mittwoch, 26. August 2009 22:04:22 schrieb Tommi Mäkitalo:
> > Hi all,
> >
> > I would prefer November 20th-22th. Can we fix
> that?
>
> > Tommi
> >
> > On Mittwoch, 26. August 2009 20:04:38 Manuel
> Schneider wrote:
> > Dear all,
> > >
> > > refering to http://www.doodle.com/xaf4tzpuwk2xf59h I fix now
> the date of
> > the Developers Meeting on November 13th -
> 15th.
> >
> > > Quoting Tomasz "if you have any horrible
> objections you'd better say it
> > now" :-)
> > >
> > > We would be happy if the openmoko / Qi-Hardware
> quys at the meeting.
> > Tomasz tries to get someone from WikiPock
> (Wikireader on Symbian) to the
> > meeting as well.
> > > Tomasz from the Wikimedia Foundation will be
> there definetely.
> >
> > > The meeting will start on Friday evening
> (around 18 - 19 hr) and end on
> > Sunday afternoon (around 16 hr).
> > >
> > > Please register yourself as soon as you have
> fixed the date in your
> > schedule on https://openzim.org/Developer_Meetings/2009-2
> >> >
> > > We will try to get a big appartment where we
> have internet, accommodation
> > and a kitchen, so we can have a decent meeting
> with full service at the
> > place.
> > >
> > > Greets from Buenos Aires,
> > >
> > >
> > > Manuel
> >
> >
> _______________________________________________
> dev-l mailing list
> > dev-l@openz
> im.org
> https://intern.openzim.org/mailman/listinfo/dev-l
> >
>
>
> --
> Regards
> Manuel Schneider
>
> Wikimedia CH - Verein zur Förderung Freien Wissens
> Wikimedia CH - Association for the advancement of free knowledge
> www.wikimedia.ch
_______________________________________________
> dev-l mailing list
> dev-l@openz
> im.orghttps://intern.openzim.org/mailman/listinfo/dev-l
>
>
Dear all,
refering to http://www.doodle.com/xaf4tzpuwk2xf59h I fix now the date of the
Developers Meeting on November 13th - 15th.
Quoting Tomasz "if you have any horrible objections you'd better say it
now" :-)
We would be happy if the openmoko / Qi-Hardware quys at the meeting. Tomasz
tries to get someone from WikiPock (Wikireader on Symbian) to the meeting as
well.
Tomasz from the Wikimedia Foundation will be there definetely.
The meeting will start on Friday evening (around 18 - 19 hr) and end on Sunday
afternoon (around 16 hr).
Please register yourself as soon as you have fixed the date in your schedule
on https://openzim.org/Developer_Meetings/2009-2
We will try to get a big appartment where we have internet, accommodation and
a kitchen, so we can have a decent meeting with full service at the place.
Greets from Buenos Aires,
Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
Dear list members,
as you know Wikimania will start in two days in Buenos Aires, Argentina.
This year I will be there again and participating in at leats three different
sessions:
* Thursday, 27th, talk "openZIM - serving the Wikipedia Offline Projects"
http://wikimania2009.wikimedia.org/wiki/Proceedings:96
* Firday, 28th, panel "Panel on Wikimedia chapters"
http://wikimania2009.wikimedia.org/wiki/Proceedings:140
* a chapters meeting is being planned:
http://wikimania2009.wikimedia.org/wiki/Chapters_meeting
As far as I know unfortunately noone else from Switzerland is going there. But
I will provide reports to the board like I did at the Chapters Meeting in
April this year as well.
If you have any suggestions or oppinions I should place at the panels don't
hesitate to contact me.
I will leave today (Monday 24th) and return on Sunday, 31st.
Greets,
Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
Hi Marc
Le mar 11/08/09 16:26, "Marc Bantle" openmoko(a)rcie.de a écrit:
> I observed that Kiwix is producing an "ad-hoc" type
> index. This may be usefull for desktops as they have
> the power to generate an index file on the fly. On
> small footprint devices this will not reasonably be
> possible, due to lacking memory and cpu resources.
Yes
> Even on a dual core desktop with 3.5 GB of memory
> Kiwix failed to produce "ad-hoc" index of the
> openzim-edition of the German Wikipedia running
> out of memory after many hours.
Yes, this is Kiwix's specific issue with really big (with a lot of text) ZIM files.
> Question 1:
> >From the change log I see that kiwix is using
> a prominent search engine (Xapian) instead of the
> mechanism ZimReader/Writer are using. Is there an
> easy way to reuse an index produced by Kiwix on
> a different machines?
Yes although this is not trivial, the Xapian database is in your ~/.www.kiwix.org directory ([md5sum].index directory)
You can copy it to every other profile/account/computer and like that Kiwix will be able to search trough a ZIM without running the indexing process. To reduce the size of the directory, you can also use "xapian-compact".
We know, they are a list of improvements to do to improve the current index management usability.
> Question 2:
> Are there plans to enable Kiwix to read reusable
> indexes of the format released for ZimReader/
> Writer?
I have nothing against to make Kiwix compatible with different search engine backends... but this is not a priority yet for me.
I think I will do it in a middle far future, as soon as I have time for that or if for any reason a user really need that.
> Question 3:
> Are there plans to enable Kiwix to produce such a
> reusable index.
Not sure to understand the question? Do you speak from the ZIM indexes ?
In this case, cf. Question2 comment.
> Question 4:
> Wouldn't it be desirable to deliver reusable indexes
> together with zim-article-databases for all those
> people with less capable devices (mids, netbooks,
> phones) on the Kiwix site?
I do not believe having only one type of search engine is good at all: usages are multiple and for this reason with have different search engines. I think the ZIM format should not forced the user to make a choice. I also think, we have to be able to spread contents without data twice (with indexes). And finaly, I do not think that having compatible indexes should be a priority because 99% of the users don't care about that (they simply use only one client).
> Question 5:
> The zim databases supplied on the Kiwix site [1]
> seem to use the articles title field as article id field,
> which - I'm sure - solves some problems for Kiwix,
> but results in a list of article ids as result of a search
> on zimreader instead of a list of article titles. Since
> both Kiwix and ZimReader are part of the openzim
> standardization effort, this confuses me a bit. Which
> format is supposed to be the standard?
Tommi already answered to that...
IMO this is the job of the indexer to find the title... if there is a HTML page with a title, it has to use it.
More globaly, IMO forcing ZIM creators with url=title is a bad idea, we never should forced (with the format) to adopt special way of representig/storing Informations. All what if possible with "normal" HTTP/HTML should be also possible with contents in a ZIM file.
Emmanuel
Hey there,
Mirko from Qi-Hardware again.
Sorry for the long period of silence, but I had to get certain things in
motion before getting back to you.
The first units (sample units) are making their way to Europe now, to Berlin
to be precise and we are still very much into OpenZim and applications using
it on the Ben NanoNote[1].
I would propose to get one of the units to a member of your developer team
so that you can have a closer look and see what could be done.
As our software effort is mainly focused on kernel development we have not
reached the level of Xorg yet, so do you know of an ncurses viewer for the
OpenZim format, which we could deploy as a reference point for other
developers to get started? Or any other low-resource viewer usng fb only?
The biggest limitation will be the 32MB RAM, which excludes any webbrowsers
if I understand our engineers correctly.
Looking forward to your reply,
/mirko
[1] http://www.qi-hardware.com/products/ben-nanonote/
Hi all,
as I wrote in an earlier post, I have compiled
ZimReader for Openmoko platform. To make more
databases available for the device I also had look at
kiwix and the ZIM-Files supplied on it's site [1].
Some question arose from that and it appears, that this
list is the right place to discuss them. Please correct me,
if I'm wrong.
I observed that Kiwix is producing an "ad-hoc" type
index. This may be usefull for desktops as they have
the power to generate an index file on the fly. On
small footprint devices this will not reasonably be
possible, due to lacking memory and cpu resources.
Even on a dual core desktop with 3.5 GB of memory
Kiwix failed to produce "ad-hoc" index of the
openzim-edition of the German Wikipedia running
out of memory after many hours.
Question 1:
>From the change log I see that kiwix is using a
prominent search engine (Xapian) instead of the
mechanism ZimReader/Writer are using. Is there an
easy way to reuse an index produced by Kiwix on
a different machines?
Question 2:
Are there plans to enable Kiwix to read reusable
indexes of the format released for ZimReader/
Writer?
Question 3:
Are there plans to enable Kiwix to produce such a
reusable index.
Question 4:
Wouldn't it be desirable to deliver reusable indexes
together with zim-article-databases for all those
people with less capable devices (mids, netbooks,
phones) on the Kiwix site?
Question 5:
The zim databases supplied on the Kiwix site [1]
seem to use the articles title field as article id field,
which - I'm sure - solves some problems for Kiwix,
but results in a list of article ids as result of a search
on zimreader instead of a list of article titles. Since
both Kiwix and ZimReader are part of the openzim
standardization effort, this confuses me a bit. Which
format is supposed to be the standard?
Question 6:
I succeeded in producing a ZIM-Format index of
the openzim-edition of the German Wikipedia using
ZimWriter on the above 3.5 GB machine. Other than
the index supplied on DVD, the generated index is
1. 5 GB of size (instead of 1.1 GB ). Any ideas why
that is?
Cheers,
Marc
[1] http://tmp.kiwix.org/zim
Hi all,
i'd like bring your attention to another index related
issue, that came up on the openmoko mailing list recently.
Evopedia [1], a Wikipedia viewer by Christian Reitwießner
also based on Josch's Mokopedia [2] project .
The viewer uses the current GPS position or that of an
article currently viewed to offer other geographically near
articles.
It looks like the idea has been around for longer,
when I ask google. The reason I come up with this is
that it raises some new requirements for indexing that
might be considered as the indexing considerations go
on.
Cheers,
Marc
[1] http://www.reitwiessner.de/openmoko/evopedia.html
[2] http://wiki.openmoko.org/wiki/Mokopedia
Le mar 18/08/09 14:25, "Marc Bantle" openmoko(a)rcie.de a écrit:
> > We have currently a few tickets often, and to my
> opinion, would be great to have an update:
>
> > New header fields
> >
>
> I'm not sure if this is related, but if yes, could you
> consider clearing up the Title issue in this context.
> Unfortunately I didn't have time to fully understand
> the issue and answer your mails. Chances are good
> I will catch up this week.
"Title issue", do you haev a link to the ticket?
You can have a look the this feature request with details here:
http://bugs.openzim.org/show_bug.cgi?id=4
> > 5 zimreader on arm: miss-aligned dirent
> header
>
> > Patch were proposed from Marc, Tommi, can you give
> him a feedback
>
>
> I'm afraid I'm the one that owes feedback. I meanwhile
> compiled recent version of cxxtools and zimlib and the two
> Bugs seem to be fixed. Do you want me to mark them "resolved"
> or how is your typical procedure ?
>
> > 6 zimreader on arm: cxxtools/m4/asmtype.m4 broken
> for arm architecture
>
> > Marc wait on a feedback
> >
>
> See above!
You are the bug reporter, if you say this is OK now, bugs seem to be fixed, this is OK for me.
I have closed and taged as fixed the following tickets:
* http://bugs.openzim.org/show_bug.cgi?id=5
* http://bugs.openzim.org/show_bug.cgi?id=6
Thank you for your feedback
Emmanuel
Hi,
We have currently a few tickets often, and to my opinion, would be great to have an update:
New header fields
This is a feature request which is blocking for me:
* to prepare an online library
* to works on stemming, stopwords, etc..
Tommi, do you think you will have time to integrate them?
2 Add Windows/ReactOS support
For me this is done... we have no makefile, but is it necessary?
5 zimreader on arm: miss-aligned dirent header
Patch were proposed from Marc, Tommi, can you give him a feedback
6 zimreader on arm: cxxtools/m4/asmtype.m4 broken for arm architecture
Marc wait on a feedback
Emmanuel
Hello, all.
We're swinging back into action here at Wikimedia Israel, after a break, and
would like to bring up the issues that still keep us from being able to
choose Kiwix as the reader to bundle with the Hebrew Wikipedia ZIM in the
Israeli One Computer Per Child project.
I'll be posting about these issues in the coming couple of weeks.
The first and most pressing one is a stable Windows version. Alas, the
organization providing the machines insists on them running Windows and
Office -- in part, this is because Microsoft Israel is one of the sponsors
of the project. So this is dictated to us externally.
But we know this is being worked on (and we'll be happy to help, at least in
beta testing, but we may be able to find time to fix bugs as well, if you'd
like to give me a quick run through the issues), and a more immediate issue
is still the *index size* when processing the Hebrew ZIM file I created some
time ago.
Here is the size of the index created by Kiwix 0.8rc1:
./kiwix/372x8bpj.default/31c26198d06ad265677b450796cc09aa.index:
total *1.1G *
-rw-r--r-- 1 rotem rotem 0 2009-07-29 14:45 flintlock
-rw-r--r-- 1 rotem rotem 12 2009-07-29 14:45 iamflint
-rw-r--r-- 1 rotem rotem 12K 2009-07-29 17:04 postlist.baseA
-rw-r--r-- 1 rotem rotem 12K 2009-07-29 17:02 postlist.baseB
-rw-r--r-- 1 rotem rotem *754M *2009-07-29 17:04 postlist.DB
-rw-r--r-- 1 rotem rotem 70 2009-07-29 17:04 record.baseA
-rw-r--r-- 1 rotem rotem 70 2009-07-29 17:02 record.baseB
-rw-r--r-- 1 rotem rotem 3.3M 2009-07-29 17:04 record.DB
-rw-r--r-- 1 rotem rotem 4.4K 2009-07-29 17:04 termlist.baseA
-rw-r--r-- 1 rotem rotem 4.3K 2009-07-29 17:02 termlist.baseB
-rw-r--r-- 1 rotem rotem *278M *2009-07-29 17:04 termlist.DB
-rw-r--r-- 1 rotem rotem 232 2009-07-29 17:04 value.baseA
-rw-r--r-- 1 rotem rotem 230 2009-07-29 17:02 value.baseB
-rw-r--r-- 1 rotem rotem 14M 2009-07-29 17:04 value.DB
The ZIM file itself is ~*300MB*, so this is unreasonable. I'm guessing
there's some sort of bug in the index creation because of the Hebrew?
Is there a bug tracker for Kiwix we can follow?
Thanks,
Asaf Bartov
Wikimedia Israel
--
Asaf Bartov <asaf(a)forum2.org>