Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have full-text search within articles (unless I missed it). I assume that this was left out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of articles (hundreds or thousands). Making an index for tens of GB of text takes hours, so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one can also reduce compression). In general, I have no idea what it means that "zero team starts to think about pre-loaded content": sounds a lot like reinventing Kiwix, which would be a disastrous idea. :)
Nemo
Hi,
Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except for tiny tiny zim files, it would be too much CPU, battery and time consuming. But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would be a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App.
Hope that helps,
renaud
On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have full-text search within articles (unless I missed it). I assume that this was left out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of articles (hundreds or thousands). Making an index for tens of GB of text takes hours, so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one can also reduce compression). In general, I have no idea what it means that "zero team starts to think about pre-loaded content": sounds a lot like reinventing Kiwix, which would be a disastrous idea. :)
Nemo
Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l
how is this done in epub format?
rupert
On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin rgaudin@gmail.com wrote:
Hi,
Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except for tiny tiny zim files, it would be too much CPU, battery and time consuming. But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would be a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App.
Hope that helps,
renaud
On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have full-text search within articles (unless I missed it). I assume that this was left out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of articles (hundreds or thousands). Making an index for tens of GB of text takes hours, so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one can also reduce compression). In general, I have no idea what it means that "zero team starts to think about pre-loaded content": sounds a lot like reinventing Kiwix, which would be a disastrous idea. :)
Nemo
Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l
On 01.10.2014 10:44, renaud gaudin wrote:
Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except for tiny tiny zim files, it would be too much CPU, battery and time consuming. But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would be a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App.
To complete Renaud's comment, here is the dev strategy to get it done:
1 - Modify the libxapian code to make it able to pack the index in one file and open it with a filehandle & offset. Here is a small technical discussion about this: http://lists.xapian.org/pipermail/xapian-discuss/2013-November/009039.html
2 - Modify current solution to integrate the search index in a ZIM file.
3 - Introduce the libxapian (Kiwix fulltext search engine) to the binary lib (libkiwix). Here is the compilation script: https://sourceforge.net/p/kiwix/kiwix/ci/master/tree/android/build-android-w...
4 - Update the JNI interface ans the Android UI.
Like told by Charles, this is on in the 2015 Roamap: https://meta.wikimedia.org/wiki/Wikimedia_CH/2015_annual_plan#Offline_Dissem...
Emmanuel