how is this done in epub format?
rupert
On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin rgaudin@gmail.com wrote:
Hi,
Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except for tiny tiny zim files, it would be too much CPU, battery and time consuming. But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would be a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App.
Hope that helps,
renaud
On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have full-text search within articles (unless I missed it). I assume that this was left out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of articles (hundreds or thousands). Making an index for tens of GB of text takes hours, so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one can also reduce compression). In general, I have no idea what it means that "zero team starts to think about pre-loaded content": sounds a lot like reinventing Kiwix, which would be a disastrous idea. :)
Nemo
Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l
I wonder if we need to think of this from user POV:
What would they want to find Where is it in the article
In order to do this we can look at the search results on mobile (and especially where they are failing today without GS)
If we find out that most of this information for example is present in the first paragraph... we may only need to index it. And it would also focus us on improving summaries as a priority...
L
On Wed, Oct 1, 2014 at 1:56 AM, rupert THURNER rupert.thurner@gmail.com wrote:
how is this done in epub format?
rupert
On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin rgaudin@gmail.com wrote:
Hi,
Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except
for
tiny tiny zim files, it would be too much CPU, battery and time
consuming.
But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would
be
a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App.
Hope that helps,
renaud
On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <nemowiki@gmail.com
wrote:
Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have full-text search within articles (unless I missed it). I assume that this was
left
out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of
articles
(hundreds or thousands). Making an index for tens of GB of text takes
hours,
so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one
can
also reduce compression). In general, I have no idea what it means that "zero team starts to think about pre-loaded content": sounds a lot like reinventing Kiwix, which
would
be a disastrous idea. :)
Nemo
Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l
From a user POV, a good first step would be to make section redirects work
on mobile. On apps and mobile web, in contrast to desktop, titles that redirect to specific sections still simply load the top of the article. Those redirected titles are among the most common things a user might search for within an article... essentially a hand-curated little search index.
-Sage
On Wednesday, October 1, 2014, Lila Tretikov lila@wikimedia.org wrote:
I wonder if we need to think of this from user POV:
What would they want to find Where is it in the article
In order to do this we can look at the search results on mobile (and especially where they are failing today without GS)
If we find out that most of this information for example is present in the first paragraph... we may only need to index it. And it would also focus us on improving summaries as a priority...
L
On Wed, Oct 1, 2014 at 1:56 AM, rupert THURNER <rupert.thurner@gmail.com javascript:_e(%7B%7D,'cvml','rupert.thurner@gmail.com');> wrote:
how is this done in epub format?
rupert
On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin <rgaudin@gmail.com javascript:_e(%7B%7D,'cvml','rgaudin@gmail.com');> wrote:
Hi,
Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except
for
tiny tiny zim files, it would be too much CPU, battery and time
consuming.
But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it
would be
a pain to setup. As soon as we integrate both the ZIM file and the index in a single
file,
we'll enable full text search on the Android App.
Hope that helps,
renaud
On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <
nemowiki@gmail.com javascript:_e(%7B%7D,'cvml','nemowiki@gmail.com');>
wrote:
Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have
full-text
search within articles (unless I missed it). I assume that this was
left
out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of
articles
(hundreds or thousands). Making an index for tens of GB of text takes
hours,
so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one
can
also reduce compression). In general, I have no idea what it means that "zero team starts to
think
about pre-loaded content": sounds a lot like reinventing Kiwix, which
would
be a disastrous idea. :)
Nemo
Offline-l mailing list Offline-l@lists.wikimedia.org
javascript:_e(%7B%7D,'cvml','Offline-l@lists.wikimedia.org');
Sage, I believe that this has just been patched. Hopefully this fix will be rolled out to users soon.
Pine On Oct 1, 2014 8:36 AM, "Sage Ross" sage@ragesoss.com wrote:
From a user POV, a good first step would be to make section redirects work on mobile. On apps and mobile web, in contrast to desktop, titles that redirect to specific sections still simply load the top of the article. Those redirected titles are among the most common things a user might search for within an article... essentially a hand-curated little search index.
-Sage
On Wednesday, October 1, 2014, Lila Tretikov lila@wikimedia.org wrote:
I wonder if we need to think of this from user POV:
What would they want to find Where is it in the article
In order to do this we can look at the search results on mobile (and especially where they are failing today without GS)
If we find out that most of this information for example is present in the first paragraph... we may only need to index it. And it would also focus us on improving summaries as a priority...
L
On Wed, Oct 1, 2014 at 1:56 AM, rupert THURNER rupert.thurner@gmail.com wrote:
how is this done in epub format?
rupert
On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin rgaudin@gmail.com wrote:
Hi,
Full text search engine was not left out for performance reasons but
for
practical ones. Yes, we don't want people to generate the index on their phone. Except
for
tiny tiny zim files, it would be too much CPU, battery and time
consuming.
But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it
would be
a pain to setup. As soon as we integrate both the ZIM file and the index in a single
file,
we'll enable full text search on the Android App.
Hope that helps,
renaud
On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <
nemowiki@gmail.com>
wrote:
Dmitry Brant, 01/10/2014 01:07:
Looking at the Kiwix app for Android, it doesn't seem to have
full-text
search within articles (unless I missed it). I assume that this was
left
out of the Android version for performance reasons.
Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of
articles
(hundreds or thousands). Making an index for tens of GB of text takes
hours,
so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one
can
also reduce compression). In general, I have no idea what it means that "zero team starts to
think
about pre-loaded content": sounds a lot like reinventing Kiwix, which
would
be a disastrous idea. :)
Nemo
Offline-l mailing list Offline-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On Wed, Oct 1, 2014 at 8:22 AM, Lila Tretikov lila@wikimedia.org wrote:
In order to do this we can look at the search results on mobile (and especially where they are failing today without GS)
One of the asks that I have for the mobile teams in this quarter is to start logging failed search results. I want us to get to the point where we never show a 'no search results' page in any language. This is especially important in apps as they are starting to think about sessions length as a metric of engagement and search is at the core of that.
If we find out that most of this information for example is present in the first paragraph... we may only need to index it. And it would also focus us on improving summaries as a priority...
Could be interesting. We'd have to user test to validate the level of expected content.
--tomasz
Tomasz Finc, 01/10/2014 23:20:
In order to do this we can look at the search results on mobile (and
especially where they are failing today without GS)
One of the asks that I have for the mobile teams in this quarter is to start logging failed search results. I want us to get to the point where we never show a 'no search results' page in any language.
So, a custom solution for the decade-old requests (especially by Wiktionary) for logs of search queries without results (cf. countless ML threads) and smarter suggestions (cf. e.g. https://bugzilla.wikimedia.org/buglist.cgi?bug_id=56320%2C47979%2C10421)? Please make sure the solution is generic enough. Interlanguage suggestions and improvements to Cirrus "did you mean", plus some interface to access failed searches, would probably solve most problems.
Nemo
I would echo Nemo's concern that this sounds a lot like reinventing Kiwix.
Also, Apple doesn't bundle third-party apps, so this would be an Android only feature.
On Wed, Oct 1, 2014 at 3:30 PM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Tomasz Finc, 01/10/2014 23:20:
In order to do this we can look at the search results on mobile (and
especially where they are failing today without GS)
One of the asks that I have for the mobile teams in this quarter is to start logging failed search results. I want us to get to the point where we never show a 'no search results' page in any language.
So, a custom solution for the decade-old requests (especially by Wiktionary) for logs of search queries without results (cf. countless ML threads) and smarter suggestions (cf. e.g. https://bugzilla.wikimedia. org/buglist.cgi?bug_id=56320%2C47979%2C10421)? Please make sure the solution is generic enough. Interlanguage suggestions and improvements to Cirrus "did you mean", plus some interface to access failed searches, would probably solve most problems.
Nemo
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
+wikipediazero@wikimedia.org
(Sorry, non-@wikimedia.org emailers who may have gotten a bounce emailing the other email address zero@wikimedia.org.)
To get caught up on this thread in case you didn't see the emails from outside the @wikimedia.org domain, see https://lists.wikimedia.org/pipermail/mobile-l/2014-October/008106.html.
-Adam
On Mon, Oct 6, 2014 at 5:24 PM, Monte Hurd mhurd@wikimedia.org wrote:
I would echo Nemo's concern that this sounds a lot like reinventing Kiwix.
Also, Apple doesn't bundle third-party apps, so this would be an Android only feature.
On Wed, Oct 1, 2014 at 3:30 PM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Tomasz Finc, 01/10/2014 23:20:
In order to do this we can look at the search results on mobile (and
especially where they are failing today without GS)
One of the asks that I have for the mobile teams in this quarter is to start logging failed search results. I want us to get to the point where we never show a 'no search results' page in any language.
So, a custom solution for the decade-old requests (especially by Wiktionary) for logs of search queries without results (cf. countless ML threads) and smarter suggestions (cf. e.g. https://bugzilla.wikimedia. org/buglist.cgi?bug_id=56320%2C47979%2C10421)? Please make sure the solution is generic enough. Interlanguage suggestions and improvements to Cirrus "did you mean", plus some interface to access failed searches, would probably solve most problems.
Nemo
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l