Re: [WikimediaMobile] [Offline-l] Thinking about offline search within our native apps

List overview All Threads
Download

newer

older

Quick test of app on iOS 8.1 beta 2

Lightning deploy? VE went missing

rupert THURNER

1 Oct 2014 1 Oct '14

11:56 a.m.

how is this done in epub format? rupert On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin <rgaudin(a)gmail.com> wrote:

...

Hi, Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except for tiny tiny zim files, it would be too much CPU, battery and time consuming. But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would be a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App. Hope that helps, renaud On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote:

Dmitry Brant, 01/10/2014 01:07:

Looking at the Kiwix app for Android, it doesn't seem to have full-text search within articles (unless I missed it). I assume that this was left out of the Android version for performance reasons.

Don't assume, ask Emmanuel (cc Offline-l). If I understand correctly, you're talking of small selections of articles (hundreds or thousands). Making an index for tens of GB of text takes hours, so Kiwix doesn't always make one (on desktop, you usually download the pre-made index). But for few articles, the problem is easier (and one can also reduce compression). In general, I have no idea what it means that "zero team starts to think about pre-loaded content": sounds a lot like reinventing Kiwix, which would be a disastrous idea. :) Nemo _______________________________________________ Offline-l mailing list Offline-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l

Show replies by date

Lila Tretikov

1 Oct 1 Oct

6:22 p.m.

New subject: [Offline-l] Thinking about offline search within our native apps

I wonder if we need to think of this from user POV: What would they want to find Where is it in the article In order to do this we can look at the search results on mobile (and especially where they are failing today without GS) If we find out that most of this information for example is present in the first paragraph... we may only need to index it. And it would also focus us on improving summaries as a priority... L On Wed, Oct 1, 2014 at 1:56 AM, rupert THURNER <rupert.thurner(a)gmail.com> wrote:

...

how is this done in epub format? rupert On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin <rgaudin(a)gmail.com> wrote:

Hi, Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except

for

tiny tiny zim files, it would be too much CPU, battery and time

consuming.

But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it would

a pain to setup. As soon as we integrate both the ZIM file and the index in a single file, we'll enable full text search on the Android App. Hope that helps, renaud On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <nemowiki(a)gmail.com wrote: > > Dmitry Brant, 01/10/2014 01:07: >> >> Looking at the Kiwix app for Android, it doesn't seem to have full-text >> search within articles (unless I missed it). I assume that this was

left

>> out of the Android version for performance reasons. > > > Don't assume, ask Emmanuel (cc Offline-l). > If I understand correctly, you're talking of small selections of

articles

> (hundreds or thousands). Making an index for tens of GB of text takes

hours,

> so Kiwix doesn't always make one (on desktop, you usually download the > pre-made index). But for few articles, the problem is easier (and one

can

> also reduce compression). > In general, I have no idea what it means that "zero team starts to think > about pre-loaded content": sounds a lot like reinventing Kiwix, which

would

be a disastrous idea. :) Nemo _______________________________________________ Offline-l mailing list Offline-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/offline-l

Sage Ross

6:36 p.m.

New subject: Thinking about offline search within our native apps

...

From a user POV, a good first step would be to make section redirects work

on mobile. On apps and mobile web, in contrast to desktop, titles that redirect to specific sections still simply load the top of the article. Those redirected titles are among the most common things a user might search for within an article... essentially a hand-curated little search index. -Sage On Wednesday, October 1, 2014, Lila Tretikov <lila(a)wikimedia.org> wrote:

...

how is this done in epub format? rupert On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin <rgaudin(a)gmail.com <javascript:_e(%7B%7D,'cvml','rgaudin@gmail.com');>> wrote:

Hi, Full text search engine was not left out for performance reasons but for practical ones. Yes, we don't want people to generate the index on their phone. Except

for

tiny tiny zim files, it would be too much CPU, battery and time

consuming.

But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it

would be

a pain to setup. As soon as we integrate both the ZIM file and the index in a single

file,

we'll enable full text search on the Android App. Hope that helps, renaud On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <

nemowiki(a)gmail.com <javascript:_e(%7B%7D,'cvml','nemowiki@gmail.com');>>

wrote: > > Dmitry Brant, 01/10/2014 01:07: >> >> Looking at the Kiwix app for Android, it doesn't seem to have

full-text

>> search within articles (unless I missed it). I assume that this was

left

>> out of the Android version for performance reasons. > > > Don't assume, ask Emmanuel (cc Offline-l). > If I understand correctly, you're talking of small selections of

articles

> (hundreds or thousands). Making an index for tens of GB of text takes

hours,

> so Kiwix doesn't always make one (on desktop, you usually download the > pre-made index). But for few articles, the problem is easier (and one

can

> also reduce compression). > In general, I have no idea what it means that "zero team starts to

think

> about pre-loaded content": sounds a lot like reinventing Kiwix, which

would

> be a disastrous idea. :) > > Nemo > > _______________________________________________ > Offline-l mailing list > Offline-l(a)lists.wikimedia.org

<javascript:_e(%7B%7D,'cvml','Offline-l@lists.wikimedia.org');>

https://lists.wikimedia.org/mailman/listinfo/offline-l

Pine W

6:58 p.m.

New subject: Thinking about offline search within our native apps

Sage, I believe that this has just been patched. Hopefully this fix will be rolled out to users soon. Pine On Oct 1, 2014 8:36 AM, "Sage Ross" <sage(a)ragesoss.com> wrote:

...

From a user POV, a good first step would be to make section redirects work on mobile. On apps and mobile web, in contrast to desktop, titles that redirect to specific sections still simply load the top of the article. Those redirected titles are among the most common things a user might search for within an article... essentially a hand-curated little search index. -Sage On Wednesday, October 1, 2014, Lila Tretikov <lila(a)wikimedia.org> wrote:

how is this done in epub format? rupert On Wed, Oct 1, 2014 at 10:44 AM, renaud gaudin <rgaudin(a)gmail.com> wrote:

Hi, Full text search engine was not left out for performance reasons but

for

practical ones. Yes, we don't want people to generate the index on their phone. Except

for

tiny tiny zim files, it would be too much CPU, battery and time

consuming.

But we do want to add full text search to Android. It could work today but the search index is a (large) folder so it

would be

a pain to setup. As soon as we integrate both the ZIM file and the index in a single

file,

we'll enable full text search on the Android App. Hope that helps, renaud On Wed, Oct 1, 2014 at 5:48 AM, Federico Leva (Nemo) <

nemowiki(a)gmail.com>

wrote: > > Dmitry Brant, 01/10/2014 01:07: >> >> Looking at the Kiwix app for Android, it doesn't seem to have

full-text

>> search within articles (unless I missed it). I assume that this was

left

>> out of the Android version for performance reasons. > > > Don't assume, ask Emmanuel (cc Offline-l). > If I understand correctly, you're talking of small selections of

articles

> (hundreds or thousands). Making an index for tens of GB of text takes

hours,

> so Kiwix doesn't always make one (on desktop, you usually download the > pre-made index). But for few articles, the problem is easier (and one

can

> also reduce compression). > In general, I have no idea what it means that "zero team starts to

think

> about pre-loaded content": sounds a lot like reinventing Kiwix, which

would

> be a disastrous idea. :) > > Nemo > > _______________________________________________ > Offline-l mailing list > Offline-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/offline-l

_______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l

Tomasz Finc

2 Oct 2 Oct

12:20 a.m.

New subject: [zero] Re: [Offline-l] Thinking about offline search within our native apps

On Wed, Oct 1, 2014 at 8:22 AM, Lila Tretikov <lila(a)wikimedia.org> wrote:

...

In order to do this we can look at the search results on mobile (and especially where they are failing today without GS)

One of the asks that I have for the mobile teams in this quarter is to start logging failed search results. I want us to get to the point where we never show a 'no search results' page in any language. This is especially important in apps as they are starting to think about sessions length as a metric of engagement and search is at the core of that.

...

If we find out that most of this information for example is present in the first paragraph... we may only need to index it. And it would also focus us on improving summaries as a priority...

Could be interesting. We'd have to user test to validate the level of expected content. --tomasz

Federico Leva (Nemo)

1:30 a.m.

New subject: [Offline-l] [zero] Re: Thinking about offline search within our native apps

Tomasz Finc, 01/10/2014 23:20:

...

In order to do this we can look at the search results on mobile (and >especially where they are failing today without GS)

So, a custom solution for the decade-old requests (especially by Wiktionary) for logs of search queries without results (cf. countless ML threads) and smarter suggestions (cf. e.g. <https://bugzilla.wikimedia.org/buglist.cgi?bug_id=56320%2C47979%2C10421>)? Please make sure the solution is generic enough. Interlanguage suggestions and improvements to Cirrus "did you mean", plus some interface to access failed searches, would probably solve most problems. Nemo

Monte Hurd

7 Oct 7 Oct

3:24 a.m.

New subject: [Offline-l] [zero] Re: Thinking about offline search within our native apps

I would echo Nemo's concern that this sounds a lot like reinventing Kiwix. Also, Apple doesn't bundle third-party apps, so this would be an Android only feature. On Wed, Oct 1, 2014 at 3:30 PM, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote:

...

Tomasz Finc, 01/10/2014 23:20:

In order to do this we can look at the search results on mobile (and

especially where they are failing today without GS)

So, a custom solution for the decade-old requests (especially by Wiktionary) for logs of search queries without results (cf. countless ML threads) and smarter suggestions (cf. e.g. <https://bugzilla.wikimedia. org/buglist.cgi?bug_id=56320%2C47979%2C10421>)? Please make sure the solution is generic enough. Interlanguage suggestions and improvements to Cirrus "did you mean", plus some interface to access failed searches, would probably solve most problems. Nemo _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l

Adam Baso

9:46 p.m.

New subject: [Offline-l] [zero] Re: Thinking about offline search within our native apps

+wikipediazero(a)wikimedia.org (Sorry, non-(a)wikimedia.org emailers who may have gotten a bounce emailing the other email address zero(a)wikimedia.org.) To get caught up on this thread in case you didn't see the emails from outside the @wikimedia.org domain, see https://lists.wikimedia.org/pipermail/mobile-l/2014-October/008106.html. -Adam On Mon, Oct 6, 2014 at 5:24 PM, Monte Hurd <mhurd(a)wikimedia.org> wrote:

...

Tomasz Finc, 01/10/2014 23:20:

In order to do this we can look at the search results on mobile (and

>especially where they are failing today without GS)

So, a custom solution for the decade-old requests (especially by Wiktionary) for logs of search queries without results (cf. countless ML threads) and smarter suggestions (cf. e.g. <https://bugzilla.wikimedia. org/buglist.cgi?bug_id=56320%2C47979%2C10421>)? Please make sure the solution is generic enough. Interlanguage suggestions and improvements to Cirrus "did you mean", plus some interface to access failed searches, would probably solve most problems. Nemo _______________________________________________ Mobile-l mailing list Mobile-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l

3483

days inactive

3489

days old

mobile-l@lists.wikimedia.org

Manage subscription

7 comments

8 participants

tags (0)

participants (8)

Adam Baso
Federico Leva (Nemo)
Lila Tretikov
Monte Hurd
Pine W
rupert THURNER
Sage Ross
Tomasz Finc