On the apps, yep.
On Thu, May 28, 2015 at 9:43 AM, Oliver Keyes <okeyes(a)wikimedia.org> wrote:
It wasn't previously on iOS, so we've just
been tracking Android? :/
On 28 May 2015 at 12:34, Adam Baso <abaso(a)wikimedia.org> wrote:
+Corey Floyd, who recently added code for this
stuff on iOS (slated for
forthcoming 4.1.4 iOS Wikipedia app release).
On Wed, May 27, 2015 at 12:33 PM, Adam Baso <abaso(a)wikimedia.org> wrote:
>
> Thought I'd step in here. People who know the mechanics of the relevant
> logging are as follows:
>
> Android: Dmitry Brant
> Desktop: Bahodir (Baha) Mansurov
> Mobile Web: Sam Smith & Baha
>
> CC'ing them.
>
> As I understand, Dan Garry's been talking with Baha already on the
desktop
> piece.
>
> It looks like on Chrome/42 UAs the clickthrough rate for a given suggest
> search form interaction is about 40%. [1]
>
> A couple of patches are pending (JS for emitting event on new EL schema)
> that will make it possible to figure out how often form submission
within a
> form interaction (ENTER/RETURN, tapping the
magnifying class) occurs as
> well.
>
> Total form interactions (keys on userSessionToken...maybe a better name
> could be used) minus clickthroughs (click-result) minus form submission
> (submit-form in the pending patches on the new EL schema) would be a
rough
> proxy of abandonment, I think.
>
> I had heard sendBeacon capable UAs were likely to have greater success
> emitting the click-result (i.e., when user clicks on a suggestion from
the
> form on the search panel on desktop) event
via mw.track, so it may make
> sense to confine queries for such analysis on desktop to known
sendBeacon
> browsers [2] to increase the odds of high
fidelity data just in case
there
> are outlier browsers that manage to somehow
emit click-result events
through
> means other than sendBeacon (it seems there
may be some of these,
assuming
> non-forged UAs).
>
> -Adam
>
> [1]
>
> > SELECT count(*) FROM Search_11670541 WHERE timestamp >= '20150526'
AND
> > timestamp < '20150527' AND event_action = 'click-result' AND
wiki =
'enwiki'
> > and userAgent LIKE
'%Chrome/42%';
> +----------+
> | count(*) |
> +----------+
> | 112 |
> +----------+
> 1 row in set (2.96 sec)
>
> > SELECT count(DISTINCT event_userSessionToken) FROM Search_11670541
WHERE
> > timestamp >= '20150526' AND
timestamp < '20150527' AND event_action =
> > 'click-result' AND wiki = 'enwiki' and userAgent LIKE
'%Chrome/42%';
> +----------------------------------------+
> | count(DISTINCT event_userSessionToken) |
> +----------------------------------------+
> | 112 |
> +----------------------------------------+
> 1 row in set (38.06 sec)
>
>
> > SELECT count(DISTINCT event_userSessionToken) FROM Search_11670541
WHERE
> > timestamp > '20150526' AND
timestamp < '20150527' AND wiki = 'enwiki'
and
> > userAgent LIKE '%Chrome/42%';
> +----------------------------------------+
> | count(DISTINCT event_userSessionToken) |
> +----------------------------------------+
> | 286 |
> +----------------------------------------+
> 1 row in set (7.26 sec)
>
>
> [2]
https://developer.mozilla.org/it/docs/Web/API/Navigator/sendBeacon
>
> On Tue, May 26, 2015 at 4:37 PM, Oliver Keyes <okeyes(a)wikimedia.org>
> wrote:
>>
>> Thanks Tomasz; great feedback! In order:
>>
>> * yeah, top percentiles were a heavily-requested thing so I built it
>> in from the get-go. Similarly, mean/median so we have some ability to
>> avoid distorting the results when the distribution changes.
>> * The 3 days data thing is a known -
>>
https://phabricator.wikimedia.org/T100056 - and is next on my to-do
>> list for bugfixes :).
>> * Glad you like the interface! It's actually functional on mobile, too
>> :D.
>> * Sample rate is crucial, yep. I'm reaching out to the authors of the
>> relevant EL schemas to find out how each was handled.
>> * Sessions < results opened makes sense in the event that users didn't
>> find what they wanted and went back to try again, but I'm not sure how
>> "session" is calculated; this is again something we lack transparency
>> around :(. Dan? You're the apps wizard.
>>
>> In supporting this: probably nothing at the moment although Nik/Kevin
>> chipping in on the relevant phabricator ticket
>> (
https://phabricator.wikimedia.org/T99762 ) to validate how much of a
>> PITA the idea of a unified schema and the associated implementations
>> are, would be good.
>>
>> I'm sort of shocked to hear "we're supposed to be presenting this
data
>> at the next metrics meeting": in the future if there are instances
>> where data is going to be up for public scrutiny, would it be possible
>> to explicitly associate time for that? My goal is to get us to the
>> point where our data is reliable all, or at least, most of the time,
>> and for a fragment of one person's time over two weeks, I think
>> progress on that is pretty fantastic. But prepping data for that kind
>> of event does change the priorities and what tasks should be worked
>> on.
>>
>> If we want to present data, generally speaking, let's discuss what we
>> can show off. If we want to present the dashboards I'll put my all
>> into making the data at least something where we know the
>> deficiencies, if not something where we consider the deficiencies
>> tolerable.
>>
>> On 26 May 2015 at 19:24, Tomasz Finc <tfinc(a)wikimedia.org> wrote:
>> > Thanks Oliver
>> >
>> > Early observations
>> >
>> > * Really happy to see top percentiles in load graphs
>> > * Mobile Web has only three days data
>> > * Interface is simple and easy to use
>> > * We need to know the sample rate
>> > * Apps have fewer sessions than results page opened
>> >
>> > Speaking over IRC it's clear that we don't have confidence in this
>> > data. We need to fix this and fix it quickly so that we can
accurately
>> > plan our work. We're supposed to
be presenting this data at the next
>> > metrics meeting and we're not a point where I feel comfortable
sharing
>> > our data let alone next steps.
>> >
>> > Oliver & Dan, what can the team do to support you guys on this? I
want
>> > you guys to own this and know that
were here to support you.
>> >
>> > Should I be adding new feature requests and bugs to
>> >
https://phabricator.wikimedia.org/tag/search-data-analytics/ ?
>> >
>> > --tomasz
>> >
>> > On Tue, May 26, 2015 at 11:04 AM, James Douglas
>> > <jdouglas(a)wikimedia.org> wrote:
>> >> This is a very exciting preview of things to come.
>> >>
>> >> Where are the data coming from? Am I just confused, or does "6
search
>
>> sessions per day" seem low?
> >>
> >> On Fri, May 22, 2015 at 2:35 PM, Oliver Keyes <okeyes(a)wikimedia.org
>> >> wrote:
>> >>>
>> >>>
http://searchdata.wmflabs.org/ - boop! This was my Friday.
Previously
>> >>> we were playing around with
them and testing what we needed with a
>> >>> static snapshot; these dashboards will now update once a day with
new
>> >>> information.
>> >>>
>> >>> It has turned up some bugs ("is the mobile schema just not
running?")
>> >>> and there are more metrics
to add. But for the time being, is
>> >>> progress
>> >>> :)
>> >>>
>> >>> --
>> >>> Oliver Keyes
>> >>> Research Analyst
>> >>> Wikimedia Foundation
>> >>>
>> >>> _______________________________________________
>> >>> Wikimedia-search-private mailing list
>> >>> Wikimedia-search-private(a)lists.wikimedia.org
>> >>>
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Wikimedia-search-private mailing list
>> >> Wikimedia-search-private(a)lists.wikimedia.org
>> >>
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private
>> >>
>> >
>> > _______________________________________________
>> > Wikimedia-search-private mailing list
>> > Wikimedia-search-private(a)lists.wikimedia.org
>> >
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private
--
Oliver Keyes
Research Analyst
Wikimedia Foundation
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
--
Oliver Keyes
Research Analyst
Wikimedia Foundation
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics