I'd like to move this to a video conference call between analytics
developers and analytics engineering to come to a mutual understanding of
what the current pain points are and what's the biggest priority.
It probably makes sense to have someone from R&D with experience in QA in
that meeting (Dario if you want a more experienced person, myself
otherwise). Not sure if you meant the same when you said analytics
engineering.
Leila
On Mon, Dec 15, 2014 at 4:37 PM, Nuria Ruiz <nuria(a)wikimedia.org
<javascript:_e(%7B%7D,'cvml','nuria@wikimedia.org');>> wrote:
>
> >QA in beta labs is good but not enough. We still need to do QA when a
> feature goes to production and currently
> This is true but at the same time, I do not see anything in the
> description of your FF events that could not be tested on beta-labs. If we
> are talking add-block that can be tested even earlier, vagrant will be a
> fine venue. All the issues related to the client (browser) not emitting
> events can be tested on the development environment with ease.
>
>
>
> On Mon, Dec 15, 2014 at 4:18 PM, Leila Zia <leila(a)wikimedia.org
> <javascript:_e(%7B%7D,'cvml','leila@wikimedia.org');>>
wrote:
>>
>>
>> On Mon, Dec 15, 2014 at 10:06 AM, Toby Negrin <tnegrin(a)wikimedia.org
>> <javascript:_e(%7B%7D,'cvml','tnegrin@wikimedia.org');>>
wrote:
>>>
>>> I share Christian's concerns -
>>>
>>> Dario/Leila - can you comment based on your recent experiences with
>>> WikiGrok?
>>>
>>
>> I agree with Christian.
>>
>> QA in beta labs is good but not enough. We still need to do QA when a
>> feature goes to production and currently, it's very hard to figure out if
>> there's a problem with logging. An example:
>>
>> While testing WikiGrok in production, we learned that after some point
>> tests from Firefox browser from my machine were not logged. We did not get
>> any errors for this. I found out about this because I was trying to
>> manually make a trace of activities and see if I can stitch them together
>> and make sense of them. We eventually figured out what was going on in that
>> case [1], but it concerns me that there may be other important events that
>> we don't log in the DB and we never know that we're not logging.
>>
>> Leila
>> [1]
>>
https://lists.wikimedia.org/pipermail/analytics/2014-December/002864.html
>>
>>
>>>
>>> Thanks
>>>
>>> -Toby
>>>
>>>
>>> > On Dec 15, 2014, at 9:42 AM, Christian Aistleitner <
>>> christian(a)quelltextlich.at
>>>
<javascript:_e(%7B%7D,'cvml','christian@quelltextlich.at');>>
wrote:
>>> >
>>> > Hi,
>>> >
>>> >> On Mon, Dec 15, 2014 at 08:34:39AM -0800, Kevin Leduc wrote:
>>> >> I closed the Phabricator task with a links to this thread and the
>>> wikitech
>>> >> doc for testing on beta cluster.
>>> >
>>> > I am fine with keeping the task closed.
>>> >
>>> > But I am somewhat surprised to see beta mentioned in the
>>> > resolution. Note that Dario's request set scope as [1]
>>> >
>>> > However, there are types of data quality issues that we only
>>> > discover when collecting data at scale and in the wild (on
>>> > browsers/platforms that we don’t necessarily test for internally).
>>> >
>>> > . That's a valid scope, but from my point of view, beta does not
match
>>> > that scope.
>>> >
>>> > Neither is beta large scale, nor is it hammered on with crazy devices.
>>> >
>>> > Beta is just a halfing the distance between EventLogging's
devserver
>>> > (Vagrant!) and production.
>>> >
>>> > Have fun,
>>> > Christian
>>> >
>>> >
>>> >
>>> > [1]
>>>
https://lists.wikimedia.org/pipermail/analytics/2014-December/002884.html
>>> >
>>> >
>>> >
>>> > --
>>> > ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
>>> > Companies' registry: 360296y in Linz
>>> > Christian Aistleitner
>>> > Kefermarkterstrasze 6a/3 Email: christian(a)quelltextlich.at
>>>
<javascript:_e(%7B%7D,'cvml','christian@quelltextlich.at');>
>>> > 4293 Gutau, Austria Phone: +43 7946 / 20 5 81
>>> > Fax: +43 7946 / 20 5 81
>>> > Homepage:
http://quelltextlich.at/
>>> > ---------------------------------------------------------------
>>> > _______________________________________________
>>> > Analytics mailing list
>>> > Analytics(a)lists.wikimedia.org
>>>
<javascript:_e(%7B%7D,'cvml','Analytics@lists.wikimedia.org');>
>>> >
https://lists.wikimedia.org/mailman/listinfo/analytics
>>>
>>> _______________________________________________
>>> Analytics mailing list
>>> Analytics(a)lists.wikimedia.org
>>>
<javascript:_e(%7B%7D,'cvml','Analytics@lists.wikimedia.org');>
>>>
https://lists.wikimedia.org/mailman/listinfo/analytics
>>>
>>
>> _______________________________________________
>> Analytics mailing list
>> Analytics(a)lists.wikimedia.org
>>
<javascript:_e(%7B%7D,'cvml','Analytics@lists.wikimedia.org');>
>>
https://lists.wikimedia.org/mailman/listinfo/analytics
>>
>>
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
>
<javascript:_e(%7B%7D,'cvml','Analytics@lists.wikimedia.org');>
>
https://lists.wikimedia.org/mailman/listinfo/analytics
>
>