I'd like to move this to a video conference call between analytics developers and analytics engineering to come to a mutual understanding of what the current pain points are and what's the biggest priority.
On Mon, Dec 15, 2014 at 4:37 PM, Nuria Ruiz <nuria@wikimedia.org> wrote:>QA in beta labs is good but not enough. We still need to do QA when a feature goes to production and currentlyThis is true but at the same time, I do not see anything in the description of your FF events that could not be tested on beta-labs. If we are talking add-block that can be tested even earlier, vagrant will be a fine venue. All the issues related to the client (browser) not emitting events can be tested on the development environment with ease.On Mon, Dec 15, 2014 at 4:18 PM, Leila Zia <leila@wikimedia.org> wrote:On Mon, Dec 15, 2014 at 10:06 AM, Toby Negrin <tnegrin@wikimedia.org> wrote:I share Christian's concerns -
Dario/Leila - can you comment based on your recent experiences with WikiGrok?I agree with Christian.
QA in beta labs is good but not enough. We still need to do QA when a feature goes to production and currently, it's very hard to figure out if there's a problem with logging. An example:
While testing WikiGrok in production, we learned that after some point tests from Firefox browser from my machine were not logged. We did not get any errors for this. I found out about this because I was trying to manually make a trace of activities and see if I can stitch them together and make sense of them. We eventually figured out what was going on in that case [1], but it concerns me that there may be other important events that we don't log in the DB and we never know that we're not logging.
Thanks
-Toby
> On Dec 15, 2014, at 9:42 AM, Christian Aistleitner <christian@quelltextlich.at> wrote:
>
> Hi,
>
>> On Mon, Dec 15, 2014 at 08:34:39AM -0800, Kevin Leduc wrote:
>> I closed the Phabricator task with a links to this thread and the wikitech
>> doc for testing on beta cluster.
>
> I am fine with keeping the task closed.
>
> But I am somewhat surprised to see beta mentioned in the
> resolution. Note that Dario's request set scope as [1]
>
> However, there are types of data quality issues that we only
> discover when collecting data at scale and in the wild (on
> browsers/platforms that we don’t necessarily test for internally).
>
> . That's a valid scope, but from my point of view, beta does not match
> that scope.
>
> Neither is beta large scale, nor is it hammered on with crazy devices.
>
> Beta is just a halfing the distance between EventLogging's devserver
> (Vagrant!) and production.
>
> Have fun,
> Christian
>
>
>
> [1] https://lists.wikimedia.org/pipermail/analytics/2014-December/002884.html
>
>
>
> --
> ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
> Companies' registry: 360296y in Linz
> Christian Aistleitner
> Kefermarkterstrasze 6a/3 Email: christian@quelltextlich.at
> 4293 Gutau, Austria Phone: +43 7946 / 20 5 81
> Fax: +43 7946 / 20 5 81
> Homepage: http://quelltextlich.at/
> ---------------------------------------------------------------
> _______________________________________________
> Analytics mailing list
> Analytics@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics
_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics