[QA] [WikimediaMobile] Failing MobileFrontend browser tests

[Tomasz Finc]

ChrisMC,

Are these failures unique to mobile? They seem look to be at the
infrastructure level so i'm guessing it would affect others.

What other information do you need from us to be able to remedy these?

--tomasz

On Wed, Jul 9, 2014 at 5:54 PM, Jon Robson <jdlrobson at gmail.com> wrote:
> Indeed. The tests have been failing for a month now, and had been
> passing green before the move to integration.wikimedia.org
> It would be really good to get these back to being useful.
>
> I'm not sure how our interaction with saucelabs changed during that
> move, but is there anything that can be done on the short term to get
> it back to how they were before when we were on cloudbees?
>
> Thanks Juliusz for the good summary of the problems!
>
>
> On Wed, Jul 9, 2014 at 3:57 PM, Juliusz Gonera <jgonera at wikimedia.org> wrote:
>> Today I worked a bit on fixing failing browser tests. The good news is that
>> some tests detected a regression in core that caused full text search on
>> mobile to not work. The bad news is that many of the failures seem to be
>> caused by problems with Saucelabs and/or beta labs, examples:
>>
>> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Wikitext%20Editor%20%28TEST%20RUN%20ON%20WIKIPEDIA%20SO%20SHOULD%20NOT%20CAUSE%20SAVES%29/Opening_editor/
>> Editor doesn't seem to load, possible causes: beta labs API error, or
>> problem with connection between saucelabs and beta labs
>>
>> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Issues/Closing_page_issues__browser_back_/
>> getaddrinfo: Name or service not known (SocketError) - seems like a problem
>> with network on saucelabs
>>
>> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Page%20actions%20menu%20when%20logged%20in/I_can_add_the_page_to_my_watchlist/
>> Saucelabs recording shows "no data received" error in Chrome, either beta
>> labs problem or saucelabs network problem
>>
>> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Generic%20special%20page%20features/Search_from_Watchlist/
>> same as above
>>
>> Those are just a few examples from recent failures, but they make tracking
>> regressions really tedious and time consuming. I know we are planning to
>> move away from Saucelabs and use our own servers to run the tests. When will
>> this happen? Is there any deadline?
>>
>> Thanks,
>> --
>> Juliusz
>>
>> _______________________________________________
>> Mobile-l mailing list
>> Mobile-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>
>
>
> --
> Jon Robson
> * http://jonrobson.me.uk
> * https://www.facebook.com/jonrobson
> * @rakugojon
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l