[QA] [WikimediaMobile] Failing MobileFrontend browser tests

[Tomasz Finc]

RobLa, is this something that we should be pulling in Greg for?

--tomasz

On Thu, Jul 10, 2014 at 11:08 AM, Arthur Richards
<arichards at wikimedia.org> wrote:
> Head's up that Chris Mcmahon is on vacation all this week. That said, it
> would be great to hear from anyone in QA about this - it has been a long
> standing issue.
>
>
> On Thu, Jul 10, 2014 at 10:51 AM, Tomasz Finc <tfinc at wikimedia.org> wrote:
>>
>> ChrisMC,
>>
>> Are these failures unique to mobile? They seem look to be at the
>> infrastructure level so i'm guessing it would affect others.
>>
>> What other information do you need from us to be able to remedy these?
>>
>> --tomasz
>>
>> On Wed, Jul 9, 2014 at 5:54 PM, Jon Robson <jdlrobson at gmail.com> wrote:
>> > Indeed. The tests have been failing for a month now, and had been
>> > passing green before the move to integration.wikimedia.org
>> > It would be really good to get these back to being useful.
>> >
>> > I'm not sure how our interaction with saucelabs changed during that
>> > move, but is there anything that can be done on the short term to get
>> > it back to how they were before when we were on cloudbees?
>> >
>> > Thanks Juliusz for the good summary of the problems!
>> >
>> >
>> > On Wed, Jul 9, 2014 at 3:57 PM, Juliusz Gonera <jgonera at wikimedia.org>
>> > wrote:
>> >> Today I worked a bit on fixing failing browser tests. The good news is
>> >> that
>> >> some tests detected a regression in core that caused full text search
>> >> on
>> >> mobile to not work. The bad news is that many of the failures seem to
>> >> be
>> >> caused by problems with Saucelabs and/or beta labs, examples:
>> >>
>> >>
>> >> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Wikitext%20Editor%20%28TEST%20RUN%20ON%20WIKIPEDIA%20SO%20SHOULD%20NOT%20CAUSE%20SAVES%29/Opening_editor/
>> >> Editor doesn't seem to load, possible causes: beta labs API error, or
>> >> problem with connection between saucelabs and beta labs
>> >>
>> >>
>> >> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Issues/Closing_page_issues__browser_back_/
>> >> getaddrinfo: Name or service not known (SocketError) - seems like a
>> >> problem
>> >> with network on saucelabs
>> >>
>> >>
>> >> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Page%20actions%20menu%20when%20logged%20in/I_can_add_the_page_to_my_watchlist/
>> >> Saucelabs recording shows "no data received" error in Chrome, either
>> >> beta
>> >> labs problem or saucelabs network problem
>> >>
>> >>
>> >> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Generic%20special%20page%20features/Search_from_Watchlist/
>> >> same as above
>> >>
>> >> Those are just a few examples from recent failures, but they make
>> >> tracking
>> >> regressions really tedious and time consuming. I know we are planning
>> >> to
>> >> move away from Saucelabs and use our own servers to run the tests. When
>> >> will
>> >> this happen? Is there any deadline?
>> >>
>> >> Thanks,
>> >> --
>> >> Juliusz
>> >>
>> >> _______________________________________________
>> >> Mobile-l mailing list
>> >> Mobile-l at lists.wikimedia.org
>> >> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>> >>
>> >
>> >
>> >
>> > --
>> > Jon Robson
>> > * http://jonrobson.me.uk
>> > * https://www.facebook.com/jonrobson
>> > * @rakugojon
>> >
>> > _______________________________________________
>> > Mobile-l mailing list
>> > Mobile-l at lists.wikimedia.org
>> > https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>
>
>
> --
> Arthur Richards
> Team Practices Lead
> [[User:Awjrichards]]
> IRC: awjr
> +1-415-839-6885 x6687