[QA] [WikimediaMobile] Failing MobileFrontend browser tests

[Arthur Richards]

Head's up that Chris Mcmahon is on vacation all this week. That said, it
would be great to hear from anyone in QA about this - it has been a long
standing issue.


On Thu, Jul 10, 2014 at 10:51 AM, Tomasz Finc <tfinc at wikimedia.org> wrote:

> ChrisMC,
>
> Are these failures unique to mobile? They seem look to be at the
> infrastructure level so i'm guessing it would affect others.
>
> What other information do you need from us to be able to remedy these?
>
> --tomasz
>
> On Wed, Jul 9, 2014 at 5:54 PM, Jon Robson <jdlrobson at gmail.com> wrote:
> > Indeed. The tests have been failing for a month now, and had been
> > passing green before the move to integration.wikimedia.org
> > It would be really good to get these back to being useful.
> >
> > I'm not sure how our interaction with saucelabs changed during that
> > move, but is there anything that can be done on the short term to get
> > it back to how they were before when we were on cloudbees?
> >
> > Thanks Juliusz for the good summary of the problems!
> >
> >
> > On Wed, Jul 9, 2014 at 3:57 PM, Juliusz Gonera <jgonera at wikimedia.org>
> wrote:
> >> Today I worked a bit on fixing failing browser tests. The good news is
> that
> >> some tests detected a regression in core that caused full text search on
> >> mobile to not work. The bad news is that many of the failures seem to be
> >> caused by problems with Saucelabs and/or beta labs, examples:
> >>
> >>
> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Wikitext%20Editor%20%28TEST%20RUN%20ON%20WIKIPEDIA%20SO%20SHOULD%20NOT%20CAUSE%20SAVES%29/Opening_editor/
> >> Editor doesn't seem to load, possible causes: beta labs API error, or
> >> problem with connection between saucelabs and beta labs
> >>
> >>
> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Issues/Closing_page_issues__browser_back_/
> >> getaddrinfo: Name or service not known (SocketError) - seems like a
> problem
> >> with network on saucelabs
> >>
> >>
> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Page%20actions%20menu%20when%20logged%20in/I_can_add_the_page_to_my_watchlist/
> >> Saucelabs recording shows "no data received" error in Chrome, either
> beta
> >> labs problem or saucelabs network problem
> >>
> >>
> https://integration.wikimedia.org/ci/view/BrowserTests/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/lastBuild/testReport/junit/%28root%29/Generic%20special%20page%20features/Search_from_Watchlist/
> >> same as above
> >>
> >> Those are just a few examples from recent failures, but they make
> tracking
> >> regressions really tedious and time consuming. I know we are planning to
> >> move away from Saucelabs and use our own servers to run the tests. When
> will
> >> this happen? Is there any deadline?
> >>
> >> Thanks,
> >> --
> >> Juliusz
> >>
> >> _______________________________________________
> >> Mobile-l mailing list
> >> Mobile-l at lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/mobile-l
> >>
> >
> >
> >
> > --
> > Jon Robson
> > * http://jonrobson.me.uk
> > * https://www.facebook.com/jonrobson
> > * @rakugojon
> >
> > _______________________________________________
> > Mobile-l mailing list
> > Mobile-l at lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/mobile-l
>



-- 
Arthur Richards
Team Practices Lead
[[User:Awjrichards]]
IRC: awjr
+1-415-839-6885 x6687
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140710/627a01a5/attachment-0001.html>