[QA] browser tests race condition with login/logout

[Željko Filipin]

On Tue, Jul 1, 2014 at 9:14 PM, Antoine Musso <hashar+wmf at free.fr> wrote:

> Wasn't Cloudbees running the tests one after the other whereas on our
> Jenkins they are now run in parallel?  That would explain it all :)
>

# TLDR

No. :)

The same problem could happen at Cloudbees, but it is more likely at
Wikimedia Jenkins.

# Moar detail.

Each Jenkins job runs Cucumber scenarios sequentially, both at Cloudbees
and Wikimedia Jenkins.

Each Cucumber scenario starts one Sauce Labs job. We are constrained to
running 2-3 Jenkins jobs in parallel at Cloudbees. Cloudbees is using a
Sauce Labs account constrained to running 2-3 Sauce Labs jobs in parallel.

I do not know if we have any limitation in running Jenkins jobs in parallel
at Wikimedia Jenkins. We also have a new Sauce Labs account that allows us
to run 10-15 Sauce Labs jobs in parallel.

Since we can run more Jenkins jobs in parallel at Wikimedia Jenkins, and
they can start more parallel Sauce Labs jobs, if in one of those jobs a
Cucumber scenario explicitly logs out a user, browsers in other Sauce Labs
jobs would be logged out.

Željko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140702/b0ac3cc9/attachment.html>