Is it by any chance possible to run cucumber tests under Safari on Cloudbees? If not, are there alternative solution to running the Cucumber tests on Safari automatically?
The browser infrastructure we use (SauceLabs) doesn't mention anything out of the ordinary for Safari 5/6/7. From what I can tell they should be available like any other[1].
Cloudbees/Jenkins just runs the shell script that runs the tests, afaik they don't provide the browsers or play a role in that.
— Krinkle
[1] https://saucelabs.com/platforms
On 15 May 2014, at 15:34, Gilles Dubuc gilles@wikimedia.org wrote:
Is it by any chance possible to run cucumber tests under Safari on Cloudbees? If not, are there alternative solution to running the Cucumber tests on Safari automatically? _______________________________________________ QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa
Le 15/05/2014 15:34, Gilles Dubuc a écrit :
Is it by any chance possible to run cucumber tests under Safari on Cloudbees? If not, are there alternative solution to running the Cucumber tests on Safari automatically?
Hello,
TL:DR; yes we can do it, but need a couple weeks as other pieces are assembled before.
Note Zeljkof was off for the last two weeks. I think he is coming back next monday.
Long story ----------
The QA browser tests project started with third parties providers since the Wikimedia CI infrastructure was not able to integrate the browser tests at that time.
Zeljkof and Chris went with two third parties:
- Cloudbees which provides Jenkins - Saucelabs which integrate Selenium and disposable virtual machines
Zeljkof and I started migrating the Cloudbees Jenkins jobs to use jenkins job builder (in integration/jenkins-job-builder-config that is the 'cloudbees' branch).
We now have to:
- fix the browser tests deployed on Wikimedia Jenkins: https://integration.wikimedia.org/ci/view/BrowserTests/
- integrate SauceLabs in the Jenkins Job Builder templates
From there, we will be to easily expand the current browser tests to use
more os/browsers stacks as long as SauceLabs support them.
Hope that clarify the current situation and short term goals.
On Thu, May 15, 2014 at 3:34 PM, Gilles Dubuc gilles@wikimedia.org wrote:
Is it by any chance possible to run cucumber tests under Safari on Cloudbees? If not, are there alternative solution to running the Cucumber tests on Safari automatically?
It is as easy as changing the value of BROWSER environment variable. Let me know if you need help with this.
Željko
I've just tried setting BROWSER to "safari" and PLATFORM to "osx", doesn't seem to work:
https://wmf.ci.cloudbees.com/job/browsertests-en.wikipedia.beta.wmflabs.org-...
On Thu, Jun 12, 2014 at 4:35 PM, Željko Filipin zfilipin@wikimedia.org wrote:
On Thu, May 15, 2014 at 3:34 PM, Gilles Dubuc gilles@wikimedia.org wrote:
Is it by any chance possible to run cucumber tests under Safari on Cloudbees? If not, are there alternative solution to running the Cucumber tests on Safari automatically?
It is as easy as changing the value of BROWSER environment variable. Let me know if you need help with this.
Željko
QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa
On Mon, Jun 16, 2014 at 1:01 PM, Gilles Dubuc gilles@wikimedia.org wrote:
I've just tried setting BROWSER to "safari" and PLATFORM to "osx", doesn't seem to work:
For future reference, this page lists all supported platforms:
https://saucelabs.com/platforms
After taking a closer look, these variables have to be changed:
export BROWSER=safari export PLATFORM='OS X 10.9' export VERSION=7
https://wmf.ci.cloudbees.com/job/browsertests-en.wikipedia.beta.wmflabs.org-...
The problem is that no tests are run. These lines in job configuration:
... # run tests bundle exec cucumber ... --tags @$BROWSER_TAG ... ...
ges executed like this:
... bundle exec cucumber ... --tags @safari ... 0 scenarios 0 steps 0m0.000s ...
Since there are no scenarios tagged @safari, no tests are run. You should either remove "--tags @$BROWSER_TAG" from job configuration, or tag at least one scenario with @safari tag.
Let me know if you need help with that.
Željko
I keep forgetting about the browser tag. I'll sort it out from here, thank you for the help.
On Mon, Jun 16, 2014 at 1:23 PM, Željko Filipin zfilipin@wikimedia.org wrote:
On Mon, Jun 16, 2014 at 1:01 PM, Gilles Dubuc gilles@wikimedia.org wrote:
I've just tried setting BROWSER to "safari" and PLATFORM to "osx", doesn't seem to work:
For future reference, this page lists all supported platforms:
https://saucelabs.com/platforms
After taking a closer look, these variables have to be changed:
export BROWSER=safari export PLATFORM='OS X 10.9' export VERSION=7
https://wmf.ci.cloudbees.com/job/browsertests-en.wikipedia.beta.wmflabs.org-...
The problem is that no tests are run. These lines in job configuration:
... # run tests bundle exec cucumber ... --tags @$BROWSER_TAG ... ...
ges executed like this:
... bundle exec cucumber ... --tags @safari ... 0 scenarios 0 steps 0m0.000s ...
Since there are no scenarios tagged @safari, no tests are run. You should either remove "--tags @$BROWSER_TAG" from job configuration, or tag at least one scenario with @safari tag.
Let me know if you need help with that.
Željko
QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa
multimedia@lists.wikimedia.org