[QA] tests that pass when run with phantomjs

[Željko Filipin]

On Thu, Nov 21, 2013 at 3:30 AM, S Page <spage at wikimedia.org> wrote:

>
> https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Running_testssays you can set BROWSER_LABEL but doesn't give the possible values.
>
> I guessed at BROWSER_LABEL=phantomjs , and it sort of worked, see below.
> Is that what you did?  Immediately my cursor froze and the Linux
> out-of-memory handler ran. :-/  I have PhantomJS 1.9.0 installed on Ubuntu
> 13.10


I do not have a lot of time to update the page right now, but I did my
best. Valid values for BROWSER_LABEL will probably change soon, so I will
not spend time now documenting it. For now, lowercase name of any major
browser will work.

Your machine should not blow up when you run tests in phantomjs.

This is how I run tests in phantomjs 1.9.2 and the latest commit of
browsertests repository.

$ export BROWSER_LABEL=phantomjs

$ export MEDIAWIKI_USER=Selenium_user
$ export MEDIAWIKI_PASSWORD=not-the-real-one

$ export MEDIAWIKI_URL=http://commons.wikimedia.beta.wmflabs.org/wiki/
$ bundle exec cucumber -t @commons.wikimedia.beta.wmflabs.org
...
Failing Scenarios:
cucumber features/upload_wizard.feature:28 # Scenario: Navigate to Release
rights page
cucumber features/upload_wizard.feature:36 # Scenario: Navigate to Describe
page
cucumber features/upload_wizard.feature:46 # Scenario: Navigate to Use page
5 scenarios (3 failed, 2 passed)
37 steps (3 failed, 17 skipped, 17 passed)
4m29.849s

$ export MEDIAWIKI_URL=http://en.wikipedia.beta.wmflabs.org/wiki/
$ bundle exec cucumber -t @en.wikipedia.beta.wmflabs.org
...
Failing Scenarios:
cucumber features/aftv5.feature:22 # Scenario: Click Yes return and click No
cucumber features/preferences_datetime.feature:15 # Scenario: Preferences
Date Time
31 scenarios (2 failed, 29 passed)
163 steps (2 failed, 5 skipped, 156 passed)
7m7.580s

$ export MEDIAWIKI_URL=http://test2.wikipedia.org/wiki/
$ bundle exec cucumber -t @test2.wikipedia.org
...
Failing Scenarios:
cucumber features/hotcat.feature:20 # Scenario: Hotcat category link
cucumber features/upload_wizard.feature:23 # Scenario: Navigate to Upload
page
cucumber features/upload_wizard.feature:28 # Scenario: Navigate to Release
rights page
cucumber features/upload_wizard.feature:36 # Scenario: Navigate to Describe
page
cucumber features/upload_wizard.feature:46 # Scenario: Navigate to Use page
39 scenarios (5 failed, 34 passed)
214 steps (5 failed, 21 skipped, 188 passed)
10m58.657s

It is interesting that commons.wikimedia.beta.wmflabs.org failures
were Watir::Wait::TimeoutError
and Watir::Exception::UnknownObjectException, but
commons.wikimedia.beta.wmflabs.org and test2.wikipedia.org were all
Net::ReadTimeout (except for one Watir::Exception::UnknownObjectException
at test2.wikipedia.org).

Željko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20131121/b56a1c71/attachment.html>