[Wikitech-l] Running Selenium tests over the speed limit with PhantomJS

A few days ago I have noticed that Selenium[1] can now drive PhantomJS[2], "a headless WebKit with JavaScript API". Intrigued by the buzzword to word ratio I have decided to take it for a test drive. # Test Drive If you are on a Mac (and use Homebrew), installation is as simple as possible: $ brew update && brew install phantomjs (You will need Ruby for this part.) $ irb => true => #<Watir::Browser:0x..f96d67ac99e7fe7c6 url="about:blank" title=""> => "http://www.google.hr/" => #<File:screenshot.png (closed)> Saving screenshots from a headless browser. Wat?[3] A bit of unscientific research (just a few test runs) follows. We have two repositories with Selenium tests, browsertests[4][5] and MobileFrontend[6][7]. # browsertests I had to fix a few things and I could not get a few tests to run[8][9][10]. I did not want to spend a lot of time on it, there were still a lot of tests that were running just fine. $ export BROWSER_LABEL=firefox $ bundle exec cucumber --tags ~@phantomjs-bug Using the default profile... 18 scenarios (18 passed) 100 steps (100 passed) 5m35.491s $ export BROWSER_LABEL=chrome $ bundle exec cucumber --tags ~@phantomjs-bug Using the default profile... 18 scenarios (18 passed) 100 steps (100 passed) 3m37.241s $ export BROWSER_LABEL=phantomjs $ bundle exec cucumber --tags ~@phantomjs-bug 18 scenarios (18 passed) 100 steps (100 passed) 1m53.282s For some reason tests always fail on phantomjs when I run them in parallel, even if only two browsers are open at the same time. I did not notice that with mobilefrontend tests. I will investigate it next week. # MobileFrontend $ export BROWSER_LABEL=firefox $ time bundle exec rake serial real 0m56.888s $ time bundle exec rake parallel real 1m9.035s $ export BROWSER_LABEL=chrome $ time bundle exec rake serial real 1m18.420s $ time bundle exec rake parallel real 1m9.510s $ export BROWSER_LABEL=phantomjs $ time bundle exec rake serial real 0m47.956s $ time bundle exec rake parallel real 0m10.521s I had to make just one line change to make all tests run in PhantomJS[11]. Does this sound interesting? Join the weekly QA/testing event on February 11[12] and I will teach you all you need to know. Can not wait until February 11? Just reply here or send me an e-mail message. IRC is fine too, just ping zeljkof. Željko -- [1] http://seleniumhq.org/ [2] http://phantomjs.org/ [3] http://knowyourmeme.com/memes/wat [4] https://gerrit.wikimedia.org/r/#/admin/projects/qa/browsertests [5] https://github.com/wikimedia/qa-browsertests [6] https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/Mobile… [7] https://github.com/wikimedia/mediawiki-extensions-MobileFrontend [8] https://gerrit.wikimedia.org/r/#/c/45737/ [9] https://gerrit.wikimedia.org/r/#/c/45738/ [10] https://gerrit.wikimedia.org/r/#/c/45765/ [11] https://gerrit.wikimedia.org/r/#/c/45182/ [12] https://www.mediawiki.org/wiki/QA/Weekly_goals