[QA] Cut testing time for Cirrus drastically

[S Page]

Thanks, I added to
https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Writing_tests#Miscellaneous

The Flow browser tests don't yet work well in PhantomJS, probably because
of rollover states and flyout menus.

On Wed, Jan 29, 2014 at 6:57 PM, Nikolas Everett <neverett at wikimedia.org>wrote:

> Yesterday I spent the time working on CirrusSearch's browser tests to
> get them working well with parallel cucumber and phantomjs. That cut
> the time to run all the tests from 24 minutes to 6.  That is super
> sweet because Cirrus gets the vast majority of its test coverage from
> these tests.
>
> So here are the steps I took to get there.  You can probably do them
> in any order if you want to follow my footsteps.  Anyway:
> 1.  Make sure all tests pass in PhantomJS.  Mostly, this shouldn't
> require any work beyond installing PhantomJS, setting BROWSER_LABEL to
> phantomjs, and running tests.
> 2.  Make sure your tests are in many small features rather than a few
> huge ones.  This took me an hour or so because Cirrus had two huge
> features that I broke into 18 or so small ones.
> 3.  Add <gem "parallel_tests"> to your gemfile and run bundler install.
> 4.  You are ready!  Yay.
>
> Running tests is now a two step thing.  First run tests in parallel.
>   bundle exec parallel_cucumber --nice -n 5 features/
> Some of them will fail because, well, parallel_cucumber seems a bit
> flaky.  Oh well.  You can rerun them like this:
>   cat cucumber_failures.log | xargs bundle exec cucumber
>
> I'm not sure how you'd do that in Windows but the above should work in
> Linux and OSX.
>

-- 
=S Page  Features engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140130/c4a28b84/attachment.html>