[QA] New browsertest feature: check for ResourceLoader errors

[Chris McMahon]

Available now for browser tests in all repositories is a way to check for
ResourceLoader errors on any page at any time.  Using this feature requires
only adding a single line at any step in any Cucumber Feature.   This check
can be used on new page loads, for example:

Given I am on my page of interest
  And page has no ResourceLoader errors
...

It should also be useful after clicking on links:

When I click on a link that leads to a new page
  And page has no ResourceLoader errors

or

When I click on a link that leads to a new page
Then I see the stuff I should see on the new page
  And page has no ResourceLoader errors
...

In order to use this feature, make sure your Gemfile.lock in your
tests/browser folder shows a version of mediawiki_selenium (0.2.12) or
higher, then insert "Given (or When or And) page has no ResourceLoader
errors" in the features where you want to check for RL errors.  That's all.

History:

The code that checks for RL errors[1] has actually been part of the
qa/browsertests repo for some time.  It was an experiment that in the past
was only ever pointed to a single page, hard-coded into the create_account
test.

It is a useful check, and it does from time to time find bugs in the
create_account test, most recently just a week ago[2] it pointed out a
problem with CentralNotice and ResourceLoader.

This is something that the Flow team has asked for, so I made a start of
using it in the Flow repo[3].

Future:

At various times we have discussed doing this check automatically at least
at the start of each test, and possibly upon each new page load, but there
are some reasons why that might not be a good idea.  Being able to
explicitly call for a RL error check in the course of any test in any repo
seems like a good compromise.  Those who wish to use the feature may do so
at will, but no one is required to use it.

I'm sure our ideas about how to use this feature will change over time, but
for now, any test in any repo can check for ResourceLoader errors at any
time by simply adding a line to a Feature.

[1]
https://git.wikimedia.org/blob/mediawiki%2Fselenium/80e58ba38409a87145118d5dcccc384ee1aca441/lib%2Fmediawiki_selenium%2Fstep_definitions%2Fresource_loader_steps.rb
[2] https://bugzilla.wikimedia.org/show_bug.cgi?id=62552
[3] https://gerrit.wikimedia.org/r/#/c/119070/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140317/8cf1c3b8/attachment.html>