[QA] Browser test guidelines and good practices

[S Page]

On Fri, Oct 3, 2014 at 1:45 PM, Chris McMahon <cmcmahon at wikimedia.org>
wrote:

>
>  I made a wiki page:
> https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Guidelines_and_good_practices
>

Great, though I think it should be merged with Writing tests.


> Comments, criticism, and updates are welcome.
>

The page says
- - - -
Sometimes when AJAX is involved this means that you have to do something
that seems like duplication but it is not:

Then(/^the top post is an open discussion$/) do
  on(FlowPage) do |page|
    page.flow_first_topic_moderation_msg_element.when_not_present
    expect(page.flow_first_topic_moderation_element).not_to be_visible
  end
end

The following is not correct, because when_not_present is not an assertion:

Then(/^the top post is an open discussion$/) do
  on(FlowPage).flow_first_topic_moderation_msg_element.when_not_present
#THIS IS NOT AN ASSERTION
end

- - - -

But the first (recommended) test never fails on the expect(), it times out
waiting for the element not to be present. And two lines is more verbose
and prone to error (I deliberately mismatched the element names above, did
you catch it?).

Is there a way to say "expect the item to *eventually* (default 5 seconds)
not be visible?" E.g.:

 expect(page.some_dynamic_element).eventually_not_to be_visible

expected visible? to eventually return false, still true after 5
seconds (RSpec::Expectations::ExpectationNotMetError)

That would make the intent of tests much clearer, without repetition.

The other problem with expect is it doesn't tell you what element failed.
when_not_present's error message below is easier to debug:

    timed out after 5 seconds, waiting for {:title=>"Edit header",
:tag_name=>"a"} to disappear (Watir::Wait::TimeoutError)

I think the fix for that is to expand expect() to be aware of what it's
testing so it can print specific failures.  Not just .to and .not_to , but
.to_eventually_be_visible , .to_match_text, etc.  At that point I would
prefer assertEventuallyVisible(), assertMatches(), etc. because you can
tell what kind of test it is at the start of the line.


Anyway, progress, thanks!
--
=S Page  Features engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/qa/attachments/20141016/b109f568/attachment.html>