[QA] Browser test guidelines and good practices

[Chris McMahon]

RSpec allows users to create custom "matchers", basically a DSL for
assertions.  (Although it's been about three years since I last made one,
and I've never done it in RSpec 3.0, only 2.0).  We should probably do that.

I filed a bug: https://bugzilla.wikimedia.org/show_bug.cgi?id=72149

On Thu, Oct 16, 2014 at 12:59 PM, S Page <spage at wikimedia.org> wrote:

> On Fri, Oct 3, 2014 at 1:45 PM, Chris McMahon <cmcmahon at wikimedia.org>
> wrote:
>
>>
>>  I made a wiki page:
>> https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Guidelines_and_good_practices
>>
>
> Great, though I think it should be merged with Writing tests.
>
>
>> Comments, criticism, and updates are welcome.
>>
>
> The page says
> - - - -
> Sometimes when AJAX is involved this means that you have to do something
> that seems like duplication but it is not:
>
> Then(/^the top post is an open discussion$/) do
>   on(FlowPage) do |page|
>     page.flow_first_topic_moderation_msg_element.when_not_present
>     expect(page.flow_first_topic_moderation_element).not_to be_visible
>   end
> end
>
> The following is not correct, because when_not_present is not an
> assertion:
>
> Then(/^the top post is an open discussion$/) do
>   on(FlowPage).flow_first_topic_moderation_msg_element.when_not_present #THIS IS NOT AN ASSERTION
> end
>
> - - - -
>
> But the first (recommended) test never fails on the expect(), it times out
> waiting for the element not to be present. And two lines is more verbose
> and prone to error (I deliberately mismatched the element names above, did
> you catch it?).
>
> Is there a way to say "expect the item to *eventually* (default 5
> seconds)  not be visible?" E.g.:
>
>  expect(page.some_dynamic_element).eventually_not_to be_visible
>
> expected visible? to eventually return false, still true after 5 seconds (RSpec::Expectations::ExpectationNotMetError)
>
> That would make the intent of tests much clearer, without repetition.
>
> The other problem with expect is it doesn't tell you what element failed.
> when_not_present's error message below is easier to debug:
>
>     timed out after 5 seconds, waiting for {:title=>"Edit header",
> :tag_name=>"a"} to disappear (Watir::Wait::TimeoutError)
>
> I think the fix for that is to expand expect() to be aware of what it's
> testing so it can print specific failures.  Not just .to and .not_to , but
> .to_eventually_be_visible , .to_match_text, etc.  At that point I would
> prefer assertEventuallyVisible(), assertMatches(), etc. because you can
> tell what kind of test it is at the start of the line.
>
>
> Anyway, progress, thanks!
> --
> =S Page  Features engineer
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/qa/attachments/20141016/a1394e29/attachment.html>