[QA] Subject for another workshop?

[Chris McMahon]

I enjoyed the session we did in San Francisco on Cucumber and Page Objects,
and wondered about other subject that might be of interest.

One suggestion I have is a workshop on analyzing why tests fail.  At any
given time we almost always have failing builds at
https://wmf.ci.cloudbees.com .  (Right now we have a lot of failing builds,
some of them are because of real bugs, some are not.)

I think it would be interesting to have a session where we look at the
error messages in Jenkins that our automation tools produce; relate that
information to the code in that test; relate the code to what the browser
did in the test; and then tell a story about what made the test fail.

What do you think?

So for instance when you look at

https://wmf.ci.cloudbees.com/job/browsertests-en.wikipedia.beta.wmflabs.org-linux-firefox/142/testReport/%28root%29/ULS%20cog%20behaviour%20on%20the%20side-bar%20for%20logged%20in%20users/Page_without_interlanguage_links_has_cog_icon_and_language_selector_on_Talk_page/

and you see something like

unable to locate element, using {:class=>"uls-settings-trigger",
:tag_name=>"span"} (Watir::Exception::UnknownObjectException)

what does that message mean?  What happened to cause that failure?

In this case the failure is because of this:
https://bugzilla.wikimedia.org/show_bug.cgi?id=50624

Having testers looking at these build failures and discussing them here
would be really useful to WMF, but it is also a subject of general interest
I think.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20130703/bf07d9c3/attachment.html>