[QA] Weekend testing americas on September 7th

[Chris McMahon]

On Thu, Aug 15, 2013 at 6:43 AM, Justin Rohrman <rohrmanj at gmail.com> wrote:

> Let's hear it :)
>

Here goes...

There is a certain tension in the testing community between advocates for
UI/browser test automation and advocates for human "sapient" testing.  I'd
like to go there.

Specifically, I'd like to do this with Weekend Testing:

* Pick a complex feature for which browser test automation exists.  Right
now I'm thinking CirrusSearch or VisualEditor.  Because we're using
Cucumber, the test Scenarios should be understandable by people who are not
programmers.  Because we're using the page_object Ruby gem, the guts of the
tests should be readable by people with programming experience in just
about any language.   Regardless of level of expertise, we should be able
to provide everyone a local test environment from which to run the
automated tests against WMF hosts either natively or with a VirtualBox VM
configured via vagrant (http://www.mediawiki.org/wiki/Vagrant). We have
done this before:

online: https://www.mediawiki.org/wiki/Meetings/2013-07-18
live: https://www.mediawiki.org/wiki/Meetings/2013-06-27

Given a robust set of automated browser tests and the ability to run them,
let us then:

* Identify deficiencies in the automated test coverage for the feature.
 This may or may not include analysis of the test code itself, but would
certainly at least include analysis of the ATDD-style stated feature
coverage in the Cucumber Scenarios.  Do we have any technical debt in our
test code?

* Identify test charters for which automated testing is not possible and
which are only testable by actual human beings.  Are there tests that
cannot be automated, and is such testing worthwhile?

Outcomes:

* Participants will be able to analyze ATDD-style automated browser tests
* Participants will be able to run Cucumber + page_object browser tests and
analyze the results of those tests
* Participants will be able to demonstrate automated browser test practices
with examples from the open WMF browser test code
* Participants will be able to begin to contribute to WMF testing efforts
if they wish, whether automated or not

This might be too ambitious.  I'm pretty sure this would be the most
technically challenging session in the history of WTA.  OTOH, we've already
built the infrastructure to do this kind of thing, let's spread the word
about what is possible.

-Chris



>
> Message: 2
>> Date: Wed, 14 Aug 2013 08:56:54 -0700
>> From: Chris McMahon <cmcmahon at wikimedia.org>
>> To: "QA (software quality assurance) for Wikimedia projects."
>>
>>         <qa at lists.wikimedia.org>
>> Subject: Re: [QA] Weekend testing americas on September 7th
>> Message-ID:
>>         <
>> CAJohBHQzWK5H9weGS-SERHe0faZRSS08WjiGw-UNb7ABe-xC0w at mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>>
>> I actually have an idea that I think would be of interest to WTA and of
>> benefit to WMF, but I'd like to encourage others to reply first..
>>
>> -C
>>
>>
>>
>>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20130815/e6688311/attachment.html>