<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Aug 20, 2013 at 1:01 PM, Quim Gil <span dir="ltr"><<a href="mailto:qgil@wikimedia.org" target="_blank">qgil@wikimedia.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 08/16/2013 07:41 AM, Justin Rohrman wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This sounds like a great idea to me. The timing would certainly be tight<br>
but I think with the VMs and Vagrant usage we should be good. I'll ping<br>
Michael and see what he thinks.<br>
</blockquote>
<br></div>
It is a great idea and it is technically ambitious. Participants must understand that they need to do some homework before the weekend. It is all well documented (and I will go through the process & docs again before the end of this month) but it takes time and good bandwidth to get your environment downloaded and installed.<br>
</blockquote><div><br></div><div>I'd like to do this but I think it will be October at the earliest. Between the Labor Day holiday and some other schedule conflicts, September is not going to happen. </div><div>-</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
<br>
<br>
Date: Thu, 15 Aug 2013 14:12:08 -0700<br>
<br>
From: Chris McMahon <<a href="mailto:cmcmahon@wikimedia.org" target="_blank">cmcmahon@wikimedia.org</a><br></div>
<mailto:<a href="mailto:cmcmahon@wikimedia.org" target="_blank">cmcmahon@wikimedia.org</a><u></u>>><div class="im"><br>
To: "QA (software quality assurance) for Wikimedia projects."<br></div>
<<a href="mailto:qa@lists.wikimedia.org" target="_blank">qa@lists.wikimedia.org</a> <mailto:<a href="mailto:qa@lists.wikimedia.org" target="_blank">qa@lists.wikimedia.org</a><u></u>>><div class="im">
<br>
Subject: Re: [QA] Weekend testing americas on September 7th<br>
Message-ID:<br>
<br>
<<a href="mailto:CAJohBHTurqx7G5EhK1Q2Ant%2BgTFZP4gPvqJ3XjzH2_x2fskRZg@mail.gmail.com" target="_blank">CAJohBHTurqx7G5EhK1Q2Ant+<u></u>gTFZP4gPvqJ3XjzH2_x2fskRZg@<u></u>mail.gmail.com</a><br></div>
<mailto:<a href="mailto:CAJohBHTurqx7G5EhK1Q2Ant%252BgTFZP4gPvqJ3XjzH2_x2fskRZg@mail.gmail.com" target="_blank">CAJohBHTurqx7G5EhK1Q2A<u></u>nt%2BgTFZP4gPvqJ3XjzH2_<u></u>x2fskRZg@mail.gmail.com</a>>><div class="im">
<br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
On Thu, Aug 15, 2013 at 6:43 AM, Justin Rohrman <<a href="mailto:rohrmanj@gmail.com" target="_blank">rohrmanj@gmail.com</a><br></div><div><div class="h5">
<mailto:<a href="mailto:rohrmanj@gmail.com" target="_blank">rohrmanj@gmail.com</a>>> wrote:<br>
<br>
> Let's hear it :)<br>
><br>
<br>
Here goes...<br>
<br>
There is a certain tension in the testing community between<br>
advocates for<br>
UI/browser test automation and advocates for human "sapient"<br>
testing. I'd<br>
like to go there.<br>
<br>
Specifically, I'd like to do this with Weekend Testing:<br>
<br>
* Pick a complex feature for which browser test automation exists.<br>
Right<br>
now I'm thinking CirrusSearch or VisualEditor. Because we're using<br>
Cucumber, the test Scenarios should be understandable by people who<br>
are not<br>
programmers. Because we're using the page_object Ruby gem, the guts<br>
of the<br>
tests should be readable by people with programming experience in just<br>
about any language. Regardless of level of expertise, we should be<br>
able<br>
to provide everyone a local test environment from which to run the<br>
automated tests against WMF hosts either natively or with a<br>
VirtualBox VM<br>
configured via vagrant (<a href="http://www.mediawiki.org/wiki/Vagrant" target="_blank">http://www.mediawiki.org/<u></u>wiki/Vagrant</a>). We have<br>
done this before:<br>
<br>
online: <a href="https://www.mediawiki.org/wiki/Meetings/2013-07-18" target="_blank">https://www.mediawiki.org/<u></u>wiki/Meetings/2013-07-18</a><br>
live: <a href="https://www.mediawiki.org/wiki/Meetings/2013-06-27" target="_blank">https://www.mediawiki.org/<u></u>wiki/Meetings/2013-06-27</a><br>
<br>
Given a robust set of automated browser tests and the ability to run<br>
them,<br>
let us then:<br>
<br>
* Identify deficiencies in the automated test coverage for the feature.<br>
This may or may not include analysis of the test code itself, but<br>
would<br>
certainly at least include analysis of the ATDD-style stated feature<br>
coverage in the Cucumber Scenarios. Do we have any technical debt<br>
in our<br>
test code?<br>
<br>
* Identify test charters for which automated testing is not possible and<br>
which are only testable by actual human beings. Are there tests that<br>
cannot be automated, and is such testing worthwhile?<br>
<br>
Outcomes:<br>
<br>
* Participants will be able to analyze ATDD-style automated browser<br>
tests<br>
* Participants will be able to run Cucumber + page_object browser<br>
tests and<br>
analyze the results of those tests<br>
* Participants will be able to demonstrate automated browser test<br>
practices<br>
with examples from the open WMF browser test code<br>
* Participants will be able to begin to contribute to WMF testing<br>
efforts<br>
if they wish, whether automated or not<br>
<br>
This might be too ambitious. I'm pretty sure this would be the most<br>
technically challenging session in the history of WTA. OTOH, we've<br>
already<br>
built the infrastructure to do this kind of thing, let's spread the word<br>
about what is possible.<br>
<br>
-Chris<br>
<br>
<br>
<br>
><br>
> Message: 2<br>
>> Date: Wed, 14 Aug 2013 08:56:54 -0700<br>
>> From: Chris McMahon <<a href="mailto:cmcmahon@wikimedia.org" target="_blank">cmcmahon@wikimedia.org</a><br></div></div>
<mailto:<a href="mailto:cmcmahon@wikimedia.org" target="_blank">cmcmahon@wikimedia.org</a><u></u>>><div class="im"><br>
>> To: "QA (software quality assurance) for Wikimedia projects."<br>
>><br></div>
>> <<a href="mailto:qa@lists.wikimedia.org" target="_blank">qa@lists.wikimedia.org</a> <mailto:<a href="mailto:qa@lists.wikimedia.org" target="_blank">qa@lists.wikimedia.org</a><u></u>>><div class="im">
<br>
>> Subject: Re: [QA] Weekend testing americas on September 7th<br>
>> Message-ID:<br>
>> <<br>
>><br>
<a href="mailto:CAJohBHQzWK5H9weGS-SERHe0faZRSS08WjiGw-UNb7ABe-xC0w@mail.gmail.com" target="_blank">CAJohBHQzWK5H9weGS-<u></u>SERHe0faZRSS08WjiGw-UNb7ABe-<u></u>xC0w@mail.gmail.com</a><br></div>
<mailto:<a href="mailto:CAJohBHQzWK5H9weGS-SERHe0faZRSS08WjiGw-UNb7ABe-xC0w@mail.gmail.com" target="_blank">CAJohBHQzWK5H9weGS-<u></u>SERHe0faZRSS08WjiGw-UNb7ABe-<u></u>xC0w@mail.gmail.com</a>>><div class="im">
<br>
>> Content-Type: text/plain; charset="iso-8859-1"<br>
>><br>
>><br>
>> I actually have an idea that I think would be of interest to WTA<br>
and of<br>
>> benefit to WMF, but I'd like to encourage others to reply first..<br>
>><br>
>> -C<br>
>><br>
>><br>
>><br>
>><br>
> ______________________________<u></u>_________________<br>
> QA mailing list<br></div>
> <a href="mailto:QA@lists.wikimedia.org" target="_blank">QA@lists.wikimedia.org</a> <mailto:<a href="mailto:QA@lists.wikimedia.org" target="_blank">QA@lists.wikimedia.org</a><u></u>><div class="im"><br>
> <a href="https://lists.wikimedia.org/mailman/listinfo/qa" target="_blank">https://lists.wikimedia.org/<u></u>mailman/listinfo/qa</a><br>
><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL:<br>
<<a href="http://lists.wikimedia.org/pipermail/qa/attachments/20130815/e6688311/attachment-0001.html" target="_blank">http://lists.wikimedia.org/<u></u>pipermail/qa/attachments/<u></u>20130815/e6688311/attachment-<u></u>0001.html</a>><br>
<br>
------------------------------<br>
<br>
______________________________<u></u>_________________<br>
QA mailing list<br></div>
<a href="mailto:QA@lists.wikimedia.org" target="_blank">QA@lists.wikimedia.org</a> <mailto:<a href="mailto:QA@lists.wikimedia.org" target="_blank">QA@lists.wikimedia.org</a><u></u>><div class="im"><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/qa" target="_blank">https://lists.wikimedia.org/<u></u>mailman/listinfo/qa</a><br>
<br>
<br>
End of QA Digest, Vol 4, Issue 13<br>
******************************<u></u>***<br>
<br>
<br>
<br>
<br></div><div class="im">
______________________________<u></u>_________________<br>
QA mailing list<br>
<a href="mailto:QA@lists.wikimedia.org" target="_blank">QA@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/qa" target="_blank">https://lists.wikimedia.org/<u></u>mailman/listinfo/qa</a><br>
<br>
</div></blockquote><span class="HOEnZb"><font color="#888888">
<br>
<br>
-- <br>
Quim Gil<br>
Technical Contributor Coordinator @ Wikimedia Foundation<br>
<a href="http://www.mediawiki.org/wiki/User:Qgil" target="_blank">http://www.mediawiki.org/wiki/<u></u>User:Qgil</a></font></span><div class="HOEnZb"><div class="h5"><br>
<br>
______________________________<u></u>_________________<br>
QA mailing list<br>
<a href="mailto:QA@lists.wikimedia.org" target="_blank">QA@lists.wikimedia.org</a><br>
<a href="https://lists.wikimedia.org/mailman/listinfo/qa" target="_blank">https://lists.wikimedia.org/<u></u>mailman/listinfo/qa</a><br>
</div></div></blockquote></div><br></div></div>