<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
The way I've dealt with this in the past (prior to working at the
WMF) was to create Git branches for automated tests that were
associated with specific upcoming release cycles. Continuous
integration tools like Jenkins make it easy to manage builds for
multiple branches, so the procedural overhead was small.<br>
<br>
This worked really well where the functional test code lived in a
separate repository from the application code under test. It might
be a little more complicated to implement where the tests are part
of the application repo (per the WMF practice), but should still be
doable.<br>
<br>
That's my 2 cents!<br>
<br>
- Jeff<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 2/5/14, 3:10 PM, Chris McMahon
wrote:<br>
</div>
<blockquote
cite="mid:CAJohBHRvoBbbb4Cc8Ne=oJkh2eub65DGV77Cp37BbKUk+-ahqA@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div><br>
</div>
<div>Having browser tests in the repos of the extensions being
tested is Phase 0. {{status|done}}</div>
<div><br>
</div>
<div>Now we need to find a way to associate browser tests with a
particular version or versions in time of the extension being
tested. </div>
<div><br>
</div>
<div>Then we we would be able to pull master branch of tests to
target beta, pull pre-release branch of tests to target
test2wiki, pull production branch of tests to target prod.
This is on my list to investigate in the next quarter.
Suggestions are welcome. </div>
<div><br>
</div>
<div>-Chris </div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Feb 5, 2014 at 2:42 PM, Jon
Robson <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jrobson@wikimedia.org" target="_blank">jrobson@wikimedia.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Anyone?<br>
<div class="HOEnZb">
<div class="h5"><br>
<br>
On Fri, Jan 31, 2014 at 3:41 PM, Jon Robson <<a
moz-do-not-send="true"
href="mailto:jrobson@wikimedia.org">jrobson@wikimedia.org</a>>
wrote:<br>
> I'm not sure if any of these have bugs but please
advise on the best<br>
> approach to deal with them.<br>
><br>
> 1) Various tests fail because they rely on code
that has been merged<br>
> to master but not deployed. e.g. new tests for code
that has been<br>
> merged to master, will fail on enwiki until enwiki
gets the new code<br>
> deployed.<br>
> Is there anyway we can run the tests from the
master branch of the<br>
> project they belong to, to stop this from
happening?<br>
><br>
><br>
> 2) Due to bugs some tests randomly fail e.g.<br>
> <a moz-do-not-send="true"
href="https://bugzilla.wikimedia.org/show_bug.cgi?id=60338"
target="_blank">https://bugzilla.wikimedia.org/show_bug.cgi?id=60338</a><br>
> Is there anyway we can flag these different in the
email?<br>
><br>
> Note: I have actually started filtering out mails
from<br>
> <a moz-do-not-send="true"
href="http://test2.m.wikipedia.org" target="_blank">test2.m.wikipedia.org</a>
and <a moz-do-not-send="true"
href="http://en.m.wikipedia.org" target="_blank">en.m.wikipedia.org</a>
because of point 1.<br>
<br>
_______________________________________________<br>
QA mailing list<br>
<a moz-do-not-send="true"
href="mailto:QA@lists.wikimedia.org">QA@lists.wikimedia.org</a><br>
<a moz-do-not-send="true"
href="https://lists.wikimedia.org/mailman/listinfo/qa"
target="_blank">https://lists.wikimedia.org/mailman/listinfo/qa</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
QA mailing list
<a class="moz-txt-link-abbreviated" href="mailto:QA@lists.wikimedia.org">QA@lists.wikimedia.org</a>
<a class="moz-txt-link-freetext" href="https://lists.wikimedia.org/mailman/listinfo/qa">https://lists.wikimedia.org/mailman/listinfo/qa</a>
</pre>
</blockquote>
<br>
</body>
</html>