<div dir="ltr">I'd like to meet with the brains in QA about Flow browser tests. Chris McMahon is going on vacation and Željko might be difficult to schedule. So here are some topics for a non-meeting. Thanks for ideas and comments, or we can meet later.<br>
<div class="" id="magicdomid3"><br></div><div class="" id="magicdomid4"><span class=""><b>Good news:</b></span></div><div class="" id="magicdomid5"><span class=""> HEADLESS=true MEDIAWIKI_USER=Selenium_user MEDIAWIKI_PASSWORD=XX MEDIAWIKI_URL=</span><span class=""><a href="http://localhost/wiki/index.php/">http://localhost/wiki/index.php/</a></span><span class=""> bundle exec cucumber features</span></div>
<div class="" id="magicdomid6"><br></div><div class="" id="magicdomid7"><span class="">all tests pass!<br><br></span></div><div class="" id="magicdomid7"><span class="">And almost all tests are passing in <a href="http://integration.wikimedia.org">integration.wikimedia.org</a>! Thanks Jon Robson et al.<br>
<br></span></div><div class="" id="magicdomid9"><span class=""><b><u>Improvements</u></b></span></div><div class="" id="magicdomid10"><br></div><div class="" id="magicdomid11"><span class=""><b>Speed up test setup</b></span></div>
<div class="" id="magicdomid12"><br></div><div class="" id="magicdomid13"><span class=""> Background runs once per scenario, when really I only need it once per feature (from Matt Flaschen)</span></div><div class="" id="magicdomid14">
<br></div><div class="" id="magicdomid15"><span class="">* <i>generally useful?</i></span><span class=""> Speed up the "Given I am logged in" by using an API call to login.</span></div><div class="" id="magicdomid16">
<br></div><div class="" id="magicdomid17"><span class="">If you look at e.g. </span><span class=""><a href="https://saucelabs.com/jobs/465f257ffdeb422e95b029c5f8df3446">https://saucelabs.com/jobs/465f257ffdeb422e95b029c5f8df3446</a></span><span class=""> , after login the test does</span></div>
<div class="" id="magicdomid18"><span class=""> POST elements</span></div><div class="" id="magicdomid19"><span class=""> using: "xpath"</span></div>
<div class="" id="magicdomid20"><span class=""> value: ".//a"</span><span class=""><br>then spends the next 8 seconds getting dozens of attribute hrefs from the Main_Page. And then it seems to repeat this. What is it doing?</span><span class=""></span></div>
<div class="" id="magicdomid23"><br></div><div class="" id="magicdomid24"><span class="">* Reuse
existing content instead of always creating new topics and posts. E.g.
for Load More/infinite scroll, instead of creating 11 topics, use
"Pending there are 11 topics"; for sorting, use "Pending there are more
than 2 topics" and add a reply to one of them; etc.<br></span></div><div class="" id="magicdomid25"><br></div><div class="" id="magicdomid26"><span class=""><b>Catch general failures</b></span></div><div class="" id="magicdomid27">
<br></div><div class="" id="magicdomid28"><span class=""><i>generally useful?</i></span><span class=""> Always check for ResourceLoader errors</span></div><div class="" id="magicdomid29"><ul class=""><li><span class="">We want a way (@RLcheck annotation?) to assert <br>
And page has no ResourceLoader errors<br>on start and end of every single page</span></li></ul></div><div class="" id="magicdomid32"><br></div><div class="" id="magicdomid34"><span class=""><i>generally useful?</i></span><span class=""> Always check for a pink errorbox on the page </span><span class=""><a href="https://bugzilla.wikimedia.org/show_bug.cgi?id=61304">https://bugzilla.wikimedia.org/show_bug.cgi?id=61304</a></span> <br>
<ul><li>Similarly this should be a default assertion.</li></ul><br></div><div class="" id="magicdomid35"><span class=""><b>Increase Flow browser test reliability</b></span></div><div class="" id="magicdomid36"><span class=""><br>
Problem:
Typical test creates a topic or post on Talk:Flow_QA, then checks elements in the
first topic or post on the board. But in Continuous Integration (and sometimes on
shared labs servers), other browsers are simultaneously hitting the same
page, and we get weird failures.</span></div><div class="" id="magicdomid37"><br></div><div class="" id="magicdomid38"><span class="">Solutions:</span></div><div class="" id="magicdomid39"><ul class=""><li><span class="">Start
a new board for some tests. This would mean adding a dedicated Flow_QA
namespace on test hosts, and adding it to $wgFlowOccupyNamespaces.</span></li></ul></div><div class="" id="magicdomid40"><ul class=""><li><span class="">Change tests to target </span><span class=""><i>the topic or post they just created.</i></span><span class=""> Tests know what the text of the topic is, so search for that and look within it for elements.</span></li>
</ul></div><div class="" id="magicdomid41"><br></div><div class="" id="magicdomid42"><span class="">Problem:
Flow tests require MEDIAWIKI_USER to have admin and oversight rights
because they assume block user, delete topic/post, and suppress
topic/post are available. Maybe tests should skip if the MEDIAWIKI_USER doesn't have the right. Or could we grant the Selenium_user these rights but only in a dedicated Flow_QA namespace? </span><span class=""><a href="https://bugzilla.wikimedia.org/show_bug.cgi?id=67158">https://bugzilla.wikimedia.org/show_bug.cgi?id=67158</a></span></div>
<div class="" id="magicdomid43"><br></div><div class="" id="magicdomid44"><br></div><div class="" id="magicdomid45"><span class=""><b>New features for new tests</b></span></div><div class="" id="magicdomid46"><br></div><div class="" id="magicdomid47">
<span class=""><i>generally useful</i></span><span class="">? To test Thank, mentions appearing in the thanked/mentioned user's notifications, and future subscription features, we need a second QA user.</span></div><div>
<br><br></div><div>Thanks,<br></div><div>-- <br><div dir="ltr">=S Page Features engineer<br></div>
</div></div>