<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Nov 15, 2013 at 12:28 PM, Jeff Hall <span dir="ltr"><<a href="mailto:jhall@wikimedia.org" target="_blank">jhall@wikimedia.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
I spent some time today looking at automated test failures via
CloudBees/Jenkins (<a href="https://wmf.ci.cloudbees.com" target="_blank">https://wmf.ci.cloudbees.com</a>), and a pretty
common theme of tests that fail inconsistently is
"Watir::Wait::TimeoutError" issues. Here's an example of a recent
failure that falls into this category:<br>
<br>
<a href="https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-chrome/463/testReport/%28root%29/UploadWizard/Navigate_to_Describe_page/" target="_blank">https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-chrome/463/testReport/(root)/UploadWizard/Navigate_to_Describe_page/</a></div>
</blockquote><div><br></div><div>Yes, this is a legitimate timeout issue, unlike the UW test failure on Nov 14, another of those maddening unexplained js errors from a ResourceLoader call: <a href="https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-chrome/461/">https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-chrome/461/</a></div>
<div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
From previous experience working with SauceLabs, I know that this is
not unusual, since by definition you're initiating a test workflow
that creates a lot of network traffic, and latencies are probably
inevitable.<br>
<br>
What I'm wondering is whether or not it might be a good idea to use
the page-object "wait_until" method more widely. For example, we
currently use it in <a href="https://github.com/wikimedia/qa-browsertests/blob/master/features/step_definitions/aftv5_steps.rb" target="_blank">aftv5_steps.rb</a>.<br></div></blockquote><div><br></div><div>You're right about the root cause, but in this case I would rather provide a longer timeout value to the when_present() call than the default 5 seconds. </div>
<div><br></div><div>You can see the line in question, line 17 at<font color="#333333" face="Consolas, Liberation Mono, Courier, monospace"><span style="font-size:12px;line-height:18px;white-space:pre"> </span></font><a href="https://github.com/wikimedia/qa-browsertests/blob/master/features/step_definitions/upload_wizard_steps.rb">https://github.com/wikimedia/qa-browsertests/blob/master/features/step_definitions/upload_wizard_steps.rb</a> which should show as</div>
<div><font color="#333333" face="Consolas, Liberation Mono, Courier, monospace"><span style="font-size:12px;line-height:18px;white-space:pre"><br></span></font></div><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">(</span><span class="" style="color:rgb(0,128,128);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">UploadPage</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">)</span><span class="" style="font-weight:bold;color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">.</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre;background-color:rgb(255,255,255)">continue_element</span><span class="" style="font-weight:bold;color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">.</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre;background-color:rgb(255,255,255)">when_present</span><span class="" style="font-weight:bold;color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">.</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre;background-color:rgb(255,255,255)">click</span></div>
<div class="gmail_quote"><font color="#333333" face="Consolas, Liberation Mono, Courier, monospace"><span style="font-size:12px;line-height:18px;white-space:pre"><br></span></font></div><div class="gmail_quote"><font color="#333333" face="Consolas, Liberation Mono, Courier, monospace"><span style="font-size:12px;line-height:18px;white-space:pre">We can make that <br>
</span></font><div><br></div><div> <span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">(</span><span class="" style="color:rgb(0,128,128);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">UploadPage</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">)</span><span class="" style="font-weight:bold;color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">.</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">continue_element</span><span class="" style="font-weight:bold;color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">.</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">when_present(15)</span><span class="" style="font-weight:bold;color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">.</span><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre">click</span></div>
<div><br></div><div><font color="#333333" face="Consolas, Liberation Mono, Courier, monospace"><span style="font-size:12px;line-height:18px;white-space:pre">and accomplish the same thing in a definite span of time, 15 seconds.</span></font></div>
<div><span class="" style="color:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;line-height:18px;white-space:pre"><br></span></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
I realize that adding any type of sleep or wait behavior to a test
just causes overall test execution time to increase, but I'm
thinking it's more important to have fewer failing tests overall, so
that folks can focus their trouble-shooting efforts on test failures
that may be a consequence of actual bugs (and not just timeouts).<br>
<br>
I'd love to hear other opinions on this topic, so please speak up if
you have an opinion ;)<br></div></blockquote><div><br></div><div>I agree that explicit sleep() calls are bad design, but I like polling for conditions:</div><div><br></div><div>when_present() polls until an element can be engaged</div>
<div>wait_until() polls until a condition returns true</div><div>wait_while() polls until a condition returns false</div><div><br></div><div>That aft5 test is a good example: </div><div><br></div><div><pre style="font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;margin-top:0px;margin-bottom:0px;color:rgb(51,51,51);line-height:18px">
<div class="" id="LC66" style="padding-left:10px"><span class="">page</span><span class="" style="font-weight:bold">.</span><span class="">wait_until</span><span class="">(</span><span class="" style="color:rgb(0,153,153)">10</span><span class="">)</span> <span class="" style="font-weight:bold">do</span></div>
<div class="" id="LC67" style="padding-left:10px"> <span class="">page</span><span class="" style="font-weight:bold">.</span><span class="">text</span><span class="" style="font-weight:bold">.</span><span class="">include?</span> <span class="" style="color:rgb(221,17,68)">'Thanks!'</span></div>
<div class="" id="LC68" style="padding-left:10px"> <span class="" style="font-weight:bold">end</span></div><div class="" id="LC69" style="padding-left:10px"> <span class="">page</span><span class="" style="font-weight:bold">.</span><span class="">text</span><span class="" style="font-weight:bold">.</span><span class="">should</span> <span class="" style="font-weight:bold">include</span> <span class="" style="color:rgb(221,17,68)">'Your post can be viewed on this feedback page.'</span></div>
<div class="" id="LC69" style="padding-left:10px"><br></div><div class="" id="LC69" style="padding-left:10px">says: "We'll hang out until the AFT post is processed. We know that the processing is finished when the page contains the text "Thanks". At that point we should have a message showing a link to the feedback page"</div>
</pre></div><div><br></div><div><br></div><div>-Chris </div></div></div></div>