The problem is it /can't/ be a random page for removing or adding from watchlist. We have to find a page that is already on the watchlist for the test for removing from watchlist and for the test adding to watchlist we have to ensure we land on a page which is not previously being watched.
We could possibly change these so that one of the Given statements is "Given I am already watching the "Foo" article" - this could then ensure the article is being watched and vice versa.
A random page cannot be relied on under any circumstances.
On Wed, Mar 19, 2014 at 2:37 PM, Chris McMahon cmcmahon@wikimedia.orgwrote:
I fixed it.
Since yesterday Cloudbees has unexpectedly given us unlimited Jenkins build executors where before we could only run two builds at once. This has made for a few race conditions, of which this was one.
This test should probably create a random page to use for clicking the watchlist on and off.
-C
On Wed, Mar 19, 2014 at 2:30 PM, Jon Robson jrobson@wikimedia.org wrote:
They are indeed false positives. I think I know what's happening here. Basically the tests are not atomic - they are trying to watch an article that is already watched and have the consequence of making that article watched. I have an idea of how to fix this.
Does Watir have a concept of a tear down step?
On Wed, Mar 19, 2014 at 1:45 PM, Arthur Richards <arichards@wikimedia.org
wrote:
I tested these manually and think they may be false positives, but can someone else take a look to make sure these tests and what's in betalabs is OK? We should get this resolved asap so we can be sure the state of things is healthy on betalabs for tomorrow's deployment.
On Wed, Mar 19, 2014 at 12:19 PM, jenkins-no-reply@cloudbees.comwrote:
- FAILURE:
MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox Build #434 https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/434/ (Wed, 19 Mar 2014 18:40:00 +0000)* *Test Result*https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/434/testReport/ 2 failed, 1 skipped Failed Tests *Test Name**Duration**Age* Manage Watchlist.Add an article to the watchlisthttps://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/434/testReport/junit/(root)/Manage%20Watchlist/Add_an_article_to_the_watchlist 37 sec1https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/433/ Manage Watchlist.Remove an article from the watchlisthttps://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/434/testReport/junit/(root)/Manage%20Watchlist/Remove_an_article_from_the_watchlist 38 sec1https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/433/ All Tests *Package**Duration**Fail**Skip**Total* (root)https://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/434/testReport/junit/(root)30 min2161 Wikitext Editor (TEST RUN ON WIKIPEDIAhttps://wmf.ci.cloudbees.com/job/MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox/434/testReport/junit/Wikitext%20Editor%20(TEST%20RUN%20ON%20WIKIPEDIA 1 min 14 sec003
-- Arthur Richards Software Engineer, Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687