The problem is it /can't/ be a random page for removing or adding from watchlist. We have to find a page that is already on the watchlist for the test for removing from watchlist and for the test adding to watchlist we have to ensure we land on a page which is not previously being watched.

We could possibly change these so that one of the Given statements is "Given I am already watching the "Foo" article" - this could then ensure the article is being watched and vice versa.

A random page cannot be relied on under any circumstances.



On Wed, Mar 19, 2014 at 2:37 PM, Chris McMahon <cmcmahon@wikimedia.org> wrote:

I fixed it. 

Since yesterday Cloudbees has unexpectedly given us unlimited Jenkins build executors where before we could only run two builds at once.  This has made for a few race conditions, of which this was one.  

This test should probably create a random page to use for clicking the watchlist on and off. 

-C 


On Wed, Mar 19, 2014 at 2:30 PM, Jon Robson <jrobson@wikimedia.org> wrote:
They are indeed false positives. I think I know what's happening here. Basically the tests are not atomic - they are trying to watch an article that is already watched and have the consequence of making that article watched. I have an idea of how to fix this.

Does Watir have a concept of a tear down step?


On Wed, Mar 19, 2014 at 1:45 PM, Arthur Richards <arichards@wikimedia.org> wrote:
I tested these manually and think they may be false positives, but can someone else take a look to make sure these tests and what's in betalabs is OK? We should get this resolved asap so we can be sure the state of things is healthy on betalabs for tomorrow's deployment.


On Wed, Mar 19, 2014 at 12:19 PM, <jenkins-no-reply@cloudbees.com> wrote:
FAILURE: MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox Build #434 (Wed, 19 Mar 2014 18:40:00 +0000)

Test Result

2 failed, 1 skipped

Failed Tests

Test NameDurationAge
Manage Watchlist.Add an article to the watchlist 37 sec1
Manage Watchlist.Remove an article from the watchlist 38 sec1

All Tests

PackageDurationFailSkipTotal
(root)30 min2161
Wikitext Editor (TEST RUN ON WIKIPEDIA 1 min 14 sec003




--
Arthur Richards
Software Engineer, Mobile
[[User:Awjrichards]]
IRC: awjr