We just pushed some new UI elements from beta to stable which has now hit beta labs. These haven't propagated to test2 or enwiki yet so these tests are now failing.

How best should we do prevent these failures from recording while these servers haven't been updated?

Is there anyway browser tests can be clever and run themselves based on the current commit on master of the server they are hitting?

On Thu, Jan 23, 2014 at 1:36 PM, <jenkins-no-reply@cloudbees.com> wrote:

FAILURE: MobileFrontend-test2.m.wikipedia.org-linux-firefox Build #218 (Thu, 23 Jan 2014 21:13:13 +0000)

Test Result

10 failed, 3 skipped

Failed Tests

Test Name Duration Age

Editor.Closing editor (browser button) 30 sec 2

Editor.Closing editor (overlay button) 31 sec 2

Language selection.Closing language overlay (browser button) 22 sec 2

Lead image tutorial.See tutorial as new user 31 sec 2

Lead image uploads.Closing upload preview (browser button) 32 sec 2

Lead image uploads.Closing upload preview (overlay button) 34 sec 2

Notification.Closing notifications (overlay button) 29 sec 2

Notification.Opening notifications 1.1 sec 2

Search.Opening search 4 ms 2

Special:Uploads tutorial.Link to tutorial for new user 32 sec 2

All Tests

Package Duration Fail Skip Total

(root) 16 min 10 3 51

Test Name	Duration	Age
Editor.Closing editor (browser button)	30 sec	2
Editor.Closing editor (overlay button)	31 sec	2
Language selection.Closing language overlay (browser button)	22 sec	2
Lead image tutorial.See tutorial as new user	31 sec	2
Lead image uploads.Closing upload preview (browser button)	32 sec	2
Lead image uploads.Closing upload preview (overlay button)	34 sec	2
Notification.Closing notifications (overlay button)	29 sec	2
Notification.Opening notifications	1.1 sec	2
Search.Opening search	4 ms	2
Special:Uploads tutorial.Link to tutorial for new user	32 sec	2

Package	Duration	Fail	Skip	Total
(root)	16 min	10	3	51