To follow up with the lists, it seems that there are already continuously running tests that email mobile-l upon failure, but we were unlucky (or lucky, depending on your worldview) that failures weren't triggered because the core article reading experience in the mobile web in general continued to hum along.

Consequently, we're working with Michelle on defining a few positive and negative tests relating to image tags, banners, and external hyperlinks.

Thanks all!

-Adam

On Tue, Sep 17, 2013 at 11:32 AM, Michelle Grover <mgrover@wikimedia.org> wrote:

Sure can are these tests already setup in cloudbees to run? 

On Tue, Sep 17, 2013 at 11:17 AM, Adam Baso <abaso@wikimedia.org> wrote:

Is there a way to have the general mobile web automation tests run continuously, alerting #wikimedia-mobile and select individuals in the case there are failing tests?

Sorry if this has been asked and answered, or is already the case and I missed it somehow.

A bug introduced to ZeroRatedMobileAccess interfered with MobileFrontend yesterday. It was spotted manually. It would be cool to increase the odds of automatically being notified of failing tests from the general mobile web (the Wikipedia Zero automation tests do positive/negative tests for in-scope Wikipedia Zero, and only a couple of negative tests for the general mobile web). To be fair, tests won't always catch errant behavior, but they improve the odds!

Props to Arthur, Jon, Juliusz, and Max spotting the issue last night and Yuri and Yuvi (Yuri !== Yuvi) getting the fix in shortly thereafter (https://bugzilla.wikimedia.org/show_bug.cgi?id=54209).

Thanks.

-Adam

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

_______________________________________________
QA mailing list
QA@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/qa