On 25/03/15 21:37, Greg Grossmeier wrote: The MobileFrontend jobs send emails to chris and qa-alerts . It is configured JJB integration/config.git /jjb/browsertests.yaml via YAML aliases [1]. The project 'MobileFrontend' has: recipients: *emails-qa One would need to define a new alias for Mobile at the top of the file. Maybe the mobile team as an alert IRC notifications are only send to #wikimedia-releng. It needs a bit more work to get make it customizable. [1] https://git.wikimedia.org/blob/integration%2Fconfig/0b91d99/jjb%2Fbrowserte… -- Antoine Musso

On Thu, Mar 26, 2015 at 3:38 PM, Jon Robson <jdlrobson(a)gmail.com> wrote: We have a couple projects in the works that will hopefully help.[1][2] They don't go so far as to name and shame (we'll leave that up to you :), but they should promote more ownership over browser tests, better communication of failure, and analysis of failure over time and by test function (feature vs smoke vs regression). If many of these browser tests are serving as regression/smoke tests, it might be worthwhile to not only separate them out, but to remove some of the layers of abstraction to make tests more maintainable. I often try to think about tests in terms of a value-to-debt ratio (i.e. "How likely is it that I'll have to fix or refactor this test down the road and is it worth it?") and, while I find quite a bit of value in Cucumber for the sake of acceptance(-esque) testing (it keeps me mindful of the UX),[3] it introduces quite a bit of debt in its layers of abstraction and indirection (it's always a red flag when you have to grep to find your test implementation). Even its creators believe the value of Cucumber lies in its collaboration/planning capacities, not in its function as a test runner.[4] It's entirely possible, as of the 1.0.0 release, to use MW-Selenium without Cucumber, so perhaps we could experiment with simple RSpec examples for these types of tests. Tooling aside, there are broader discussions that need to take place among those that are practicing or have practiced TDD/BDD in the organization and how we might (or might not) promote theses practices among others. I'll be trying to form such a group next quarter (will it be a 'guild'?, a 'league'?, no need for tough decisions just yet[5]), so let's maintain a dialogue on that front if you're interested and willing. [1] https://phabricator.wikimedia.org/T88706 [2] https://phabricator.wikimedia.org/T89375 [3] https://gerrit.wikimedia.org/r/#/c/196492/10/features/role_settings.feature [4] https://cukes.info/blog/2014/03/03/the-worlds-most-misunderstood-collaborat… [5] https://www.youtube.com/watch?v=6KSiyaqnZYs Have you tried to reproduce these failures by executing the tests locally, against either Beta or your local MW? I'll be in the office tomorrow if you want advice on how to best go about it. Dan -- Dan Duvall Automation Engineer Wikimedia Foundation