This looks like a good time to fork this conversation as this is a good
problem to fix. How can we notify developers when they break other things
in the stack and how?
On 7 Mar 2014 05:36, "Bartosz Dziewoński" <matma.rex(a)gmail.com> wrote:
(continued, about the browser testing)
(tl;dr where are the tests and how do I know they fail?)
So, we have some slick browser tests. Awesome! But what's not good is
that the tests run off-site, the results are not reported back to
gerrit nor Bugzilla (unless someone manually files a bug, usually
Chris) not IRC nor anywhere else, and are generally non-discoverable
until someone shouts at you for breaking them. (As Tim guessed, I did
not know about any failures until Jon told me.)
In fact, I still have no idea what exactly the tests encompass (I've
heard about some browser tests for VE because I lurk a lot, never
heard of any for core) or where to find them or how to run them.
Either I'm slow or we have a serious documentation failure here.
Can something be done about it? Can we have the results reported
somewhere visible – preferably to gerrit, as jenkins already reports
some post-merge checks there? Or maybe we can have automatically filed
bug reports if the build breaks? A bot reporting test status on
#wikimedia-dev? Anything?
(I understand that the tests take too long to run them in the
pre-merge checks.)
(Jon proposed reverting problematic changes outright, but to me that
seems like a bit of an overreaction – bugs in tests and false
positives happen, let's not make a huge fuss out of that.)
(to be continued: about the deployment)
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l