[QA] [reading-wmf] Recently failing browser tests

[Gergo Tisza]

On Thu, Jun 4, 2015 at 11:23 AM, Jon Robson <jrobson at wikimedia.org> wrote:

> Another thing I'd be interested in exploring is using JavaScript to write
> tests since a lot more devs are comfortable with JS than Ruby but I suspect
> this is a big undertaking?
>

This is probably not what you meant but the ratio of browser tests to unit
tests (QUnit) in Gather seems very unhealthy to me. QUnit tests run in a
headless browser are very fast so they can be run pre-merge, they are less
flaky, fully parallel, and catching errors via unit tests is generally more
productive as they can pinpoint which part of the code is wrong while
browser tests usually need a fair amount of debugging. IMO if you are
looking for low-hanging fruits then your time is better spent on writing
more QUnit tests than doing complicated things with browser tests.

Re: browser test language, there has been some debate about this (both PHP
and JS have mature gherkin-based tools, and most developers are more
familiar with them than with Ruby), but the opinion of QA at the time was
that a language which the browser testing community tends to be familiar
with is better than a language the MediaWiki community tends to be familiar
with.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/qa/attachments/20150604/9584e073/attachment.html>