[QA] [WikimediaMobile] Visual regression testing

[Chris McMahon]

Also keep in mind that there is a GSoC participant doing a very similar
project on behalf of the Language team:

Vikas S Yaligar
Automatic cross-language screenshots for user documentation


On Mon, Apr 21, 2014 at 11:42 AM, Tomasz Finc <tfinc at wikimedia.org> wrote:

> Great. Eager to see us test with this.
>
> Do we have any of its early screenshot comparisons up?
>
> --tomasz
>
> On Fri, Apr 18, 2014 at 4:33 PM, Juliusz Gonera <jgonera at wikimedia.org>
> wrote:
> > I conducted a little bit of research on visual regression testing, mainly
> > for the mobile team for now, but it could be easily reused for other
> teams.
> > I had a look at three existing solutions that seem somewhat popular and
> are
> > actively developed:
> >
> > * Wraith (https://github.com/BBC-News/wraith)
> > * PhantomCSS (https://github.com/Huddle/PhantomCSS)
> > * Huxley (https://github.com/facebook/huxley)
> >
> > They all have their own pros and cons, but in my opinion they share one
> > important disadvantage: they can't be easily integrated with our current
> > browser testing setup. For all the aforementioned tools we would have to
> > create from scratch a completely separate set of tests just for visual
> > regression testing instead of extending our existing browser tests.
> >
> > I spent a few hours in my spare time tinkering with an alternative idea
> > which would enable us to add visual regression testing to our
> Cucumber/Watir
> > tests. The result is a small prototype available at
> > https://github.com/jgonera/photographer. There is no docs or anything
> yet,
> > but I prepared a simple demo patch for MobileFrontend:
> > https://gerrit.wikimedia.org/r/#/c/126878/.
> >
> > The idea is to add a new method for Cucumber steps (snap) that takes a
> > screenshot of the current browser state and compares it with a screenshot
> > taken in one of the previous test runs. To update screenshots that are
> used
> > as a reference you run tests with env var PHOTOGRAPHER=update. If newly
> > taken screenshot differs by too many pixels from an old one, the test
> will
> > fail.
> >
> > It's still only an early prototype, but I'd appreciate any comments about
> > this idea.
> >
> > --
> > Juliusz
> >
> > _______________________________________________
> > Mobile-l mailing list
> > Mobile-l at lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140421/33f171fb/attachment.html>