[QA] [WikimediaMobile] Visual regression testing

[Tomasz Finc]

Great. Eager to see us test with this.

Do we have any of its early screenshot comparisons up?

--tomasz

On Fri, Apr 18, 2014 at 4:33 PM, Juliusz Gonera <jgonera at wikimedia.org> wrote:
> I conducted a little bit of research on visual regression testing, mainly
> for the mobile team for now, but it could be easily reused for other teams.
> I had a look at three existing solutions that seem somewhat popular and are
> actively developed:
>
> * Wraith (https://github.com/BBC-News/wraith)
> * PhantomCSS (https://github.com/Huddle/PhantomCSS)
> * Huxley (https://github.com/facebook/huxley)
>
> They all have their own pros and cons, but in my opinion they share one
> important disadvantage: they can't be easily integrated with our current
> browser testing setup. For all the aforementioned tools we would have to
> create from scratch a completely separate set of tests just for visual
> regression testing instead of extending our existing browser tests.
>
> I spent a few hours in my spare time tinkering with an alternative idea
> which would enable us to add visual regression testing to our Cucumber/Watir
> tests. The result is a small prototype available at
> https://github.com/jgonera/photographer. There is no docs or anything yet,
> but I prepared a simple demo patch for MobileFrontend:
> https://gerrit.wikimedia.org/r/#/c/126878/.
>
> The idea is to add a new method for Cucumber steps (snap) that takes a
> screenshot of the current browser state and compares it with a screenshot
> taken in one of the previous test runs. To update screenshots that are used
> as a reference you run tests with env var PHOTOGRAPHER=update. If newly
> taken screenshot differs by too many pixels from an old one, the test will
> fail.
>
> It's still only an early prototype, but I'd appreciate any comments about
> this idea.
>
> --
> Juliusz
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l