[QA] [WikimediaMobile] Visual regression testing

[Chris McMahon]

Language people also working on screen shots:
https://gerrit.wikimedia.org/r/#/c/127899/


On Mon, Apr 21, 2014 at 2:25 PM, Juliusz Gonera <jgonera at wikimedia.org>wrote:

> Nope, it's an early prototype. It basically only makes an updated Cucumber
> test fail if new screenshot differs from reference screenshot by more than
> X%. Adding visual comparison of two screenshots would be the next step and
> would not be too hard. I might work on it one evening this week.
>
>
>
> On 04/21/2014 11:42 AM, Tomasz Finc wrote:
>
>> Great. Eager to see us test with this.
>>
>> Do we have any of its early screenshot comparisons up?
>>
>> --tomasz
>>
>> On Fri, Apr 18, 2014 at 4:33 PM, Juliusz Gonera <jgonera at wikimedia.org>
>> wrote:
>>
>>> I conducted a little bit of research on visual regression testing, mainly
>>> for the mobile team for now, but it could be easily reused for other
>>> teams.
>>> I had a look at three existing solutions that seem somewhat popular and
>>> are
>>> actively developed:
>>>
>>> * Wraith (https://github.com/BBC-News/wraith)
>>> * PhantomCSS (https://github.com/Huddle/PhantomCSS)
>>> * Huxley (https://github.com/facebook/huxley)
>>>
>>> They all have their own pros and cons, but in my opinion they share one
>>> important disadvantage: they can't be easily integrated with our current
>>> browser testing setup. For all the aforementioned tools we would have to
>>> create from scratch a completely separate set of tests just for visual
>>> regression testing instead of extending our existing browser tests.
>>>
>>> I spent a few hours in my spare time tinkering with an alternative idea
>>> which would enable us to add visual regression testing to our
>>> Cucumber/Watir
>>> tests. The result is a small prototype available at
>>> https://github.com/jgonera/photographer. There is no docs or anything
>>> yet,
>>> but I prepared a simple demo patch for MobileFrontend:
>>> https://gerrit.wikimedia.org/r/#/c/126878/.
>>>
>>> The idea is to add a new method for Cucumber steps (snap) that takes a
>>> screenshot of the current browser state and compares it with a screenshot
>>> taken in one of the previous test runs. To update screenshots that are
>>> used
>>> as a reference you run tests with env var PHOTOGRAPHER=update. If newly
>>> taken screenshot differs by too many pixels from an old one, the test
>>> will
>>> fail.
>>>
>>> It's still only an early prototype, but I'd appreciate any comments about
>>> this idea.
>>>
>>> --
>>> Juliusz
>>>
>>> _______________________________________________
>>> Mobile-l mailing list
>>> Mobile-l at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>
>>
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140422/95bc7eac/attachment.html>