So the script that actually runs the browser test is not in a generic
useful form but it is:
https://gist.github.com/jdlrobson/32b607f8009e897ee80c

It uses the GerritCommandLine tool to do grabbing and reviewing
https://github.com/jdlrobson/GerritCommandLine

Ideally if we can use labs-tools-gerrit-to-redis for identifying
patches and then pulling them down we wouldn't need the
GerritCommandLine tool since the code to submit a review is pretty
trivial and captured in this function:
https://github.com/jdlrobson/GerritCommandLine/blob/master/gerrit.py#L277

I've also put this in the task
https://phabricator.wikimedia.org/T101069#1379462

We'll probably want an instance per extension, to simplify having to
worry about dependencies (we can just run a git update on all
extensions after each checkout)

On Thu, Jun 18, 2015 at 4:32 AM, Joaquin Oltra Hernandez
<jhernandez@wikimedia.org> wrote:
> Awesome Jon! I'm so happy to finally see this developing :DD
>
> Loving the : `Browserbot happy!`
>
> I've noticed it can report either the name of the failing test or the full
> log. What do you think if we show that, and a url with the pasted log
> somewhere publicly to not put too much noise on the comments but still be
> able to see it? Something like https://phabricator.wikimedia.org/paste/...
>
> +1 to where is the source.
> +1 to documenting how you've set it all up on wiki somewhere.
>
> I also think we need a catchy phrase for the -1s!
>
> Thanks for you work on this, we'll get more focused time for it soon.
>
> On Thu, Jun 18, 2015 at 11:33 AM, Sam Smith <samsmith@wikimedia.org> wrote:
>>
>> I agree with Florian everything that you've written should be in a public
>> version control system.
>>
>> Second, I'd ask that you document your experiences so far in getting this
>> set up and how it works so that other members of the vertical can help to
>> maintain it moving forward.
>>
>> Third, great work!!1
>>
>> <3
>>
>> –Sam
>>
>> On Thu, Jun 18, 2015 at 7:09 AM, florian.schmidt.welzow@t-online.de
>> <florian.schmidt.welzow@t-online.de> wrote:
>>>
>>>
>>> > It's currently working via a script that you can find here:
>>> > /srv/mediawiki/extensions/Gather/tests/browser/Barry.sh
>>>
>>> It would be great to have the script in a public version control system
>>> (e.g. github?), especially for people, e.g. volunteers, who can't ssh to
>>> gather-browser-tests.eqiad.wmflabs[1]
>>>
>>> [1] all people, who're not members of
>>> https://wikitech.wikimedia.org/wiki/Nova_Resource:Mobile-smoketests
>>>
>>> Best,
>>> Florian
>>>
>>> -----Original-Nachricht-----
>>> Betreff: [WikimediaMobile] [Update] Browser tests per patch
>>> Datum: Thu, 18 Jun 2015 03:27:32 +0200
>>> Von: Jon Robson <jrobson@wikimedia.org>
>>> An: "QA (software quality assurance) for Wikimedia projects."
>>> <qa@lists.wikimedia.org>, mobile-l <mobile-l@lists.wikimedia.org>
>>>
>>> Background: mobile wants to gain more confidence in its browser tests
>>> by running a subset of browser tests on a case by case basis [0].
>>>
>>> Good news: I've got a proof of concept running and Barry the browser
>>> test bot has given some legitimate helpful reviews to Gather [1].
>>>
>>> Even better news: It's proving itself valuable already [2].
>>> As you can see in the messages the bot has posted on [3] we have a
>>> couple of options on display option format for his reviews.
>>>
>>> So.. hopefully this short experience has sold you all already.
>>>
>>> This script is currently a manual job and needs a bit of tweaking
>>> before we can put it in a cron job/run it always - it needs to watch
>>> for new commits and then run a modification of the above script on a
>>> per case basis (if two versions of it run in parallel we have an
>>> issue).
>>>
>>> Definitely something we should push for next sprint!
>>>
>>> Long live Barry bot!
>>>
>>> Devs... (everyone else now of what follows is likely to be useful):
>>> I got the labs instance up and running on:
>>> http://gather-browser-tests.wmflabs.org/wiki/Main_Page
>>>
>>> Most of you in readership team should be able to ssh
>>> gather-browser-tests.eqiad.wmflabs
>>> Let me know if you have no access.
>>>
>>> It's currently working via a script that you can find here:
>>> /srv/mediawiki/extensions/Gather/tests/browser/Barry.sh
>>>
>>> [0] https://phabricator.wikimedia.org/T100293
>>> [1]
>>> https://gerrit.wikimedia.org/r/#/q/reviewer:jdlrobson%252Bbarry%2540gmail.com+status:open,n,z
>>> [2] https://gerrit.wikimedia.org/r/#/c/218731/
>>>
>>> _______________________________________________
>>> Mobile-l mailing list
>>> Mobile-l@lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mobile-l mailing list
>>> Mobile-l@lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>>
>>
>> _______________________________________________
>> Mobile-l mailing list
>> Mobile-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>
>
> _______________________________________________
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>

--
Jon Robson
* http://jonrobson.me.uk
* https://www.facebook.com/jonrobson
* @rakugojon

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l