Some of you may have noticed a bot [1] providing reviews for the Mobilefrontend and Gather extensions.

This is a grass routes experiment [2] to see if we can reduce regressions by running browser tests against every single commit. It's very crude, and we're going to have to maintain it but we see this as a crude stop gap solution until we get gerrit-bot taking care of this for us.

Obviously we want to do this for all extensions but we wanted to get something good enough that is not scaleable to start exploring this.

So far it has caught various bugs for us and our browser test builds are starting to finally becoming consistently green, a few beta labs flakes aside [3].

Running tests on beta labs is still useful but now we can use it to identify tests caused by other extensions. We were finding too often our tests were failing due to us neglecting them.

In case others are interested in how this is working and want to set one up themselves I've documented this here: https://www.mediawiki.org/wiki/Reading/Setting_up_a_browser_test_bot

Please let me now if you have any questions and feel free to edit and improve this page. If you want to jump into the code that's doing this and know Python check out: https://github.com/jdlrobson/Barry-the-Browser-Test-Bot (Patches welcomed and apologies in advance for the code)

[1] https://gerrit.wikimedia.org/r/#/q/reviewer:jdlrobson%252Bbarry%2540gmail.co... [2] https://phabricator.wikimedia.org/T100293 [3] https://integration.wikimedia.org/ci/view/Mobile/job/browsertests-MobileFron...