Over the last week Željko and I changed the QA automated browser tests to target test environments on the beta cluster as well as on test2wiki, where they had been running for some time.

This was possible because of a number of improvements made to beta since the QA Quarterly Review and the f2f meeting in SF in February:

* automatic updates to the beta database * enabling Search on beta * Varnish on beta to support MobileFrontend * etc.

There is still more to do:

* get more extensions running and configured on beta * change settings to more closely match prod and/or test2 as appropriate * moar tests

Automated browser tests have a proven record of exposing real issues in the test2wiki environment before code is put in production. Here are some of the bugs we have identified by way of browser tests so far: https://bugzilla.wikimedia.org/buglist.cgi?bug_status=UNCONFIRMED&bug_st...

By running these tests in the beta cluster also, we gain both better test coverage features in disparate environments and also a longer window to identify issues, since beta is updated much more often than test2wiki.

As always, the current status of the browser test builds are available in Jenkins at https://wmf.ci.cloudbees.com, and any changes to the status of those builds are reported by wmf-jenkins-bot in #wikimedia-dev