[QA] Two big improvements for flaky tests

[Greg Grossmeier]

These are huge. Thanks Chris for the perseverance here. Well done :)

<quote name="Chris McMahon" date="2014-07-25" time="09:29:34 -0700">
> This month we've made two big improvements that should greatly reduce the
> number of flaky test failures in all of the builds.
> 
> For one thing, the MobileFrontend test that used to walk through a dozen
> steps to protect a page in the UI and then logout, is now using the API to
> protect the test page, and not logging out prevents flaky failures when
> another test that expects to be logged-in unexpectedly gets logged out.  We
> have seen this before in VisualEditor tests in particular.
> 
> Another thing is that we should no longer see the beta labs db become
> read-only.  Some time ago we had requests from the Language team to be able
> to test in situations where there is replication lag between master and
> slave dbs.  At the same time, the Flow team wanted to be able to test
> deploying in a situation with master and slave dbs. So we created a slave
> db for beta labs.
> 
> What we only discovered recently is that there is a process that monitors
> the replication lag time between master and slave dbs, and when that time
> goes over 5 seconds, the db is set to read-only. This is fine in
> production, but it was causing a lot of problems in beta labs.  We have now
> set the value for replication lag to 5 minutes instead of 5 seconds, and
> this has stopped a number of kinds of faillures:
> 
> * the explicit "database is read-only" failure from the VisualEditor tests
> * the generic "Save failed" failure message in MobileFrontend tests
> * the unexplained 90-second timeouts in Flow tests
> 
> Sorry for the inconvenience, but things should be much improved from now
> on.
> -Chris

> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa


-- 
| Greg Grossmeier            GPG: B2FA 27B1 F7EB D327 6B8E |
| identi.ca: @greg                A18D 1138 8E47 FAC8 1C7D |