[QA] Two big improvements for flaky tests

[Arthur Richards]

Wonderful news, thank you Chris :)

Is https://bugzilla.wikimedia.org/show_bug.cgi?id=68465 related to the db
issues that are now resolved, or is that a different issue?


On Fri, Jul 25, 2014 at 9:29 AM, Chris McMahon <cmcmahon at wikimedia.org>
wrote:

>
> This month we've made two big improvements that should greatly reduce the
> number of flaky test failures in all of the builds.
>
> For one thing, the MobileFrontend test that used to walk through a dozen
> steps to protect a page in the UI and then logout, is now using the API to
> protect the test page, and not logging out prevents flaky failures when
> another test that expects to be logged-in unexpectedly gets logged out.  We
> have seen this before in VisualEditor tests in particular.
>
> Another thing is that we should no longer see the beta labs db become
> read-only.  Some time ago we had requests from the Language team to be able
> to test in situations where there is replication lag between master and
> slave dbs.  At the same time, the Flow team wanted to be able to test
> deploying in a situation with master and slave dbs. So we created a slave
> db for beta labs.
>
> What we only discovered recently is that there is a process that monitors
> the replication lag time between master and slave dbs, and when that time
> goes over 5 seconds, the db is set to read-only. This is fine in
> production, but it was causing a lot of problems in beta labs.  We have now
> set the value for replication lag to 5 minutes instead of 5 seconds, and
> this has stopped a number of kinds of faillures:
>
> * the explicit "database is read-only" failure from the VisualEditor tests
> * the generic "Save failed" failure message in MobileFrontend tests
> * the unexplained 90-second timeouts in Flow tests
>
> Sorry for the inconvenience, but things should be much improved from now
> on.
> -Chris
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa
>
>


-- 
Arthur Richards
Team Practices Manager
[[User:Awjrichards]]
IRC: awjr
+1-415-839-6885 x6687
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140725/d6b085bc/attachment-0001.html>