On Thu, Sep 23, 2010 at 9:46 AM, Dan Nessett dnessett@yahoo.com wrote:
I am very much in favor of keeping it simple. I think the issue is whether we will support more than one regression test (or individual test associated with a regression test) running concurrently on the same test wiki. If not, then I agree, no switching logic is necessary.
*nod*
It might be a good idea to divide the test set up into, say 'channels' or 'bundles' which are independent of each other, but whose individual steps must run in sequence. If the tests are designed well, you should be able to run tests from multiple 'channels' on the same wiki simultaneously -- just as in the real world, multiple users are doing multiple things on your wiki at the same time.
So one test set might be: * create page A-{{unique-id}} as user One-{{unique-id}} * open editing page as user One-{{unique-id}} * open and save the page as user Two-{{unique-id}} * save the page as user One-{{unique-id}} * confirm edit conflict / merging behavior was as expected
And another might be: * register a new user account User--{{unique-id}} * change skin option in preferences * confirm that the skin changed as expected
These tests don't interfere with each other -- indeed if they did, that would be information you'd need to know about a serious bug!
Most test sets should be fairly separate like this; only some that change global state (say, a site administrator using a global configuration panel to change the default skin) would need to be run separately.
-- brion