[QA] Acceptance tests that require account creation

[Jeff Hall]

FYI, after some consultation with Chris, we decided to implement 
Mediawiki API setup steps in the MobileFrontEnd repo first, so all the 
bits now live in this commit[1]

As things stand now, only wiki articles are being created via the API 
before each test Scenario is run, but code for user account creation is 
part of that commit as well (although that method doesn't currently get 
invoked).

So if this solution seems workable to all concerned, it will accomplish 
the goal of allowing each browser test to create whatever wiki articles 
it needs to satisfy test assertions, and we can easily expand the scope 
later to also allow each browser test to create user logins as needed.

- Jeff


[1] https://gerrit.wikimedia.org/r/#/c/106548/




On 1/7/14, 8:26 AM, Chris McMahon wrote:
>
> I'm Ccing the QA mail list, this might be the place for this discussion.
>
> What you're talking about is often called "hermetic" test 
> environments.  Where at run time a test environment is created 
> sufficient to support the tests being run.  Right now we're thinking 
> about this in two different ways:
>
> One is, as you mentioned, creating a set of tests that are intended to 
> run in a bare wiki with no test data.  We're identifying these tests 
> now in order to create a PhantomJS smoke test suite.
>
> We are also exploring using the Mediawiki API to create or update 
> pages and users at run time to avoid the overhead of doing it through 
> the UI.  I've had Jeff Hall start looking into this: 
> https://gerrit.wikimedia.org/r/#/c/105835/ .
>
> Another avenue we have discussed is to use Vagrant.  I haven't gone 
> very far down that path, but it should be possible to share a Vagrant 
> image with essential test data on it.
>
> As to why we don't have this today, my first priority was to create a 
> viable *shared* test environment with shared test data.  We needed 
> beta labs and test2wiki working properly to support fast deployments 
> and to avoid the "works on my machine" problems.  Now that beta and 
> test2 are functioning reasonably well, we can turn attention to the 
> hermetic test environments.
>
> -Chris
>
>
> On Mon, Jan 6, 2014 at 6:43 PM, Jon Robson <jrobson at wikimedia.org 
> <mailto:jrobson at wikimedia.org>> wrote:
>
>     The best way to get round this would be to setup a new database every
>     time the tests begin and remove it at the end of the tests.
>     Somewhat related to
>     https://bugzilla.wikimedia.org/show_bug.cgi?id=56057
>
>     Chris any ideas?
>
>
>     On Mon, Jan 6, 2014 at 5:33 PM, Ryan Kaldari
>     <rkaldari at wikimedia.org <mailto:rkaldari at wikimedia.org>> wrote:
>     > Right now, we have a lot of important features that don't have any
>     > acceptance tests because in order to test them you have to
>     create a new
>     > account. This would result in thousands of bogus accounts being
>     created on
>     > all the wikis we test on. Does anyone have any ideas about how
>     we could
>     > solve this problem?
>     >
>     > Ryan Kaldari
>
>
>
>
> _______________________________________________
> QA mailing list
> QA at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/qa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140113/f12fc2dd/attachment.html>