[QA] Flagging some Beta Labs issues

[Chris McMahon]

On Tue, Aug 19, 2014 at 4:06 PM, Maryana Pinchuk <mpinchuk at wikimedia.org>
wrote:

> On Tue, Aug 19, 2014 at 3:06 PM, S Page <spage at wikimedia.org> wrote:
>
>>
>> Labs instances are cheap, and reliable once set up.
>>
>
> Err, no, not at all!
>
> For one: setting up a labs instance requires a software engineer – which I
> am not, nor is anyone on the Design/UX team. And that's just setting up a
> totally empty vanilla install; if you want to test anything in any
> semblance of a production-like environment, you need to import tons of
> content, templates, extensions, scripts
>

We're investing in vagrant right now for just these reasons, and Dan Duvall
(marxarelli on IRC) is the point person for supporting useful
configurations in vagrant. We were discussing the possibility of using the
vagrant config to provision labs instances, also. We would like to get to
where non-software-engineers can flip a minimum of switches and get a
useful dev/test env quickly.  Dan is in SF, so do please discuss this sort
of thing with him when you see him.


> So, to be clear, I'm very much in favor of keeping *one* test environment
> that we all share, and it's okay if some things are sometimes buggy or
> weird in the process. Having another environment that simply mirrors
> production (a.k.a. what it sounds like you're saying Beta Labs was
> originally scoped as, Chris) would be great, too, but it's not as valuable
> as having a place to test things that aren't yet in production before they
> go live (a.k.a. the master branch).
>

What I would like (that I am not entirely sure we can get to, but I want to
try) is "beta1" that runs the pre-release master branch of everything now
in prod *and nothing but what is in prod*; and beta2 that runs the
pre-release master branch of everything in prod *plus whatever will be in
prod eventually but is not now*.  They would both be targets for automated
and exploratory testing, but beta1 would theoretically have fewer
surprises.  Hypothetically, if we had this arrangement right now today,
beta1 and beta2 would have the same code, but HHVM would be running only in
beta2, since it is not yet in production, but will be.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140819/65dc88b1/attachment.html>