[QA] Flagging some Beta Labs issues

[S Page]

On Tue, Aug 19, 2014 at 12:11 PM, James Forrester <jforrester at wikimedia.org>
wrote:

> On 19 August 2014 11:19, S Page <spage at wikimedia.org> wrote:
>
>>
>> On Mon, Aug 18, 2014 at 4:35 PM, Maryana Pinchuk <mpinchuk at wikimedia.org>
>> wrote:
>>
>>> and Design/UX for running remote and in-person user tests
>>>
>>
>> That seems crazy. Beta labs constantly updates with the latest merged
>> changes to core and dozens of extensions, I think 288 times a day. Any
>> commit could break it, and they regularly do! Beta labs is there to test to
>> find breaking changes before they go live, not for user testing.
>>
>
> I disagree. ​It's generally a reliable platform from which to user-test
> master; it's rare for code to get merged into master that breaks anything,
> let alone breaks the whole cluster.
>

You may be right. It's been hard to tell after the fact if a code check-in
caused a transient failure in Flow automated tests on Beta labs. Beta labs
config changes, trying new things like HHVM/Cirrus/CentralLogin, and
operational issues may be bigger sources of the instability Maryana noted.
Performing UX testing with the public on a "generally reliable platform" is
a gamble, I'm glad it's working for you. I expect Beta labs to regularly be
unstable, know to retry and ping people when it fails, and am happy when it
works. Maybe Maryana and Abbey had different expectations.

Maybe we should document better how to set up a reasonably performant labs
>> instance with a decent set  of wiki pages, templates, images, etc. Then UX
>> can spin up ux-wikimania.wmflabs.org and be isolated from the firehose
>> of changes.
>>
>
> ​For the love of all that is good in the world, can we *please* not
> create yet another fork-wiki in Labs that someone needs to waste cycles
> updating
>

Labs instances are cheap, and reliable once set up. Create one, try
stuff, *delete
it*. It's the third step we're bad at :)

Cheers, and again thanks for Beta labs.

--
=S Page  Features engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140819/4ee030ea/attachment-0001.html>