It's less about it being not appropriate for perf tests and more so
that you can't directly compare bare metal to a virtual instance. Load
testing virts against virts is fine as long as your factor the drift
in shared resourcing.
Agree that it's not apples to apples, but can't you still get average
memory and/or CPU usage? I only suggested it since I thought it would be
a cheaper way to run a cursory assessment, which is all I assumed we needed
at this point.
This is also why I mentioned gradual rollout and feature flags in my
earlier reply. Ideally we can defer establishing performance SLAs until
after we're ready to promote this from "experiment" to a first-class stack
citizen.
Anyway, on with the experiment! Definitely excited for us to learn more
about this as a team (and organization).
On Tue, Mar 10, 2015 at 2:02 PM, Tomasz Finc <tfinc(a)wikimedia.org> wrote:
> On Tue, Mar 10, 2015 at 10:49 AM, Max Semenik <msemenik(a)wikimedia.org>
> wrote:
> > As a shared virtualized environment, Labs is not appropriate for
> performance
> > measurements. Also, its SLA is too lax for production app use.
>
> Moving back to mobile-l@
It's less about it being not appropriate for perf tests and more so
that you can't directly compare bare metal to a virtual instance. Load
testing virts against virts is fine as long as your factor the drift
in shared resourcing.
>
> As for LABS production use I'm curious about would it would take to
> make this work in the future. LABS is a low cost env to spin up
> hardware that is easily discarded after. To me this is critical in
> supporting our engineers teams experiment with low cost.
>
> I know that isn't something that we do currently but either we need to
> get production into a state where its simpler to spin up new
> production instances/products/extensions/etc or we allow LABS to take
> some small production traffic for experiments.
>
> CC'ing Yuvi to get his thoughts.
>
> --tomasz
>
--
EN Wikipedia user page:
https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle