On Wed, Nov 21, 2012 at 3:45 PM, Quim Gil <qgil(a)wikimedia.org> wrote:
Here is a first stab for a draft proposal to organize
our volunteer
testing activities:
http://www.mediawiki.org/wiki/**Talk:QA/Strategy#Manual_**testing_strategy&…
Written after some lousy discussions with Chris and Sumana, and reading a
bunch of related wiki pages. Your feedback is welcome.
Ideally this _theory_ will be immediately applicable to some pilots that
we can run in the upcoming weeks. The Language and Mobile teams seem to be
ready for a try - maybe even before the end of the year. Visual Editor and
Editor Engagement teams might come next in January.
This proposal feels detached from reality. Right now features teams mostly
do one of the following, in my experience:
1). Product managers and developers do their own manual QA. For PMs this
aligns with verifying requirements, for developers it's checking their own
work. It can be a pain in the ass but it works for the most part.
2). A lucky few teams have dedicated QA help, like mobile.
In either situation, manual QA tends to be done on a tight deadline,
requires an intimiate understanding of the goals and requirements, and
within a very specific scope.
I don't have a lot of experience working at a large open source project, as
a caveat, so I haven't had the opportunity to see volunteer QA in action.
But considering my current working situation, I would rather continue doing
my own QA rather than rely on a volunteer who cannot be held to a deadline
and is not personally responsible for the quality of our work. The only
solutions in my mind are A) much more robust automated testing. B) hiring
experienced QA people. Anything else is just going to slow us down.
Steven