[Foundation-l] Usability Study Results (Sneak Preview)

[Brian]

Ok, I'll agree that the motiviations and size of this pilot study are
reasonable. Then I'd just like to know how much money was spent getting
these answers.  If you're not planning to measure the subjects
scientifically and you just want to figure out what the big issues are then
the premise of the lab itself comes into question.

On Thu, May 7, 2009 at 10:32 PM, Robert Rohde <rarohde at gmail.com> wrote:

> On Thu, May 7, 2009 at 8:58 PM, Brian <Brian.Mingus at colorado.edu> wrote:
> > Quite frankly the advice that you should only use five subjects makes no
> > sense. The appeal to Nielsen's authority is not going to work on me or
> > anyone else who understands why the scientific method exists. It's
> > unscientific thinking and it's going cause to you waste money. You're
> going
> > to draw conclusions based on results that simply aren't valid, and you
> won't
> > know it until the study is over and you didn't make progress.
> >
> > Careful analysis of site data could allow you to draw some conclusions.
> I'm
> > curious how you're planning to go about that. Dependent/independent
> > variables?
>
> If five subjects, chosen at random, all have the same problem, then
> with 95% confidence you can predict that at least half of the
> population will report having this problem.
>
> This kind of work generally focuses on BIG problems, and you don't
> need a huge sample to identify some of the most common issues.  In
> things like UI development it would be surprising if there weren't
> complaints reported by most of the subjects.  You may overlook some
> other problems, but when coming up with a list of common problems to
> work on, I would say that 15 subjects is plenty.
>
> -Robert Rohde
>
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>