[Foundation-l] Usability Study Results (Sneak Preview)

[Brian]

Sounds easy. I wonder why this "study" doesn't mention a p value. The grant
must not have been large enough to fund someone with any experience using R,
or god forbid, a pencil.

On Thu, May 7, 2009 at 10:32 PM, Robert Rohde <rarohde at gmail.com> wrote:

> On Thu, May 7, 2009 at 8:58 PM, Brian <Brian.Mingus at colorado.edu> wrote:
> > Quite frankly the advice that you should only use five subjects makes no
> > sense. The appeal to Nielsen's authority is not going to work on me or
> > anyone else who understands why the scientific method exists. It's
> > unscientific thinking and it's going cause to you waste money. You're
> going
> > to draw conclusions based on results that simply aren't valid, and you
> won't
> > know it until the study is over and you didn't make progress.
> >
> > Careful analysis of site data could allow you to draw some conclusions.
> I'm
> > curious how you're planning to go about that. Dependent/independent
> > variables?
>
> If five subjects, chosen at random, all have the same problem, then
> with 95% confidence you can predict that at least half of the
> population will report having this problem.
>
> This kind of work generally focuses on BIG problems, and you don't
> need a huge sample to identify some of the most common issues.  In
> things like UI development it would be surprising if there weren't
> complaints reported by most of the subjects.  You may overlook some
> other problems, but when coming up with a list of common problems to
> work on, I would say that 15 subjects is plenty.
>
> -Robert Rohde
>
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>