[Foundation-l] Usability Study Results (Sneak Preview)

[Brian]

Quite frankly the advice that you should only use five subjects makes no
sense. The appeal to Nielsen's authority is not going to work on me or
anyone else who understands why the scientific method exists. It's
unscientific thinking and it's going cause to you waste money. You're going
to draw conclusions based on results that simply aren't valid, and you won't
know it until the study is over and you didn't make progress.

Careful analysis of site data could allow you to draw some conclusions. I'm
curious how you're planning to go about that. Dependent/independent
variables?

On Thu, May 7, 2009 at 7:54 PM, Erik Moeller <erik at wikimedia.org> wrote:

> 2009/5/7 Brian <Brian.Mingus at colorado.edu>:
> > This all goes back to how you aim to quantify improvement in usability.
> > These samples sizes are so small that it will be hard (or even
> impossible)
> > to evaluate your progress based on statistical significance. You've got
> to
> > prove to us that its really getting better, and doesn't just look
> prettier.
>
> That's why, in addition to follow-up tests, the usability team is
> working closely with Erik Zachte to make sure we've got good editing
> metrics that show whether our changes are actually making a
> substantial difference in engaging new contributors.
> --
> Erik Möller
> Deputy Director, Wikimedia Foundation
>
> Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
>
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>