Quite frankly the advice that you should only use five subjects makes no
sense. The appeal to Nielsen's authority is not going to work on me or
anyone else who understands why the scientific method exists. It's
unscientific thinking and it's going cause to you waste money. You're going
to draw conclusions based on results that simply aren't valid, and you won't
know it until the study is over and you didn't make progress.
Careful analysis of site data could allow you to draw some conclusions. I'm
curious how you're planning to go about that. Dependent/independent
variables?
On Thu, May 7, 2009 at 7:54 PM, Erik Moeller <erik(a)wikimedia.org> wrote:
2009/5/7 Brian <Brian.Mingus(a)colorado.edu>du>:
This all goes back to how you aim to quantify
improvement in usability.
These samples sizes are so small that it will be hard (or even
impossible)
to evaluate your progress based on statistical
significance. You've got
to
prove to us that its really getting better, and
doesn't just look
prettier.
That's why, in addition to follow-up tests, the usability team is
working closely with Erik Zachte to make sure we've got good editing
metrics that show whether our changes are actually making a
substantial difference in engaging new contributors.
--
Erik Möller
Deputy Director, Wikimedia Foundation
Support Free Knowledge:
http://wikimediafoundation.org/wiki/Donate
_______________________________________________
foundation-l mailing list
foundation-l(a)lists.wikimedia.org
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/foundation-l