[Foundation-l] Usability study in progress

[Erik Moeller]

2009/3/25 John at Darkstar <vacuum at jeb.no>:
> Wikipedians should not be used to asses usabillity problems with
> Wikipedia, this is rule number one if you want to get information about
> why a newbie has problems with a system.

I'm not sure about your definition of "Wikipedians" above, but the
recruiting procedure uses a screening process to recruit _readers_ of
Wikipedia with no editing experience. Our goal here is to at the end
of the day make improvements to convert more readers to editors.

> Ten participants are not nearly enough, they can
> only give you some clue about the real problem.

There are different philosophies of usability, including a philosophy
of agile testing with few test subjects (
http://www.useit.com/alertbox/20000319.html ). There's general
agreement that with these kinds of tests, you'll quickly see
diminishing returns - adding many more people doesn't actually help
you discover many more problems. Moreover, resources are not infinite:
finding a good balance in terms of the number of testers allows you to
conduct more tests later, with the goal of validating whether the
changes you've made actually have had the intended effect.

Erik
-- 
Erik Möller
Deputy Director, Wikimedia Foundation

Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate