[Foundation-l] Usability Study Results (Sneak Preview)

[Brian]

I've been told in private messages that I'm not allowed to ask these
questions and that my style is excessively rude.
Pardon me for continuing, I think they are important issues. If I come
across rude to you just imagine me in person where I am much more likable in
conversation. Otherwise, get over it!

On Fri, May 8, 2009 at 8:28 AM, Brian <Brian.Mingus at colorado.edu> wrote:

> Ok, I'll agree that the motiviations and size of this pilot study are
> reasonable. Then I'd just like to know how much money was spent getting
> these answers.  If you're not planning to measure the subjects
> scientifically and you just want to figure out what the big issues are then
> the premise of the lab itself comes into question.
>
> On Thu, May 7, 2009 at 10:32 PM, Robert Rohde <rarohde at gmail.com> wrote:
>
>> On Thu, May 7, 2009 at 8:58 PM, Brian <Brian.Mingus at colorado.edu> wrote:
>> > Quite frankly the advice that you should only use five subjects makes no
>> > sense. The appeal to Nielsen's authority is not going to work on me or
>> > anyone else who understands why the scientific method exists. It's
>> > unscientific thinking and it's going cause to you waste money. You're
>> going
>> > to draw conclusions based on results that simply aren't valid, and you
>> won't
>> > know it until the study is over and you didn't make progress.
>> >
>> > Careful analysis of site data could allow you to draw some conclusions.
>> I'm
>> > curious how you're planning to go about that. Dependent/independent
>> > variables?
>>
>> If five subjects, chosen at random, all have the same problem, then
>> with 95% confidence you can predict that at least half of the
>> population will report having this problem.
>>
>> This kind of work generally focuses on BIG problems, and you don't
>> need a huge sample to identify some of the most common issues.  In
>> things like UI development it would be surprising if there weren't
>> complaints reported by most of the subjects.  You may overlook some
>> other problems, but when coming up with a list of common problems to
>> work on, I would say that 15 subjects is plenty.
>>
>> -Robert Rohde
>>
>> _______________________________________________
>> foundation-l mailing list
>> foundation-l at lists.wikimedia.org
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>>
>
>