[Foundation-l] Usability Study Results (Sneak Preview)

[Brian]

I will have no part in your efforts to redefine the scientific method on its
talk page.

On Thu, May 7, 2009 at 10:35 PM, Sage Ross
<ragesoss+wikipedia at gmail.com<ragesoss%2Bwikipedia at gmail.com>
> wrote:

> On Thu, May 7, 2009 at 11:58 PM, Brian <Brian.Mingus at colorado.edu> wrote:
> > Quite frankly the advice that you should only use five subjects makes no
> > sense. The appeal to Nielsen's authority is not going to work on me or
> > anyone else who understands why the scientific method exists. It's
> > unscientific thinking and it's going cause to you waste money. You're
> going
> > to draw conclusions based on results that simply aren't valid, and you
> won't
> > know it until the study is over and you didn't make progress.
> >
> > Careful analysis of site data could allow you to draw some conclusions.
> I'm
> > curious how you're planning to go about that. Dependent/independent
> > variables?
> >
>
> An exercise in statistical thinking: when everyone or almost everyone
> cites problem X, how many people does it take to reach statistical
> significance that X is a problem worth addressing?  Even if the
> results are a statistical fluke and in reality only 20% of new users
> run into trouble with problem X, that's still a problem worth
> addressing.
>
> The fact that so many of the 15 people had the same problems, and
> those problems also align with common sense, is a strong indication
> that the study has found some things worth fixing.
>
> There is more than one way to come to reliable conclusions.  Any time
> I see someone invoking "the [singular] scientific method", as if there
> is only one and it is set in stone and universally agreed upon by all
> rational people, I have trouble taking them seriously.  See
> [[Talk:Scientific method]].
>
> -Sage (User:Ragesoss)
>
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>