+1
Cheap hallway testing is so incredibly useful that I dedicated my time
in Berlin last year to giving a crash course in it. I am not sure it was
effective in inspiring or educating people on how to do this, but
everyone is welcome to revisit the slides here:
http://wikitech.wikimedia.org/index.php?title=File:Trevor_Parscal_-_Wikimed…
Yesterday we had our first "on our own" series of user tests, conducted
by Parul Vora. While she is train in the kung-fu of user testing, she
personally helped me put this set of slides together. I also pulled from
my own experiences being involved in this kind of testing earlier in my
career.
My general pitch is, we should all be in the habit of doing whatever it
takes to view users as they interact with our creations. I often use my
wife, and now sometimes my 3-year old daughter to help me. Usually just
showing someone a picture of a screen and asking "how would you do X?"
is amazingly revealing. Higher-fidelity testing is great, but it's
designed to squeeze the last bit of juice out of the lemon. In my
experience the majority of it comes out quite easily in even the most
causal of circumstances.
My secondary pitch, which is not captured in these slides but was
verbalized in Berlin was my view that we should user-test APIs with
developers. This would especially be useful for our public HTTP API, but
even PHP and JavaScript APIs could benefit from this. This differs from
posting to the list and saying "does anyone have any better ideas".
Instead we would design APIs around use-cases, and then observe users in
those use-cases succeeding or failing.
Bottom line - I know from experience that if we can even subtly
introduce user testing as a factor in our decision making process, the
impact will be amazing.
- Trevor
On 12/2/10 6:46 AM, Neil Kandalgaonkar wrote:
Hi there -- I don't post much here, but I was the
programmer on the
Multimedia Usability Project, which primarily focused on making uploads
easier. The outside funding for that project just ended, so I think it's
a good time to talk about what (if anything) we will do in the future
along these lines.
Going forward, we ought not to think about usability as the
responsibility a few people in San Francisco. I have been asking myself
how we could end the need for usability projects, and instead make that
part of everyone's practices.
What makes you a usability engineer? My personal belief is that it isn't
(primarily) a matter of having special knowledge.
You become a usability software engineer when you see five average users
utterly fail to accomplish the task you wanted them to be able to
accomplish.
Programming is a hubristic enterprise, but for UI, these negative
feelings are essential: watching ordinary users get angry and frustrated
dealing with what you've created, even feeling a certain shame and
embarassment that you got it so wrong. Only then do you see how large
the conceptual gap is between you and the average user -- but you also
usually come out of the experience with an immediate understanding of
how to fix things.
So is there a way to have *everybody* who develops software for end
users in our community have that experience? Maybe.
At the WMF, for these Usability Projects, we had to do formal studies
with expert consultants, because these were grant-funded projects and we
needed to present scientific data. Doing those studies is expensive and
time-consuming.
But as a developer, it was more valuable to do "hallway usability
testing" in an informal way. There are lots of startups these days that
try to deliver such lightweight user testing over the web; could we do
the same? Or, possibly we don't even need software; maybe what we need
is a tradition of doing this for everything we release.
So how about that? If there were an easy way to integrate usability
testing into your process as a developer, would you be interested? And
what should that look like?