+1
Cheap hallway testing is so incredibly useful that I dedicated my time in Berlin last year to giving a crash course in it. I am not sure it was effective in inspiring or educating people on how to do this, but everyone is welcome to revisit the slides here:
http://wikitech.wikimedia.org/index.php?title=File:Trevor_Parscal_-_Wikimedi...
Yesterday we had our first "on our own" series of user tests, conducted by Parul Vora. While she is train in the kung-fu of user testing, she personally helped me put this set of slides together. I also pulled from my own experiences being involved in this kind of testing earlier in my career.
My general pitch is, we should all be in the habit of doing whatever it takes to view users as they interact with our creations. I often use my wife, and now sometimes my 3-year old daughter to help me. Usually just showing someone a picture of a screen and asking "how would you do X?" is amazingly revealing. Higher-fidelity testing is great, but it's designed to squeeze the last bit of juice out of the lemon. In my experience the majority of it comes out quite easily in even the most causal of circumstances.
My secondary pitch, which is not captured in these slides but was verbalized in Berlin was my view that we should user-test APIs with developers. This would especially be useful for our public HTTP API, but even PHP and JavaScript APIs could benefit from this. This differs from posting to the list and saying "does anyone have any better ideas". Instead we would design APIs around use-cases, and then observe users in those use-cases succeeding or failing.
Bottom line - I know from experience that if we can even subtly introduce user testing as a factor in our decision making process, the impact will be amazing.
- Trevor
On 12/2/10 6:46 AM, Neil Kandalgaonkar wrote:
Hi there -- I don't post much here, but I was the programmer on the Multimedia Usability Project, which primarily focused on making uploads easier. The outside funding for that project just ended, so I think it's a good time to talk about what (if anything) we will do in the future along these lines.
Going forward, we ought not to think about usability as the responsibility a few people in San Francisco. I have been asking myself how we could end the need for usability projects, and instead make that part of everyone's practices.
What makes you a usability engineer? My personal belief is that it isn't (primarily) a matter of having special knowledge.
You become a usability software engineer when you see five average users utterly fail to accomplish the task you wanted them to be able to accomplish.
Programming is a hubristic enterprise, but for UI, these negative feelings are essential: watching ordinary users get angry and frustrated dealing with what you've created, even feeling a certain shame and embarassment that you got it so wrong. Only then do you see how large the conceptual gap is between you and the average user -- but you also usually come out of the experience with an immediate understanding of how to fix things.
So is there a way to have *everybody* who develops software for end users in our community have that experience? Maybe.
At the WMF, for these Usability Projects, we had to do formal studies with expert consultants, because these were grant-funded projects and we needed to present scientific data. Doing those studies is expensive and time-consuming.
But as a developer, it was more valuable to do "hallway usability testing" in an informal way. There are lots of startups these days that try to deliver such lightweight user testing over the web; could we do the same? Or, possibly we don't even need software; maybe what we need is a tradition of doing this for everything we release.
So how about that? If there were an easy way to integrate usability testing into your process as a developer, would you be interested? And what should that look like?