Hi everyone,
So as you know I did some user testing a few weeks ago. Here's the messages I took home from it, and how I think we can address those. Most of these have been covered in meetings and in our planning for the next sprint, but I think it's still worth making explicit.
*1) The preview step, as it stands, is not useful. *A lot of new users don't understand that they're being presented with a preview and think instead that their edit is already saved. Meanwhile, experienced editors are annoyed at not having the ability to choose when they preview like they do on desktop. For our MVP we're moving towards taking preview out as a discrete step, and also using text on buttons to make it clearer what actions are happening.
*2) Searching and starting the edit workflow is really clear to people.* People know exactly how to search, what was happening, and that hitting the pencil let them edit the article. That's great.
*3) People don't notice the edit summary box.* This is probably a sub-problem of finding the preview step confusing. When I pointed it out to people, they happily started using it, so it's not that they can't be bothered using the feature, it's just that they're not discovering it. We're going to look into ways of making the edit summary box more noticeable in combination with the work to resolve point 1.
*4) When people are pointed to the canned edit summaries dialogue on iOS, they use it fluidly.* It was a big hit with people and they understood it instantly. This was such a success that I think we should look at getting it onto Android further down the line.
Like I said, we're working on most of this stuff already. This is mostly just for posterity.
Thanks!
Dan
Is this for apps or mobile web or both? On 2 Jun 2014 10:42, "Dan Garry" dgarry@wikimedia.org wrote:
Hi everyone,
So as you know I did some user testing a few weeks ago. Here's the messages I took home from it, and how I think we can address those. Most of these have been covered in meetings and in our planning for the next sprint, but I think it's still worth making explicit.
*1) The preview step, as it stands, is not useful. *A lot of new users don't understand that they're being presented with a preview and think instead that their edit is already saved. Meanwhile, experienced editors are annoyed at not having the ability to choose when they preview like they do on desktop. For our MVP we're moving towards taking preview out as a discrete step, and also using text on buttons to make it clearer what actions are happening.
*2) Searching and starting the edit workflow is really clear to people.* People know exactly how to search, what was happening, and that hitting the pencil let them edit the article. That's great.
*3) People don't notice the edit summary box.* This is probably a sub-problem of finding the preview step confusing. When I pointed it out to people, they happily started using it, so it's not that they can't be bothered using the feature, it's just that they're not discovering it. We're going to look into ways of making the edit summary box more noticeable in combination with the work to resolve point 1.
*4) When people are pointed to the canned edit summaries dialogue on iOS, they use it fluidly.* It was a big hit with people and they understood it instantly. This was such a success that I think we should look at getting it onto Android further down the line.
Like I said, we're working on most of this stuff already. This is mostly just for posterity.
Thanks!
Dan
-- Dan Garry Associate Product Manager for Platform and Mobile Apps Wikimedia Foundation
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Oops, sorry! This was just for apps.
Dan
On 2 June 2014 10:49, Jon Robson jrobson@wikimedia.org wrote:
Is this for apps or mobile web or both? On 2 Jun 2014 10:42, "Dan Garry" dgarry@wikimedia.org wrote:
Hi everyone,
So as you know I did some user testing a few weeks ago. Here's the messages I took home from it, and how I think we can address those. Most of these have been covered in meetings and in our planning for the next sprint, but I think it's still worth making explicit.
*1) The preview step, as it stands, is not useful. *A lot of new users don't understand that they're being presented with a preview and think instead that their edit is already saved. Meanwhile, experienced editors are annoyed at not having the ability to choose when they preview like they do on desktop. For our MVP we're moving towards taking preview out as a discrete step, and also using text on buttons to make it clearer what actions are happening.
*2) Searching and starting the edit workflow is really clear to people.* People know exactly how to search, what was happening, and that hitting the pencil let them edit the article. That's great.
*3) People don't notice the edit summary box.* This is probably a sub-problem of finding the preview step confusing. When I pointed it out to people, they happily started using it, so it's not that they can't be bothered using the feature, it's just that they're not discovering it. We're going to look into ways of making the edit summary box more noticeable in combination with the work to resolve point 1.
*4) When people are pointed to the canned edit summaries dialogue on iOS, they use it fluidly.* It was a big hit with people and they understood it instantly. This was such a success that I think we should look at getting it onto Android further down the line.
Like I said, we're working on most of this stuff already. This is mostly just for posterity.
Thanks!
Dan
-- Dan Garry Associate Product Manager for Platform and Mobile Apps Wikimedia Foundation
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On Mon, Jun 2, 2014 at 10:53 AM, Dan Garry dgarry@wikimedia.org wrote:
Oops, sorry! This was just for apps.
However, the lessons about preview and summary discoverability are directly applicable to web too.
On 2 Jun 2014 10:54, "Max Semenik" maxsem.wiki@gmail.com wrote:
On Mon, Jun 2, 2014 at 10:53 AM, Dan Garry dgarry@wikimedia.org wrote:
Oops, sorry! This was just for apps.
However, the lessons about preview and summary discoverability are
directly applicable to web too.
Maybe. Were similar tests run on web?
-- Best regards, Max Semenik ([[User:MaxSem]])
Some bits (e.g. "canned edit summaries were helpful") probably apply more than others (e.g. "preview isn't helpful", which isn't a fair test since preview works quite differently between web and apps).
I only do apps, so I only focussed on that. :-)
Dan
On 2 June 2014 11:54, Jon Robson jrobson@wikimedia.org wrote:
On 2 Jun 2014 10:54, "Max Semenik" maxsem.wiki@gmail.com wrote:
On Mon, Jun 2, 2014 at 10:53 AM, Dan Garry dgarry@wikimedia.org wrote:
Oops, sorry! This was just for apps.
However, the lessons about preview and summary discoverability are
directly applicable to web too.
Maybe. Were similar tests run on web?
-- Best regards, Max Semenik ([[User:MaxSem]])
On Mon, Jun 2, 2014 at 6:54 PM, Jon Robson jrobson@wikimedia.org wrote:
On 2 Jun 2014 10:54, "Max Semenik" maxsem.wiki@gmail.com wrote:
On Mon, Jun 2, 2014 at 10:53 AM, Dan Garry dgarry@wikimedia.org wrote:
Oops, sorry! This was just for apps.
However, the lessons about preview and summary discoverability are
directly applicable to web too.
Maybe. Were similar tests run on web?
Moiz and Maryana have been doing this: https://trello.com/c/Jmp3bdKn/15-ve-tablet-user-testing
Not sure if they've wrapped up testing yet.
*> 4) When people are pointed to the canned edit summaries dialogue on iOS, they use it fluidly.*
It was a big hit with people and they understood it instantly.
I'm really glad to hear a confirmation of this.
This was such a success that I think we should look at getting it onto
Android further down the line.
And on mobile web. And on desktop.
I am basically repeating what I said in March: http://lists.wikimedia.org/pipermail/mobile-l/2014-March/006605.html
-- Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי http://aharoni.wordpress.com “We're living in pieces, I want to live in peace.” – T. Moore
On Mon, Jun 2, 2014 at 10:42 AM, Dan Garry dgarry@wikimedia.org wrote:
*1) The preview step, as it stands, is not useful. *A lot of new users don't understand that they're being presented with a preview and think instead that their edit is already saved. Meanwhile, experienced editors are annoyed at not having the ability to choose when they preview like they do on desktop. For our MVP we're moving towards taking preview out as a discrete step, and also using text on buttons to make it clearer what actions are happening.
*2) Searching and starting the edit workflow is really clear to people.* People know exactly how to search, what was happening, and that hitting the pencil let them edit the article. That's great.
*3) People don't notice the edit summary box.* This is probably a sub-problem of finding the preview step confusing. When I pointed it out to people, they happily started using it, so it's not that they can't be bothered using the feature, it's just that they're not discovering it. We're going to look into ways of making the edit summary box more noticeable in combination with the work to resolve point 1.
Great to hear about these. I definitely agree about the Preview step.
*4) When people are pointed to the canned edit summaries dialogue on iOS, they use it fluidly.* It was a big hit with people and they understood it instantly. This was such a success that I think we should look at getting it onto Android further down the line.
Testing this out, and with my Wikipedia admin/power user hat on, my big concern at the moment is that users can choose as many canned edit summaries as they want. If users end up choosing more than one canned summary they get less useful and more cluttered to read in RecentChanges/watchlists etc.
On 2 June 2014 13:50, Steven Walling swalling@wikimedia.org wrote:
Testing this out, and with my Wikipedia admin/power user hat on, my big concern at the moment is that users can choose as many canned edit summaries as they want. If users end up choosing more than one canned summary they get less useful and more cluttered to read in RecentChanges/watchlists etc.
Agreed. We want to get it down to three or so, for quick little tasks like "fixed typo" or "added link". Then the problem of selecting them all is minimised.
Dan
Always that I hear about canned summaries I wonder if we could make them context-sensitive to make options more likely to be relevant to what the user just edited. That is, provide the "fixed typo" option when the edit consists on changing 5 characters or less, provide "Added link" when the edit consists on adding a link, etc.
On Mon, Jun 2, 2014 at 1:55 PM, Dan Garry dgarry@wikimedia.org wrote:
On 2 June 2014 13:50, Steven Walling swalling@wikimedia.org wrote:
Testing this out, and with my Wikipedia admin/power user hat on, my big concern at the moment is that users can choose as many canned edit summaries as they want. If users end up choosing more than one canned summary they get less useful and more cluttered to read in RecentChanges/watchlists etc.
Agreed. We want to get it down to three or so, for quick little tasks like "fixed typo" or "added link". Then the problem of selecting them all is minimised.
Dan
-- Dan Garry Associate Product Manager for Platform and Mobile Apps Wikimedia Foundation
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l