I have run two sets of user tests with Winter to determine the usability of in-header page action icons and whether or not users can easily recognize their personal tools menu and the search box, especially after scrolling in the page. The results are not encouraging for the in-header icons.
Two tests were run because the first test had some possible errors and confusions. The second test (Winter Harness Two Electric Boogaloo) had some modifications to the flow to avoid people getting lost on user talk pages (this was the result of the first version of Harness Two telling them to click on the Speech Bubble icon in the top right if they were lost - but they had scrolled to the top already). H2:EB corrected this by directing them to the context action ribbon.
* https://www.mediawiki.org/wiki/Winter#Harness_Two:_Winter * https://www.mediawiki.org/wiki/Winter#Harness_Two:_Electric_Boogaloo
* Only *one* user out of ten correctly used the in-header page action icons. Further, his language indicated that he had some icon confusion: he thought that only a few icons were added, and that existing ones were moved (e.g., history and edit were new, but the watch list icon was still the same). * Most users were unable to recognize the personal tools section as being "my stuff" unless their username was ''also'' included. Once the username was gone, it was invisible to them (with rare exceptions, and mostly by accident then). * No users had trouble with the search box. At all. * Most users ended up using the context action ribbon and ignored other navigation hints. * The in-header TOC might as well be invisible. * At this point, with 10 testers and a 10% success rate, I'd say that the benefits of putting page icons in the header are outweighed by the negatives of losing the username for discoverability.
Discussion here: https://www.mediawiki.org/w/index.php?title=Talk:Winter&workflow=rr0ke21...
--- Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
I should also add some other interesting tid-bits, not necessarily associated with Winter:
* Everyone gets baffled by article talk page templates. * Few people actually recognize that there are “discussions” happening. Most indicate that they expect a different type of discussion. * A few people didn’t know that there were tables of contents. * One person said that he didn’t even know that we had accounts, and yet also indicated that he noticed we moved the search from the *left sidebar* to the top. That’s odd: you have to have an account to use Monobook.
I also wonder if we won’t be served by running a test where we do NOT goof with the personal tools section on scroll and do NOT add page action icons to the header, to see if that helps with discoverability of the user menu after scroll. That could be a very short test.
On Mar 15, 2014, at 1:12 PM, Brandon Harris bharris@wikimedia.org wrote:
I have run two sets of user tests with Winter to determine the usability of in-header page action icons and whether or not users can easily recognize their personal tools menu and the search box, especially after scrolling in the page. The results are not encouraging for the in-header icons.
--- Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
On 2014-03-15, 1:20 PM, Brandon Harris wrote:
* One person said that he didn’t even know that we had accounts, and yet also indicated that he noticed we moved the search from the *left sidebar* to the top. That’s odd: you have to have an account to use Monobook.
Or just use Wikipedia before WMF switched to Vector.
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
On Mar 15, 2014, at 3:39 PM, Daniel Friesen daniel@nadir-seen-fire.com wrote:
Or just use Wikipedia before WMF switched to Vector.
Except it would have to have been four years since he’d been to the site. We serve Vector to IP users.
--- Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
Brandon, I think running two more short tests, one where the article tools are only in the fixed header, and one where they are only in the article (fixed header remains personal tools) would be illuminating.
*Jared Zimmerman * \ Director of User Experience \ Wikimedia Foundation
M : +1 415 609 4043 | : @JaredZimmermanhttps://twitter.com/JaredZimmerman
On Sat, Mar 15, 2014 at 1:20 PM, Brandon Harris bharris@wikimedia.orgwrote:
I should also add some other interesting tid-bits, not necessarilyassociated with Winter:
* Everyone gets baffled by article talk page templates. * Few people actually recognize that there are"discussions" happening. Most indicate that they expect a different type of discussion. * A few people didn't know that there were tables of contents. * One person said that he didn't even know that we had accounts, and yet also indicated that he noticed we moved the search from the *left sidebar* to the top. That's odd: you have to have an account to use Monobook.
I also wonder if we won't be served by running a test where we doNOT goof with the personal tools section on scroll and do NOT add page action icons to the header, to see if that helps with discoverability of the user menu after scroll. That could be a very short test.
On Mar 15, 2014, at 1:12 PM, Brandon Harris bharris@wikimedia.org wrote:
I have run two sets of user tests with Winter to determine the usability
of in-header page action icons and whether or not users can easily recognize their personal tools menu and the search box, especially after scrolling in the page. The results are not encouraging for the in-header icons.
Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
Design mailing list Design@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/design
Should I run these off Harness 1 or Harness 2? I would suggest a modified version of 1, where we don’t test finding the edit button, but also clean up some of the edges we saw in test 2 (like having to move the usertesting control panel).
On Mar 15, 2014, at 4:45 PM, Jared Zimmerman jared.zimmerman@wikimedia.org wrote:
Brandon, I think running two more short tests, one where the article tools are only in the fixed header, and one where they are only in the article (fixed header remains personal tools) would be illuminating.
--- Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
Dear all, I have went to a vacation so I am sorry for this late reply, I love this new design, what may I help? Gabirle
2014-03-16 8:58 GMT+08:00 Brandon Harris bharris@wikimedia.org:
Should I run these off Harness 1 or Harness 2? I would suggest amodified version of 1, where we don't test finding the edit button, but also clean up some of the edges we saw in test 2 (like having to move the usertesting control panel).
On Mar 15, 2014, at 4:45 PM, Jared Zimmerman < jared.zimmerman@wikimedia.org> wrote:
Brandon, I think running two more short tests, one where the article
tools are only in the fixed header, and one where they are only in the article (fixed header remains personal tools) would be illuminating.
Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
Design mailing list Design@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/design