I have published the results of two additional tests on Winter. These were competing
tests and I expected one to fail fairly strongly (and was not disappointed).
The first test was identical to the "Electric Boogaloo" test. However, the
Winter interface had been modified thus:
* No page-context actions are loaded into the fixed header.
* The user tools section does not shrink/collapse (thus always showing the username).
This test was a fairly significant success. All testers completed all tasks.
Most users quickly pointed out that they knew they were "logged in" because of
their username appearing in the corner and noted that they have a set of personal or
profile tools there. More than one tester referred to this area as "their
profile". No users had difficulty finding their contributions because of this.
With one exception, all users found the context action ribbon quickly. The one who did not
later was observed using it with proficiency.
Many users unconsciously navigated with the context action ribbon afterwards.
The search box remains mostly discoverable. Several testers unconsciously used it in the
sticky format without scrolling to the top.
No one used the table of contents in the header.
The second test was the polar opposite. For this test, the in page context actions (the
ones below the article title) were removed. The page actions in the header were visible
at all times, and the user tools menu did not collapse.
Most users encountered signficant icon confusion. In some cases, the confusion was enough
to prevent them from accomplishing a task. In other cases, the confusion will obviously be
problematic as cental editing themes are confused.
The discoverability of the user tools section, when the username is present, is very high
and remains so.
The discoverability of the page action icons is almost non-existent. Several users
expressed a desire for action labels. Many users failed to accomplish tasks that required
these icons, or assumed they had achieved success when they did not. It is my belief that
at least two testers assumed that the page action icons were just parts of their personal
tools, and never looked there for page actions.
Over all tests, no one has ever used the in-header Table of Contents.
Test videos and more highlights available here:
https://www.mediawiki.org/wiki/Winter#Harness_Three:_Winter:_No_Header_Icons
https://www.mediawiki.org/wiki/Winter#Harness_Four:_Winter:_Only_Header_Act…
Barring additional test desires, I'm going to collate all findings across all tests.
My current recommendations are:
* The in-header actions are confusing and should be removed.
* The user tools menu should not collapse as its discoverability significantly degrades
when the user name is not present.
* The in-header table of contents has very low discoverability. No one has ever used it,
and more than one tester has expressed confusion about it when it was found. We should
remove it or explore alternatives.
* The search box remains highly discoverable, but it is my belief (backed by some tester
statements) that the icon is a large part of this. Changing the text in the box reduces
discoverability some but not enough that I would eliminate the behavior.
* A link to an article's talk page should be included in the in-page table of
contents. Several users seem to want to go there.
Please discuss here:
https://www.mediawiki.org/w/index.php?title=Talk:Winter&workflow=rr9fa8…
---
Brandon Harris, Senior Designer, Wikimedia Foundation
Support Free Knowledge:
http://wikimediafoundation.org/wiki/Donate