Great research, thanks for sharing!
I'm looking forward to further diving into more subjective nuance, like the usefulness of each model for different reading use cases (quick fact checking vs exploratory learning for example).
At some point I saw POC mocks of a mix between expanded and collapsed, where the section was collapsed, but it showed a small summary of the section below the title, like a teaser. It would be very interesting to test that kind of design too and see how it fares with the other two.
On Tue, Jun 21, 2016 at 10:21 AM, Tilman Bayer tbayer@wikimedia.org wrote:
Hi,
as most on this list will be aware, on the mobile web version of Wikipedia, all top-level sections below the lead section are currently shown collapsed on initial view. Users can tap on a section heading to show the content, and to collapse it again. To examine the tradeoffs of this solution and inform future product decisions, we ran an experiment where 0.05% of mobile web users were shown all pages with every section expanded on initial load, instrumented alongside a control group of 0.05% that kept seeing the standard view where all sections all initially collapsed.
A high-level summary of results is now available at
https://meta.wikimedia.org/wiki/Research:Collapsed_vs_uncollapsed_section_vi... . In particular:
- Readers in the test group (sections expanded) tend to stay longer on the
page
- Readers in the test group tend to spend more time reading, and less
time navigating
- Readers in the test group tend to scroll more sections into view
than readers in the control group open
- Readers in the test group tend to stay shorter on the page than
readers using the Android Wikipedia app (which offers a TOC for easier navigation, something not yet available in the mobile web test group)
Comments and questions are welcome, feel free to use the talk page for them too.
Note that this experiment only measured some aspects, and that the results don't yet allow the unambiguous conclusion that it would be better to switch to the uncollapsed view. That said, they certainly suggest that such a change should be considered. It is being planned to examine this question further with some user testing sessions.
(As an experiment, I've taken the opportunity to write this up this analysis as a page in the research namespace on Meta, instead of on Phabricator or in form of an email as done on other occasions. Feedback on the format is welcome too.) -- Tilman Bayer Senior Analyst Wikimedia Foundation IRC (Freenode): HaeB