Great research, thanks for sharing!

I'm looking forward to further diving into more subjective nuance, like the usefulness of each model for different reading use cases (quick fact checking vs exploratory learning for example).

At some point I saw POC mocks of a mix between expanded and collapsed, where the section was collapsed, but it showed a small summary of the section below the title, like a teaser. It would be very interesting to test that kind of design too and see how it fares with the other two.

On Tue, Jun 21, 2016 at 10:21 AM, Tilman Bayer <tbayer@wikimedia.org> wrote:

Hi,

as most on this list will be aware, on the mobile web version of
Wikipedia, all top-level sections below the lead section are currently
shown collapsed on initial view. Users can tap on a section heading to
show the content, and to collapse it again.
To examine the tradeoffs of this solution and inform future product
decisions, we ran an experiment where 0.05% of mobile web users were
shown all pages with every section expanded on initial load,
instrumented alongside a control group of 0.05% that kept seeing the
standard view where all sections all initially collapsed.

A high-level summary of results is now available at
https://meta.wikimedia.org/wiki/Research:Collapsed_vs_uncollapsed_section_view_on_mobile_web
. In particular:

* Readers in the test group (sections expanded) tend to stay longer on the page
* Readers in the test group tend to spend more time reading, and less
time navigating
* Readers in the test group tend to scroll more sections into view
than readers in the control group open
* Readers in the test group tend to stay shorter on the page than
readers using the Android Wikipedia app (which offers a TOC for easier
navigation, something not yet available in the mobile web test group)

Comments and questions are welcome, feel free to use the talk page for them too.

Note that this experiment only measured some aspects, and that the
results don't yet allow the unambiguous conclusion that it would be
better to switch to the uncollapsed view. That said, they certainly
suggest that such a change should be considered. It is being planned
to examine this question further with some user testing sessions.

(As an experiment, I've taken the opportunity to write this up this
analysis as a page in the research namespace on Meta, instead of on
Phabricator or in form of an email as done on other occasions.
Feedback on the format is welcome too.)
--
Tilman Bayer
Senior Analyst
Wikimedia Foundation
IRC (Freenode): HaeB