The on-wiki version of this newsletter is here:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2022-06-10
--
Design researcher Jeff Howard did another round of research
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-09-24> in
order to prioritize issues in the run up to launch, and beyond. The full
report of the user research has been published
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Design/Wikifunctions_usability_tests_2022>
on
Meta. Aishwarya, the designer on the Abstract Wikipedia team, has read and
analysed the research results, and summarized them in a slide deck
<https://commons.wikimedia.org/wiki/File:Wikifunctions_usability_tests_2022_-_Function_Editor_and_Function_Page_Improvements.pdf>.
This week, Aishwarya presented the deck to the team, and we are offering
here a short summary of the presentation.
The goal for designing the function page is two-fold: to be understandable
and usable to technical people of all backgrounds, and welcoming to people
with low levels of programming expertise. Technical contributors should
understand the function creation workflow and the Wikifunctions mental
model. Seven technical participants were interviewed using Aishwarya’s
designs in Figma
<https://www.figma.com/proto/05qjdoiV05MtZD2vEfbDDe/User-testing-function-flow?page-id=0%3A1&node-id=2%3A966&viewport=328%2C48%2C0.09&scaling=min-zoom&starting-point-node-id=2%3A966>
(click
anywhere on the screen to progress through the slides and remember to
expand your window).
The interviewees raised many great questions, validated a lot of our design
work, and identified several areas for improvement. Overall, the report
validated that we have met the stated design goals of the user interface
being understandable and usable for technical people, but the report also
highlighted that the contributors did not really understand the function
creation workflow and the general Wikifunctions mental model. In short,
they could get everything done, but were often confused about what they
were doing and why it was presented in that way.
I will not go into the many things that worked out well. You can read about
them in the full report and also in the slides. I do want to call out the
praise for the work summary diagram, which is consistent with many other
reactions we also got in the chat and in other interactions with Wikimedia
community members. I also want to use the chance to congratulate Aishwarya
on her design work, and seeing it validated so positively. We are all very
much looking forward to getting the implemented design out there for you to
play with, and learning more about how we can improve it.
Two points were called out by the interviewees in particular as causes of
surprise or confusion: the split between function definitions and their
implementations, and the multi-lingual nature of Wikifunctions.
In Wikifunctions, we allow each function to have several implementations
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Updates/2021-06-17>. We
achieve this by having implementations be their own pages on Wikifunctions.
Such a separation is not a novel concept: programming languages such as C++
or Ada had header and implementation files for decades, and object oriented
languages <https://en.wikipedia.org/wiki/Object-oriented_programming> have
interfaces that can be implemented by different classes. But interviewees
have repeatedly wanted to jump right into providing the implementation.
They were confused that they could publish a function's definition even
before having provided an implementation. This was also a request we have
seen in previous user tests.
As a side remark, the little word *“publish”* really did a lot of heavy
lifting here. A long time ago, Wikipedia used to use the word *“save”* for
the button that let the contributor store an edit, and this was changed to
*“publish”* in 2017, based on user research that found wiki users surprised
and alarmed that merely 'saving' an edit would put it online, in public,
for everyone to see, forever. This user study reiterated the point that the
word *“publish”* makes it clear that the contribution will indeed go live
to the whole world. But at the same time, several interviewees felt that
just a function definition, without any implementations yet, didn’t seem to
be useful to be published. The word *“publish”* really brought out that
contrast, and helped us identify this discrepancy in the user’s mental
model.
The second point that raised quite strong reactions was the multi-lingual
nature of Wikifunctions. That is one of the points that is often questioned
in the design of Wikifunctions, often unprompted: why does it have to be
multi-lingual? Why labels in different languages? Doesn’t everyone who
wants to code just learn English? To quote one of the interviewees, *“usually
people who speak other languages are just expected to learn English to
code”*.
Because the world of coding is indeed so English-centered, it is very
difficult to find people with coding experience who don’t speak basic
English, and indeed all interviewed contributors spoke English.
There have been a number of research studies showing that the
English-centricity of programming is a major barrier
<https://dl.acm.org/doi/abs/10.1145/3173574.3173970> for many people.
People who can use their own language to code achieve results faster
<https://dl.acm.org/doi/abs/10.1145/3051457.3051464>. For parents that
don’t speak English, it is more difficult to help their children
<https://dl.acm.org/doi/abs/10.1145/3173574.3174196> to learn programming.
Based on these and other research results, we choose to intentionally
deviate from the recommendations of our own user research, as we believe
that this aligns better with the Wikimedia 2030 movement strategy
recommendations
<https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recommendations>,
particularly towards knowledge equity.
There were many smaller, but very good points raised. The contributors
asked for a space to describe the functions in more detail (that’s planned
for Phase ι
<https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases#Phase_%CE%B9_(iota):_documentation_of_objects>,
which is up next in our development plan). The term *“aliases”* confused
users. The list of types was too simple. The example table was identified
as a place that probably won’t scale for complex entries. The difference
between the words *“available”* and *“proposed”* and *“verified”* in the
tables showing implementations and testers was confusing. And there were
quite a few more.
We also identified a number of larger areas that could be improved: making
the use of language more consistent throughout, displaying more meta-data
immediately, and improving the text to make the distinction between
definitions and implementations clearer. We are going to work on these
design challenges.
We are relieved and pleased to see that the designs allowed all the
contributors to fulfill their tasks. We are more than excited to implement
these designs, and get them to you. We would love to hear from you, if you
have ideas or suggestions around the issues discussed here, or in the full
report.
Thanks to all the contributors who were interviewed, thanks to Jeff for
performing the research, and thanks to Aishwarya for summarizing the
results.
Updates as of June 3: Fix-it week
- May 30 – June 3 was a ‘Fix-it’ week for the Abstract Wikipedia team.
During this week, the team paused the development of new features and
focused on tasks related to technical debt.
- Design update: This week, the team kicked off the design work for the
‘lists’ component.