[Apps] Description editing prototype testing - Mobile-l

26 Mar 2015

Hi all,
Please find the goals for and top-level findings from guerrilla testing
description editing on Wikipedia Alpha below. More in-depth information
including tasks/questions posed and raw notes on the 5 participants can be
found here
<https://www.mediawiki.org/wiki/Guerrilla_Testing,_March_24,_2015:_Description_editing>.
There may be typos and small formatting errors (I'll be proofing the page
in the next hour).
Happy quarterly planning--Daisy
---
Goal

The goal of this research was to observe people interact with the CTA ("Tap
to add a description!") line in the header under article titles in
Wikipedia Alpha. After tapping the CTA, users experience 3 editable
scenarios: 1. an meaningful description suggestion, 2. a blank form field,
and 3. a random/irrelevant description suggestion.

   1. Do users notice the CTA prompt? How effective are they in triggering
   action?
   2. How do users feel about the CTA?
   3. How effective are descriptions that are auto-generated and
   meaningful? Do they assist users with finalizing the description or confuse
   users as to why they are prompted to edit a description that is already
   automated and correct?
   4. How effective is not giving a user a pre-populated description field?
   5. How effective are descriptions that are auto-generated and random?
   6. How do users feel about the process of editing description lines
   overall?

Findings: Patterns Observed

   1. 3 of 5 users required some level of facilitator prompting to notice
   the CTA.
   2. Interactivity breakdown:
      1. 2 users would most likely overlook this field, 2 users might
      notice/interact depending on the situation, and 1 user was not sure.
   3. All users either specifically indicated field interaction was easy
   and intuitive or had no specific complaint or struggle that was observed.
   Only 1 user was briefly confused about the blank SF description field,
   thinking he couldn't type because he didn't see a blinking cursor.
   4. 3 scenarios feedback breakdown
      1. Meaningful description suggestion
         1. most helpful: 2 users
         2. most helpful, but pointless because I can't see it on page: 1
         user
         3. confusing: 1 user
      2. Blank
         1. fine if you know about topic: 1 user
         2. fine and having the CTA here made most sense: 1 user
         3. most engaging: 1 user
         4. easiest: 1 user
      3. Random/irrelevant description suggestion
         1. if visible, could prompt action: 2 users
      5. 2 of 5 users expressed some level of confusion around why the CTA
   hides the description. One user was confused about why he was prompted to
   action when the description was correct on Picasso. Another user was
   confused about the same thing, and also mentioned that he would be much
   more likely to take action on the random article if he could see that the
   description was incorrect. The latter also mentioned that the CTA really
   only makes sense on the SF blank description page.
   6. 1 user was confused about whether these descriptions were for himself
   or for all of Wikipedia.
   7. No users indicated confusion about the CTA pop-up language.

-- 
Daisy Chen
Wikimedia Foundation

Imagine a world in which every single human being can freely share in
the sum of all knowledge. Help us make it a reality!

*https://donate.wikimedia.org <https://donate.wikimedia.org/>*