Hi all,

Please find the goals for and top-level findings from guerrilla testing description editing on Wikipedia Alpha below. More in-depth information including tasks/questions posed and raw notes on the 5 participants can be found here. There may be typos and small formatting errors (I'll be proofing the page in the next hour).

Happy quarterly planning--
Daisy

---

Goal

The goal of this research was to observe people interact with the CTA ("Tap to add a description!") line in the header under article titles in Wikipedia Alpha. After tapping the CTA, users experience 3 editable scenarios: 1. an meaningful description suggestion, 2. a blank form field, and 3. a random/irrelevant description suggestion.

  1. Do users notice the CTA prompt? How effective are they in triggering action?
  2. How do users feel about the CTA?
  3. How effective are descriptions that are auto-generated and meaningful? Do they assist users with finalizing the description or confuse users as to why they are prompted to edit a description that is already automated and correct?
  4. How effective is not giving a user a pre-populated description field?
  5. How effective are descriptions that are auto-generated and random?
  6. How do users feel about the process of editing description lines overall?

Findings: Patterns Observed

  1. 3 of 5 users required some level of facilitator prompting to notice the CTA.
  2. Interactivity breakdown:
    1. 2 users would most likely overlook this field, 2 users might notice/interact depending on the situation, and 1 user was not sure.
  3. All users either specifically indicated field interaction was easy and intuitive or had no specific complaint or struggle that was observed. Only 1 user was briefly confused about the blank SF description field, thinking he couldn't type because he didn't see a blinking cursor.
  4. 3 scenarios feedback breakdown
    1. Meaningful description suggestion
      1. most helpful: 2 users
      2. most helpful, but pointless because I can't see it on page: 1 user
      3. confusing: 1 user
    2. Blank
      1. fine if you know about topic: 1 user
      2. fine and having the CTA here made most sense: 1 user
      3. most engaging: 1 user
      4. easiest: 1 user
    3. Random/irrelevant description suggestion
      1. if visible, could prompt action: 2 users
  5. 2 of 5 users expressed some level of confusion around why the CTA hides the description. One user was confused about why he was prompted to action when the description was correct on Picasso. Another user was confused about the same thing, and also mentioned that he would be much more likely to take action on the random article if he could see that the description was incorrect. The latter also mentioned that the CTA really only makes sense on the SF blank description page.
  6. 1 user was confused about whether these descriptions were for himself or for all of Wikipedia.
  7. No users indicated confusion about the CTA pop-up language.

--
Daisy Chen
Wikimedia Foundation

Imagine a world in which every single human being can freely share in
the sum of all knowledge. Help us make it a reality!

https://donate.wikimedia.org