Hi folks,
I wanted to highlight a few really interesting pieces of data/stats
regarding the release of the Suggested Edits feature on Wikipedia app for
Android. These come from the daily report
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html>,
which is also where you'll find a brief description of the feature.
First, at this time 25.3% of editors (whose contributions are being tracked
since launch of the backend) have unlocked the feature by making the 5+ title
description edits
<https://www.mediawiki.org/wiki/Wikimedia_Apps/Short_descriptions>
currently required to unlock it. (See: unlock stats
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Unlock-stats>)
That's 632 editors out of the 2495 editors who have made at least one title
description edit since April 5th. We have plans to experiment with this
threshold and see what happens if we lower the barrier to entry.
By the way, we don’t expect all logged-in users to edit or unlock the
feature (by making the required number of title description edits), as
there are incentives on the mobile apps to use an account just for reading
(e.g. reading list syncing
<https://wikimediafoundation.org/2018/05/25/synced-reading-lists/>).
However, perhaps we should advertise this ability better (especially to
logged-in users) and that those title descriptions don’t require any
knowledge of wikitext.
And since the production release, the feature has had a steady stream of
20+ users unlocking it per day. What are our users doing with it once they
unlock it? They’ve been using it! (Sorry if the text in the included graph
is too small to be legible, it's larger in the report.)
Nearly half of all title description edits made with the Android app each
day are coming in from editors using the Suggested Edits feature to add &
translate descriptions. More than half, even, on some days! Furthermore,
some of those edits are made by users who have previously used the feature.
Every day we have some editors who are using Suggested Edits for the first
time, but there are also quite a few who are returning to the Editor Tasks
screen & contributing more. (See: edit stats
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Edits-and-editors>
)
“Okay, so what’s the quality of those 200-400 descriptions being added
every day?” you might ask. Well, one way we can check that is to check how
many of those edits are reverted within 48 hours. Turns out, almost none of
them:
This is especially impressive when compared to the proportion of other
title description edits that are reverted. (See: revert rate
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Revert-rate>
)
When the user goes to the Suggested Edits screen and opens a task, they
begin receiving suggestions of articles to add descriptions to (or
translate descriptions, if they have unlocked that next tier of Suggested
Edits). On average, users express interest in editing 30-40% of those
suggestions. Among the suggestions they tapped to edit, they end up
actually making an edit around 60% of the time (although the average varies
from 40% to 70%). (See: interactions
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Interaction-with-suggestions>
and other engagement stats
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Usage-stats>
)
Since the suggested edits are currently completely random, this leaves us
with a lot of room for improvement by, say, employing machine learning and
simple recommendation systems to suggest articles without title
descriptions that are similar to articles the user has added title
descriptions to previously. (Just a thought.) For example, in my own
experience with the suggestions I tend to skip articles that I don’t feel
confident enough to write short descriptions for, which are often articles
well outside my interests.
We’re still in the first month of the production release, so it’s hard to
draw conclusions about the longevity of this feature. These early numbers
are promising, and hopefully the number of editors using this feature
continues to grow because then those editors might be inspired to edit
articles too (if they haven’t yet). Of course, if we see people get bored
over time we might have to consider ways to encourage/inspire long-term
use. We also have plans to explore ways to recognize users for their
contributions.
So congratulations to the Android & Reading Infrastructure teams and
congrats to Rita Ho (now on the Growth team) for an impressive release. We
all look forward to the addition of image caption translation and seeing
the impact of the expanded Suggested Edits v2 on Structured Data on Commons.
Thanks for reading! :D
Cheers,
Mikhail
--
*Mikhail Popov*, Data Analyst (he/him)
Product Analytics <https://www.mediawiki.org/wiki/Product_Analytics> //
Audiences <https://www.mediawiki.org/wiki/Wikimedia_Audiences> // Wikimedia
Foundation <https://wikimediafoundation.org/>
Other info (including PGP):
https://people.wikimedia.org/~bearloga/