I think that the mobile Suggested Edits feature and the related analysis
are interesting. I am forwarding text from that discussion.
Also, I was happy to see the recognition of two new user groups. They are
the WikiClassics User Group and the Wikimédiens du Bénin User Group.
What's making you happy this week? You are welcome to comment in any
language.
Pine
(
https://meta.wikimedia.org/wiki/User:Pine )
---------- Forwarded message ---------
From: Charlotte Gauthier
Date: Tue, May 14, 2019, 08:07
Subject: Re: [WikimediaMobile] Stats & data from launch of Suggested Edits
on Wikipedia Android app
Thanks for this great info, Mikhail, and for the work that went into
producing this useful - and beautiful! - report.
We do indeed have plans to experiment with the thresholds to find the
optimum balance point between wide adoption and maintaining contribution
quality, to extend the feature with new microcontribution types, and to
begin intensive testing to find ways to encourage users to explore more of
the feature, and to inspire long-term use. Advertising new contribution
types and re-activation of editors who have previously tried the feature
will be one of the mainstays of our push messaging strategy.
If anyone is interested in our particular plans over the next year, don't
hesitate to get in touch. We've already been having wider discussions with
several teams on how we could possibly work together towards mutual goals.
Cheers,
Charlotte
On Fri, May 10, 2019 at 9:37 PM Mikhail Popov <mpopov(a)wikimedia.org> wrote:
Hi folks,
I wanted to highlight a few really interesting pieces of data/stats
regarding the release of the Suggested Edits feature on Wikipedia app for
Android. These come from the daily report
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html>,
which is also where you'll find a brief description of the feature.
First, at this time 25.3% of editors (whose contributions are being
tracked since launch of the backend) have unlocked the feature by making
the 5+ title description edits
<https://www.mediawiki.org/wiki/Wikimedia_Apps/Short_descriptions>
currently required to unlock it. (See: unlock stats
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Unlock-stats>)
That's 632 editors out of the 2495 editors who have made at least one title
description edit since April 5th. We have plans to experiment with this
threshold and see what happens if we lower the barrier to entry.
By the way, we don’t expect all logged-in users to edit or unlock the
feature (by making the required number of title description edits), as
there are incentives on the mobile apps to use an account just for reading
(e.g. reading list syncing
<https://wikimediafoundation.org/2018/05/25/synced-reading-lists/>).
However, perhaps we should advertise this ability better (especially to
logged-in users) and that those title descriptions don’t require any
knowledge of wikitext.
And since the production release, the feature has had a steady stream of
20+ users unlocking it per day. What are our users doing with it once they
unlock it? They’ve been using it! (Sorry if the text in the included graph
is too small to be legible, it's larger in the report.)
Nearly half of all title description edits made with the Android app each
day are coming in from editors using the Suggested Edits feature to add &
translate descriptions. More than half, even, on some days! Furthermore,
some of those edits are made by users who have previously used the feature.
Every day we have some editors who are using Suggested Edits for the first
time, but there are also quite a few who are returning to the Editor Tasks
screen & contributing more. (See: edit stats
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Edits-and-editors>
)
“Okay, so what’s the quality of those 200-400 descriptions being added
every day?” you might ask. Well, one way we can check that is to check how
many of those edits are reverted within 48 hours. Turns out, almost none
of them:
This is especially impressive when compared to the proportion of other
title description edits that are reverted. (See: revert rate
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Revert-rate>
)
When the user goes to the Suggested Edits screen and opens a task, they
begin receiving suggestions of articles to add descriptions to (or
translate descriptions, if they have unlocked that next tier of Suggested
Edits). On average, users express interest in editing 30-40% of those
suggestions. Among the suggestions they tapped to edit, they end up
actually making an edit around 60% of the time (although the average varies
from 40% to 70%). (See: interactions
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Interaction-with-suggestions>
and other engagement stats
<https://analytics.wikimedia.org/datasets/wikipedia-android-app-reports/suggested-edits-v1.html#Usage-stats>
)
Since the suggested edits are currently completely random, this leaves us
with a lot of room for improvement by, say, employing machine learning and
simple recommendation systems to suggest articles without title
descriptions that are similar to articles the user has added title
descriptions to previously. (Just a thought.) For example, in my own
experience with the suggestions I tend to skip articles that I don’t feel
confident enough to write short descriptions for, which are often articles
well outside my interests.
We’re still in the first month of the production release, so it’s hard to
draw conclusions about the longevity of this feature. These early numbers
are promising, and hopefully the number of editors using this feature
continues to grow because then those editors might be inspired to edit
articles too (if they haven’t yet). Of course, if we see people get bored
over time we might have to consider ways to encourage/inspire long-term
use. We also have plans to explore ways to recognize users for their
contributions.
So congratulations to the Android & Reading Infrastructure teams and
congrats to Rita Ho (now on the Growth team) for an impressive release. We
all look forward to the addition of image caption translation and seeing
the impact of the expanded Suggested Edits v2 on Structured Data on Commons.
Thanks for reading! :D
Cheers,
Mikhail
--
*Mikhail Popov*, Data Analyst (he/him)
Product Analytics <https://www.mediawiki.org/wiki/Product_Analytics> //
Audiences <https://www.mediawiki.org/wiki/Wikimedia_Audiences> // Wikimedia
Foundation <https://wikimediafoundation.org/>
Other info (including PGP):
https://people.wikimedia.org/~bearloga/
_______________________________________________
Mobile-l mailing list
Mobile-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l