I thought if we had a "primary" badge or KPI system it was the content focussed
ones and especially those related to Featured articles. Editcountitis is seen by many as a
bit of a joke. But there many others including articles created and length of service. I
do like the idea of celebrating our most thanked editors but I don't think the
necessary information is currently public.
Regards
Jonathan
On 6 Oct 2015, at 07:33, Kerry Raymond
<kerry.raymond(a)gmail.com> wrote:
Certainly there are a lot of sites with badges that do seem to encourage certain
behaviour. On Wikipedia, we have edit count and that seems to generate editcountitis which
(when gamed) tends to favour lots of little housekeeping edits over content edits. But one
of the things with badges on most sites is that the site assigns the badge. Here on
Wikipedia, I can put any badge I want on my User Page (the pre-existing ones are mostly
edit-count based but I can roll my own as some users do). Indeed as I discovered, other
people can put badges on my user page and presumably take them away. As edit count is our
primary KPI, it doesn't address "cultural" attributes. Should we be making
more of an effort to promote other KPIs that emphasise positive behaviour like thanks
(given and received)? Unfortunately our main interaction mechanism is writing on talk
pages and it's hard to tell whether any contribution on a talk page is a
"positive" behaviour or a negative one (short of some kind of sentiment
analysis). This is an unfortunate consequence of using a wiki for a conversation rather
than some more purpose-built tool.
In principle one takes a KPI and then creates a badge to reward a behaviour that improves
that KPI. But that's all easier said than done.
For content improvements, there are probably some things we can do. For example, I
presume looking at the edit deltas, we could tell if an edit to an article added a
citation (a pair of ref tag in the new version that weren't there in the old version).
Adding citations is a desirable behaviour that we could report on and give badges for
(although obviously whether or not that citation in any way supports the claim cannot be
determined, so the "gaming" of this is to add random citations to offline
sources to lots of articles, which cannot be easily verified). In which case maybe we need
to give a better score to an online citation on that grounds it is more likely to be
verifiable).
But positive "culture" or positive social behaviour is harder to detect and
reward. For example, we'd like to close the gendergap but firstly we don't have
KPI that measures it on an ongoing basis because we don't actually know which
contributors are male/female. And even if we had that KPI, what users or their behaviours
would we reward for having positive impact on that KPI? In real-life, we might reward a
customer who introduces a new customer. Or we might have a "finders fee" for
someone who introduces a "new hire". How could we reward introducing new women
to Wikipedia or encouraging them (perhaps through mentoring) to contribute more? Or would
we reward contributors who contribute to articles about "women's topics"
(which is addressing the content gendergap rather than the contributor gendergap, which
aren't the same thing although many believe them to be closely linked). [I won't
disgress into the challenge of deciding how "female" an article topic is.]
On some sites, you need certain badges to "unlock" certain extra
functionalities. Are we happy for RfA to be a question of collecting up enough badges?
AFAIK, the only auto-implemented badge we have on Wikipedia is the
"auto-confirm" (4 days and 10 edits from memory).
I think badges are a good idea but I think the way Wikipedia is implemented makes it
challenging to machine-identify desirable behaviours to reward (particularly for
social/culture metrics). I think badges have (in the most part) to be machine-calculated
and awarded or else it just becomes a popularity content (who's mates with who). I
know Aaron (or someone) was toying with the idea of putting a value on each edit
(presumably based on some training set of edit data that humans rated). I think it's
not impossible to come up with some set of dimensions on which an edit might be valued
and, using some human evaluations on a test set, come up with some kind of values for each
dimension. It might be rough in the first instance but I guess if it incorporated some
ongoing feedback mechanism, it could improve over time.
A cheap thing that we could do (and I don't think we do) is have edit count badges
for "last week", "last month", "last year". ATM we only
have "lifetime" counts, which makes it hard for the new user to get any quick
positive acknowledgements for their efforts.
Kerry
-----Original Message-----
From: Wiki-research-l [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of
Robert West
Sent: Tuesday, 6 October 2015 1:05 PM
To: Research into Wikimedia content and communities
<wiki-research-l(a)lists.wikimedia.org>
Cc: Marti Johnson <mjohnson(a)wikimedia.org>rg>; Patrick Earley
<pearley(a)wikimedia.org>rg>; Jacob Orlowitz <ocaasi(a)wikimedia.org>
Subject: Re: [Wiki-research-l] Reinforcing or incentivizing desired user behavior
This paper is on using badges to steer user behavior:
https://www.cs.cornell.edu/home/kleinber/www13-badges.pdf
On Mon, Oct 5, 2015 at 7:42 PM, Pine W
<wiki.pine(a)gmail.com> wrote:
Hi all,
Some of us plan to have a conversation at the WCONUSA unconference
sessions about ENWP culture. Are there any recommended readings that
you could suggest as preparation, particularly on the subject of how
to reinforce or incentivize desirable user behavior? I think that
Jonathan may have done some research on this topic for the Teahouse,
and Ocassi may have for done research for TWA. I'm interested in
applicable research as preparation both for the unconference
discussion and for my planned video series that intends to inform and inspire new
editors.
Thanks,
Pine
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
Up for a little language game? --
http://www.unfun.me
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l