As much as I like the barnstar system, it's
highly subjective and
inconsistent. I'd like to see a more systematic approach. Perhaps this
could be combined with some of Aaron's work about edit quality.
On Sat, Oct 17, 2015 at 2:52 AM, WereSpielChequers <
werespielchequers(a)gmail.com> wrote:
We have a complex set of "badges",
some, as Kerry pointed out are
available to everyone who qualifies for them, some are based on the
statistics of your account - tenure, edit count, articles created. Others
are based on things you've been awarded by others, the bronze stars for
featured articles, but also userboxes for everything from userrights to
number of DYKs. Barnstars are a key subset that can only be awarded by
others. There are Barnstars available for a huge range of things, even
civility and diplomacy. It would be interesting and probably salutary to do
a study on which Barnstars are awarded, my suspicion is that the anti
vandalism ones may well be the most frequent. I would also encourage
everyone to lead by example and actually use the Barnstar system for people
who have made extraordinary contributions. But be careful not to devalue
the system by for example giving one to everyone who reports a bug in
visual editor - in the past when we had lots of adolescents and teenagers
in the community there was a craze for creating secret pages with a
Barnstar award for finding them; so if you give out Barnstars too freely
you risk being thought of as the sort of immature adolescent that usually
makes that sort of mistake.
Regards
Jonathan
On 14 Oct 2015, at 02:42, Luis Villa <lvilla(a)wikimedia.org> wrote:
I think there's a lot to be done there (probably will blog soon about my
weekend experimenting with Genius, which had pretty extensive systems for
this).
It is an interesting prioritization question: doing it
thoroughly/systematically would require a lot of software investment,
especially since we don't have structured conversation pages (which are the
basis for a lot of similar contributor recognition systems).
Luis
On Sun, Oct 11, 2015 at 5:25 AM, Pine W <wiki.pine(a)gmail.com> wrote:
Kerry,
Thanks so much for the comments. I will bring up the subjects of badges
and cobtributor KPIs with Luis and/or Lila when I have a minute to refine
my thinking.
Pine
On Oct 6, 2015 2:33 AM, "Kerry Raymond" <kerry.raymond(a)gmail.com>
wrote:
> Certainly there are a lot of sites with badges that do seem to
> encourage certain behaviour. On Wikipedia, we have edit count and that
> seems to generate editcountitis which (when gamed) tends to favour lots of
> little housekeeping edits over content edits. But one of the things with
> badges on most sites is that the site assigns the badge. Here on Wikipedia,
> I can put any badge I want on my User Page (the pre-existing ones are
> mostly edit-count based but I can roll my own as some users do). Indeed as
> I discovered, other people can put badges on my user page and presumably
> take them away. As edit count is our primary KPI, it doesn't address
> "cultural" attributes. Should we be making more of an effort to promote
> other KPIs that emphasise positive behaviour like thanks (given and
> received)? Unfortunately our main interaction mechanism is writing on talk
> pages and it's hard to tell whether any contribution on a talk page is a
> "positive" behaviour or a negative one (short of some kind of sentiment
> analysis). This is an unfortunate consequence of using a wiki for a
> conversation rather than some more purpose-built tool.
>
> In principle one takes a KPI and then creates a badge to reward a
> behaviour that improves that KPI. But that's all easier said than done.
>
> For content improvements, there are probably some things we can do.
> For example, I presume looking at the edit deltas, we could tell if an edit
> to an article added a citation (a pair of ref tag in the new version that
> weren't there in the old version). Adding citations is a desirable
> behaviour that we could report on and give badges for (although obviously
> whether or not that citation in any way supports the claim cannot be
> determined, so the "gaming" of this is to add random citations to offline
> sources to lots of articles, which cannot be easily verified). In which
> case maybe we need to give a better score to an online citation on that
> grounds it is more likely to be verifiable).
>
> But positive "culture" or positive social behaviour is harder to
> detect and reward. For example, we'd like to close the gendergap but
> firstly we don't have KPI that measures it on an ongoing basis because we
> don't actually know which contributors are male/female. And even if we had
> that KPI, what users or their behaviours would we reward for having
> positive impact on that KPI? In real-life, we might reward a customer who
> introduces a new customer. Or we might have a "finders fee" for someone
who
> introduces a "new hire". How could we reward introducing new women to
> Wikipedia or encouraging them (perhaps through mentoring) to contribute
> more? Or would we reward contributors who contribute to articles about
> "women's topics" (which is addressing the content gendergap rather than
the
> contributor gendergap, which aren't the same thing although many believe
> them to be closely linked). [I won't disgress into the challenge of
> deciding how "female" an article topic is.]
>
> On some sites, you need certain badges to "unlock" certain extra
> functionalities. Are we happy for RfA to be a question of collecting up
> enough badges? AFAIK, the only auto-implemented badge we have on Wikipedia
> is the "auto-confirm" (4 days and 10 edits from memory).
>
> I think badges are a good idea but I think the way Wikipedia is
> implemented makes it challenging to machine-identify desirable behaviours
> to reward (particularly for social/culture metrics). I think badges have
> (in the most part) to be machine-calculated and awarded or else it just
> becomes a popularity content (who's mates with who). I know Aaron (or
> someone) was toying with the idea of putting a value on each edit
> (presumably based on some training set of edit data that humans rated). I
> think it's not impossible to come up with some set of dimensions on which
> an edit might be valued and, using some human evaluations on a test set,
> come up with some kind of values for each dimension. It might be rough in
> the first instance but I guess if it incorporated some ongoing feedback
> mechanism, it could improve over time.
>
> A cheap thing that we could do (and I don't think we do) is have edit
> count badges for "last week", "last month", "last
year". ATM we only have
> "lifetime" counts, which makes it hard for the new user to get any quick
> positive acknowledgements for their efforts.
>
> Kerry
>
> -----Original Message-----
> From: Wiki-research-l [mailto:
> wiki-research-l-bounces(a)lists.wikimedia.org] On Behalf Of Robert West
> Sent: Tuesday, 6 October 2015 1:05 PM
> To: Research into Wikimedia content and communities <
> wiki-research-l(a)lists.wikimedia.org>
> Cc: Marti Johnson <mjohnson(a)wikimedia.org>rg>; Patrick Earley <
> pearley(a)wikimedia.org>gt;; Jacob Orlowitz <ocaasi(a)wikimedia.org>
> Subject: Re: [Wiki-research-l] Reinforcing or incentivizing desired
> user behavior
>
> This paper is on using badges to steer user behavior:
>
https://www.cs.cornell.edu/home/kleinber/www13-badges.pdf
>
> On Mon, Oct 5, 2015 at 7:42 PM, Pine W <wiki.pine(a)gmail.com> wrote:
> > Hi all,
> >
> > Some of us plan to have a conversation at the WCONUSA unconference
> > sessions about ENWP culture. Are there any recommended readings that
> > you could suggest as preparation, particularly on the subject of how
> > to reinforce or incentivize desirable user behavior? I think that
> > Jonathan may have done some research on this topic for the Teahouse,
> > and Ocassi may have for done research for TWA. I'm interested in
> > applicable research as preparation both for the unconference
> > discussion and for my planned video series that intends to inform
> and inspire new editors.
> >
> > Thanks,
> > Pine
> >
> >
> > _______________________________________________
> > Wiki-research-l mailing list
> > Wiki-research-l(a)lists.wikimedia.org
> >
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
> >
>
>
>
> --
> Up for a little language game? --
http://www.unfun.me
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
Luis Villa
Sr. Director of Community Engagement
Wikimedia Foundation
*Working towards a world in which every single human being can freely
share in the sum of all knowledge.*
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org