[Wikimedia-l] Editor retention (was Re: "Big data" benefits and limitations (relevance: WMF editor engagement, fundraising, and HR practices))

[Steven Walling]

On Thu, Jan 3, 2013 at 11:02 PM, Erik Moeller <erik at wikimedia.org> wrote:

> On Thu, Jan 3, 2013 at 8:13 PM, Tim Starling <tstarling at wikimedia.org>
> wrote:
>
> > It should be obvious that what is missing is discipline. An
> > arbitration committee with expanded scope, with full-time members
> > funded by the WMF (at arm's length for legal reasons), could go a long
> > way towards solving the problem. Some users will be reformed when
> > their technical power is threatened (be that editing or admin access),
> > others will just leave as soon as their reputation is at stake.
>
> I do agree that better mechanisms for dispute resolution, dealing with
> topic warring, article ownership, and plain old incivility are needed.
> But I don't believe that those issues are at the heart of the "editor
> retention problem" as you seem to suggest, but rather, that they tend
> to occur later in the editor lifecycle, among a subset of editors
> which in fact already has survived many of the primary factors that
> deter new editors and are therefore relatively likely to retain. The
> new editor experience is characterized more by templating and assembly
> line style enforcement of existing policies than it is by incivility,
> topic warring, article ownership and incivility.
>
> I'm wondering whether the key findings in Halfaker's recent "rise and
> decline" paper resonate with you:
> http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/
>
> Existing data like the above supports strongly the notion that
> well-intentioned, good faith contributors are much more heavily
> discouraged in 2012 than they were in 2004 or 2005, but this can be
> explained in significant part with the influx of bad faith
> contributors that have necessitated increasingly heavy handed ways to
> control against bad edits (Huggle, Twinkle, AbuseFilter, etc.) --
> which catch good faith editors in the crossfire -- as well as
> increasing expectations of what constitutes an acceptable quality edit
> / page creation.
>
> In an environment where most folks who show up want to help, it's easy
> to be welcoming and supportive of new contributors. As Wikipedia had
> to deal with more and more spammers, crackpots and assholes, while
> simultaneously being more and more scrutinized in terms of quality and
> reliability, new users have increasingly been seen as "guilty until
> proven innocent" and are dealt not so much in a deliberately uncivil,
> but more in an assembly line robotic fashion that's highly
> discouraging. Templating with standard messages, no matter how
> friendly, is much more common than explicit incivility toward a new
> user and lack of any form of personal encouragement or gratitude.
>
> If that is correct, then the answer -- at least for very new users --
> isn't first and foremost a more "disciplined" enforcement of existing
> policies. Rather, new editors are simply treated in a manner that's
> discouraging more than it is encouraging, without that treatment being
> in violation of any policy -- indeed, with various policies in fact
> calling for precisely such discouraging actions to be taken in order
> to preserve quality, to enforce notability and sourcing policies, etc.
>
> The answer, then, is to find ways to make the new user experience more
> encouraging and pleasurable, such as:
>
> * simplifying the interface so that we can at least get rid of
> technical reasons that lead to early edits being unsuccessful and
> reverted (Visual Editor, talk page replacement, notifications, etc.);
> * making it easy to find things to do that are relatively low-risk
> (something the E3 team is experimenting with right now) so that new
> editors can have a more ladder-like experience of becoming good
> contributors;
> * guiding the new user in a clear and instructive manner, and pointing
> them to places where they can get help from another human being (cf.
> Teahouse)
>
> More disruptive technical solutions could include:
>
> * safer alternative work/collaboration spaces that don't suffer from
> the contention issues of the main article space (sandboxes on
> steroids)
> * easier ways for new users to re-do an edit that has been reverted
> (cf. https://meta.wikimedia.org/wiki/Research:Improve_your_edit )
> * real-time mechanisms for coaching, collaboration (chat, real-time
> collaborative editing) and mentor matchmaking
>
> More disruptive policy-level changes would include rethinking some of
> the more problematic quality-related policies, especially notability.
>
> That's not to say that we should ignore the deeper social issues that
> arise in maintaining a universal encyclopedia in a radically open
> manner (and indeed, the community has learned, evolved and continually
> improved its ways of dealing with those issues). But most new users
> give up well before encountering those issues. When new editors
> complain about Wikipedia being mean, they complain more often about
> reverts, templating, deletion nominations, etc. -- none of which are
> in fact inherently "uncivil" according to Wikipedia's own policies,
> but rather part of its overzealous immune system. In other words,
> rudeness is in the eye of the beholder.
>
> All best,
> Erik
>

I am very much in agreement with Erik's description of the issue and
potential solutions.

Like he said, E3 is running an experiment where we ask people who just
registered on English Wikipedia to try their hand at copyediting some
articles, if they don't have an activity in mind already. We're still doing
analysis of the first phase, so the rest of the team will shoot me if I
trot out any specific data, but suffice it to say that we've got a decent
chunk of new editors to try this out so far. When you inspect the edits of
users who did this, you see some patterns which match the conclusions of
the paper Erik cites above:

Vandals or those making mistakes are getting reverted and warned. Those who
actually made constructive edits like asked are either ignored, or given a
welcome template with a laundry list of policies and other documentation. A
very small number of the latter group figure out how to find more to do,
and keep doing it. In short, we are focusing on the negative as a
community, and leaving potentially helpful new community members to try and
figure everything out on their own.

In any case, sorry for the earlier snark. Suffice it to say that it peeved
me to hear anyone try and chalk up an extremely complex problem like
Wikipedia editor retention to some kind of basic flaw in human nature,
which causes grumpy oldbies to hoard power. I can save my debate with Tim
and Risker concerning their disturbing ideas about oligarchy for later,
since I don't want anyone really cares to hear me expound on that.

Steven