[Wikimedia-l] Editor retention (was Re: "Big data" benefits and limitations (relevance: WMF editor engagement, fundraising, and HR practices))

[Tim Starling]

On 04/01/13 18:02, Erik Moeller wrote:
> I do agree that better mechanisms for dispute resolution, dealing with
> topic warring, article ownership, and plain old incivility are needed.
> But I don't believe that those issues are at the heart of the "editor
> retention problem" as you seem to suggest, but rather, that they tend
> to occur later in the editor lifecycle, among a subset of editors
> which in fact already has survived many of the primary factors that
> deter new editors and are therefore relatively likely to retain. The
> new editor experience is characterized more by templating and assembly
> line style enforcement of existing policies than it is by incivility,
> topic warring, article ownership and incivility.
> 
> I'm wondering whether the key findings in Halfaker's recent "rise and
> decline" paper resonate with you:
> http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/

Yes, they do resonate with me. The paper says that established users
who use Huggle and similar tools do not follow best practices when
they revert the edits of new users. This leads to poor editor
retention. I am saying that an expanded arbcom and its delegated
officers should reprimand those Huggle users.

I am not saying that the editor retention problem is the kind of thing
that the arbcom currently deals with. I think the arbcom is limited in
the kinds of problems it can deal with because its mandate and
resources are limited.

> Existing data like the above supports strongly the notion that
> well-intentioned, good faith contributors are much more heavily
> discouraged in 2012 than they were in 2004 or 2005, but this can be
> explained in significant part with the influx of bad faith
> contributors that have necessitated increasingly heavy handed ways to
> control against bad edits (Huggle, Twinkle, AbuseFilter, etc.) --
> which catch good faith editors in the crossfire -- as well as
> increasing expectations of what constitutes an acceptable quality edit
> / page creation.

We need ways to deal with bad faith edits that don't require
destruction of the project to achieve their purpose.

For example, requiring phone number verification for new users from
developed countries would be less damaging.

When a Huggle user drives away a new good faith user, that new user
might not return for decades. You can't reverse it no matter what new
policies you introduce, you just have to wait for another person to be
born and grow up. It would be less damaging to tell them "sorry, we
can't accept any new users from Comcast this year, try again next year!"

Note that the total edit rate has declined from 4.5M in January 2007
to 3.5M per month in October 2012. As a metric of the workload that
places on very active users, consider that figure divided by the
number of users with more than 100 edits per month: it works out to
950 per very active user per month in January 2007, up to 1078 per
very active user per month in October 2012.

So it is hard for me to believe that the total review workload has
increased over that period to such an extent that our only option is
now to revert both good and bad edits on sight, with no discussion.
Presumably the proportion of bad edits has increased, but it should be
quicker to deal with simple vandalism than to review a good faith edit
and engage with the editor.

But we can always do new user phone number verification if enforcing
the revert policy turns out to be too hard, right?

-- Tim Starling