[Foundation-l] [Wikimedia Announcements] Fwd: Announcement: New editor engagement experiments team!

[Theo10011]

Thanks for this email Birgitte. I greatly enjoyed reading it, it gives
insight in not just your own motivation, but mine and several others who I
have come to know. I apologize for my following lengthy response as well.
This is a well-articulated, reasoned response, that should stand apart from
the ongoing discussion.

This does not mean I don't disagree with some of your points in the
discussion. I believe we have two fundamentally different perspectives on
this. It shapes our opinion of where we are and where we are headed
towards. The central difference resides on the difference between an editor
and a member of the crowd. I do not believe every individuals can become an
editor. I should make a clear distinction here that I am referring to
active editors, not just every reader who can incidentally make a
correction to never repeat again. The edits stand on their own, the
individuals might not. That is where we differ on, the crowd we are both
referring to is composed of a large majority of those, and very few actual
editors. The conversion rate between the two has been out of proportion for
some time now.

It may be that collected edits might be what you are referring to here, not
the individual contributor. Collected edits form the wisdom of the crowd,
they are irrespective of who they came from. Editors, curators, new
contributors, vandals, PR agents, occupy the entire spectrum of the crowd.
The issue is between the normal ecosystem that came to be on its own, and
the artificial albeit temporary addition to the equation.

Activities undertaken to artificially boost one side, by incentives and
outreach effort, have not yielded positive results. We are having this
discussion because there is a trend that has developed. The past measures
have not yielded favorable results. It has contrarily, in some cases,
increased the already heavy burden on one side, the backlogs have only
increased through them, so have copyright violations and so on. These
attempts artificially inflate and unbalance the ecosystem, by temporarily
bringing in an unmotivated crowd for the sake of statistics.

On Sun, Mar 25, 2012 at 12:07 PM, <Birgitte_sb at yahoo.com> wrote:

>
> I snipped previous emails because your summary is accurate and this ended
> up being massive. Fair warning.
>
> Let's say this doesn't happen.  Things stay exactly as they are now. No
> increase in vandals nor PR agents nor anything other kind contributor for
> the rest of the year.  Do you imagine the workload for admins and veteran
> editors to be acceptable? Do you imagine the quality of articles to be
> acceptable? They are not.  I and am not talking about award-winning levels
> of quality. I am speaking articles right now that were tagged as being
> inaccurate, contradictory, or biased many months ago yet still are
> unaddressed.  I am thinking of known contributor's of copyright problems
> whose edits are cataloged and are waiting for someone willing to tediously
> review them. I suspect a large factor in the attrition of veteran editors
> is the current workload as it stands.  It is hard to stay motivated when
> you can't hardly notice your work has made any dent in the backlog.
>

Yes, there is a difference between the actual workload, generated by
inaccuracies, copyright violations, policies, plain old vandalism, and the
one brought in artificially to address a trend. One side is already having
a hard time with regular tasks, veteran editors are facing attrition, new
editors are sometimes adding to the backlog instead of lowering it.

Temporarily Incentivizing and bringing in a large number of unmotivated
editors for the sake of numbers, only exasperates the problem. I believe
this is the cost of experimentation MZMcBride and others were referring to.
It only increases the workload over the normal, by temporarily recruiting
one side of the crowd from which only a minority will continue editing.


>
> I suppose I simply see the bigger concern to be: What if we don't add
> 1,000 new curators who care to learn how to interpret copyright law and
> 3,000 new contributors who are willing respond to RfCs and participate in
> peer review?  The vandals will come as inevitably as 8 year-olds transform
> into 12 year-olds. The PR agents are equally likely to remain consistent. I
> don't really understand the basis of the concern that this outreach is
> expected to add more vandals and PR agents. Why is it so suspect that this
> project could add sincere and useful people which are, perhaps in some
> aspects of their personality and/or circles of interest, simply a different
> kind of person than you and I who self-selected to contribute without any
> such an overt program? But truthfully while there are certainly tasks I
> selected to work on my own, because I find them inherently captivating
> (poetry) or because I am inherently driven to understand and make sense out
> what is presented as arbitrary and seeming senseless to me (copyright law),
> there are many contributions of significance I have made only because an
> overt effort was made asking me to contribute personally (peer review
> Evolution pre-FA) or by a generalized campaign (Proofreads of the Month).
>  So perhaps, the people brought in by such outreach won't be such a
> different kind of contributor after all.
>

We can not easily recruit curators. We can make it a part of someone's
curriculum, provide workshops and teach classes, but it can not motivate
someone to do so beyond what is required. Curators of content,
would indisputably have to be self-motivated. New contributors might not
understand what an RfC is, how to do peer review, our job could only be to
educated them to the best of our abilities. A job, we haven't been doing
too well to begin with.

There is also something to be said about the costs. These experiments
monetarily costs a good deal, not to mention the workload it requires from
volunteers. All of which pales in comparison to the cost of making new
editors on English Wikipedia, a priority; the focus for WMF remains those
new editors. It is in my mind a travesty to focus on them and overlook the
existing veteran editors and curators. They are deserving of more attention
than what they receive, this problem only compounds to an abject
proportion, when you start moving away from English Wikipedia, to sister
projects and other languages. Curators are not easy to recruit, while
veteran editors face attrition, our measures completely ignore this core
group that basically power the projects everyday, for something that might
affect the statistics and the conversion rates for tomorrow.

I am not suspect that we can not add well meaning and sincere individuals
willing to contribute, I, like others, am only basing it on the past
experiments. It is not an easy task, past attempts to engineer the
community and contributors have not yielded the expected results, it does
not mean that future experiments will not succeed. The priority there
should be containing any such fallout from the existing ecosystem.

Disproportionate work load, is another issue. The automated tools and their
proliferate usage is symptomatic that the existing community does not have
the time to deal with the workload under normal circumstances. This is in a
closed environment of its own, the tools are community made and used; a
reaction to the disproportionate distribution. Add to it attempts to
engineer the community and incentivize individuals who don't have the
patience to commit, this would get far too disproportionate, you would have
more attrition than ever before. And sadly, those attrition statistics
aren't followed as closely as the new editors.


> If the underlying concern is that there are not enough veteran editors
> willing to educate them, or that these newcomers won't conform to our ways
> sufficently. Well then maybe the newcomers can educate us instead. We are
> great at some things we have done, but we are crap at a whole bunch of
> other things.  And was all trial and error to begin with! If new people
> come and want to do things differently, I can only imagine they will be
> trying to change the crap things not the great ones.
>

Perhaps. But I have rarely seen a large, functional, existing communities
change to accommodate new members. From an example already cited on this
thread, there might be more credence to this claim already.


> I disagree with this summary.  In fact, the wisdom of crowds is considered
> wiser because it assumes the members are *not* homogeneous. The model,
> however, does not give more weight to members with qualities that normally
> would earn them weight in more traditional models, which the point most
> people find counter-intuitive. The reason a larger crowd is supposed to be
> wiser is because a larger crowd is assumed to be more diverse.  I do not
> see see why a diversity of motivations to participate should be any less
> desirable than other forms of diversity.
>

There is probably truth in that statement. But I still believe that
motivation separates the crowd, when half the contributors are trying to
work in good faith and be productive, the other half might be consciously
trying to sabotage the wisdom. No statistical method of analysis can take
this into account. If people intentionally promote lies and falsehood
within a group, it only distorts the end results.


> I agree I was speaking of the wisdom of crowds model more than the wiki
> model. I do think there is a wiki-model which has emerged by happenstance
> and in fleshing it out below realized that the wisdom of crowds is not
> really inherent to it. For the record, I  believe the key facets of a
> working wiki are as follows:
>
> *Low barriers to participation
> *Self-Governance by participants
> *Participation is transparent
> *Critical mass of participation is maintained
>
> These are what seem to separate wikis which flourish from wikis which
> whither.  But why use a wiki if you do not want to form the wisdom of
> crowds? Looking up the key factors for wisdom of the crowds are (these I am
> not just throwing out there like the above which I really did write before
> looking this up!):
>
> *Diversity of opinion
> *Independence
> *Decentralization
> *Aggregation of output
>
> So you can so see why wiki's are such a good model for forming the wisdom
> of the crowds. Critical mass will often naturally grant a diversity of
> opinion. Self-Governance inherently grants independence and often leads to
> decentralization (although I can imagine wiki participants choosing a
> centralized governance model and losing that one). Transparency means
> records which preserves all the raw data needed for aggregation. So the two
> models are a natural fit and tend to feed into one another. After all, it
> is hard to imagine a critical mass of participants governing themselves in
> any way at all familiar to our experience without forming the wisdom of the
> crowds.
>

I agree with that summarization.


>
> I addressed some of this above and I hope the inline replies do not annoy
> you, it easier for me to think of it in pieces.  I am probably less
> "social" than the average contributors, but still I feel the community
> aspect is truly necessary.  Especially, with regard to the more tedious
> chores of curation. If all we needed was people to share their writings on
> subjects they are passionate about, I would hold your opinion.  But
> community is the power driving much necessary and tedious work (copyvio!).
> Honestly I burned out on copyright years ago. But I remember thinking
> myself once upon a time thinking to myself "I cannot leave all of this work
> for Moonriddengirl to do, no one else helping her! I' ll at least fix X
> before calling it a night." I do not believe I had any purely social
> interaction with her at the time of that experience (or ever!), but just
> receiving some explanation from her and seeing her doing so much good work
> made me feel an obligation to pitch in.  I am certain SJ means community
> building as working together on a common objective and necessary education
> of any contributors new to some area of the wiki, rather than socializing
> for it's own sake.  Proofread of the month at Wikisource is probably a good
> example, but he can correct me if I have misunderstood. Along these lines,
> I personally find community to be supremely motivating.
>

Ok there is a distinction here. You were part of the community first, you
*chose* at some point to work on something in this ecosystem without any
incentive. The community aspect came in later, to retain and constantly
engage you. There are different approaches to this. I hope you will agree
that there are several prolific editors who are even less social than
anyone. They can continue to contribute year after year without so much as
a single community-oriented interaction. There are those who remain
near-anonymous, not revealing a single facet about their personality or
engaging any other editor. It used to be once that they were not in the
minority, as it may be now, but their work is not affected by lack of the
community aspect that might drive others. In my opinion, community aspect
is more important to retention and engagement, than recruitment.

It takes most new editors months until their first direct interaction
on-wiki. I have known editors who said they went for an year before they
had a direct interaction with another editor. It was usually limited to
warnings and corrections, some have spent even longer without it. It rarely
affects their work, it does affect their own motivation if they continue
editing in vacuum.


>
> Truthfully I am highly self motivated to explore and understand things,
> the community, however, is what motivates me to share my understanding.  If
> I didn't feel community engagement, I imagine I would have merely lurked
> till I got busy and forgot I had been fascinated by the place. I am sure I
> would still consult Wikipedia and Wikisource, even if I hadn't ever
> belonged to the community but I would rarely remember to pull back the
> curtain and lurk. I have lurked in scores of places over the years,
> sometimes one thing in particular will really motivate to comment. Mostly I
> begin to comment and find the barrier to participating troublesome and
> change my mind, less often I actually leave a comment on the topic that
> motivated me. Nowhere else have I stuck around more than three months. And
> I am not even sure how you guys laid a claim on me. And it is not the
> overall project, although I really still find the whole idea as marvelous
> as ever, it the people that I bound to more than the idea.  As I said I
> really don't have the strong individual social ties that I see among others
> here. So I mean the collective of people, the collective which also sees
> this Marvelous Possibility which barely resembles the feeble attempts we
> have managed so far. I feel this collective belongs a little bit to me and
> owns a little bit of me in return. That it would be irresponsible of me,
> no, worse dishonorable of me, not to tell you guys that I see rocks ahead
> when I happen to see rocks ahead. Or to stay quiet when people are talking
> about the sky falling and it all seems rather normal to me.  I know people
> can perceive me as angry sometimes (often?), but really I am not so
> attached to what is finally decided out of my line of sight.  However, I do
> feel this obligation with you guys that I feel no where else outside of my
> employer, with those things sent for public consumption. That I have share
> any strong convictions with you, even when I find it unpleasant, in order
> to sleep in good conscience. It matters much less to me whether or not my
> opinion prevails than that I offered it, that I discharged my duty. So this
> is all rather anecdotal, but I believe community engagement is really the
> only thing that has ever driven my participation past the curious stage. I
> wanted to help the hard-working and helpful *people* I observed on the
> wikis long before I understood the full implications of the project.  And
> then at some point, which I can't pin down, it had become my community
> where I was obligated to share myself, which is an even stranger thing in
> my experience. This part is probably incomprehensible, but I cannot
> articulate these thoughts any clearer. I tried not to use the word
> community to steer clear of begging the question in describing community
> engagement, but substitute community for any of that if makes more sense.
>

And this is the part I am most grateful to you for writing. I have nothing
more to say about this then, Thank you. I see my own self in more than half
of what you are describing.(I hope Sj and a couple of other board members
can see the similarities in the part about being perceived angry, warning
when I see rocks, and sharing my strong convictions.)


> Maybe my non-article writing background is influencing me here, but there
> really are simple tasks anyone on Facebook could do (Proofread of the
> month). There are things that can be done entirely individually without
> *needing* to understand a single policy tome (peer review). I also imagine
> a lot of potential for breaking down existing backlogs into some
> incremental tasks which could used as introduction to more complex issues.
> I am sure there is all kinds of other stuff I haven't been exposed as well.
>  And really do you think people on Facebook really care that much about
> some song or what they ate? Those people are just sitting in front of a
> computer feeling bored. They are being directly prompted  to "post your
> status"  while eating a plate of food and the radio playing in the
> background.  They could equally satisfy their boredom being asked to do
> something useful on a wiki.  I don't  imagine Facebook really appeals to
> them any more than it appeals to me. It probably just appeals to them more
> than nothing at all.


There is still some requirement to contribute I suppose, just as it would
be to reading a newspaper or writing a letter - Not everyone is cut out to
be an editor. I don't believe that those are the same skill sets that
entail pressing "Like" to something, and editing an article. The
distinction again, is editing not performing simplified task that would
only require someone to press a button. I would reiterate that Facebook is
a social networking platforms, after all the "Likes", status updates and
"lol"s, there would be nothing left of value. In our case, what we are left
with in the end, is the goal, the other aspects of a community might only
be incidental in the end. No one is stopping anyone from relieving their
boredom on Wikipedia, the person just has to choose. Maybe we can find
incremental tasks and setup ways to make certain tasks as easy as playing a
game on FB, but then again, isn't that one of the criticism about automated
tools and turning patrolling into a game.

Regards
Theo