[Foundation-l] Community, collaboration, and cognitive biases

Tue Jun 8 22:54:41 UTC 2010

Hoi,
As you may know, I had reservations about the fact that the UX was funded
for the English language Wikipedia only. This turned out well; there was
attention for right to left languages, testing environments for several
languages were created. In the tools there was time to include character
sets for several scripts. At translatewiki.net there are 137 languages with
some localisations for the UX software.

It has been a joy to see Naoko do her work; her hands on and agile
involvement has been a major contributing factor for the success of this
project. Some people will say that it is not perfect but it never is and
that is to be expected. <grin> Yes, her team gelled together ... I was happy
to observe ...

If there is one thing that will help close the gap between staff and
programming volunteers, then it is a "bugmeister" ie someone who takes the
time to analyse and comment on code provided from outside the projects. One
practical example; the Babel extension would be really welcome particularly
on smaller projects. There is nobody who has time or inclination to look at
it.. at the same time it is live on several wikis and there are several
projects who asked for it.
Thanks,
       GerardM

On 9 June 2010 00:31, Erik Moeller <erik at wikimedia.org> wrote:

> Hello all,
>
> the massive thread regarding the default sidebar language link
> expansion state has surfaced a number of fundamental and significant
> questions regarding the working relationship between the Wikimedia
> Foundation and the larger Wikimedia volunteer community. This message
> represents my snapshot-in-time thinking about these questions, and I
> hope others will join this thread to meaningfully advance this IMO
> very important conversation.
>
> Tl;dr version: I believe that the transparency of Wikimedia's
> engineering processes, and the general quality of these processes, has
> significantly improved over the last year. At the same time, I agree
> with those who are observing a widening gap between staff and
> volunteer contributions, and I think we need to work together to close
> this gap in full awareness of the cognitive biases present in all of
> us.
>
> 1) Increase in quality and transparency of our engineering processes:
>
> I'm very grateful for the hard work accomplished by the User
> Experience Team to-date. When we started the usability project, we had
> never run a discrete engineering effort of comparable scale before,
> and indeed, most of our larger-scale engineering projects tended to be
> drowned by day-to-day emergencies and bug fixes. The team consists
> primarily of people who didn't have a long prior history in Wikimedia
> projects, nor with each other: they had to grow as a team, understand
> our enormously complex community, while also delivering results. I
> want to use this opportunity to explicitly thank Brion, who played a
> key role in the orientation of the team.
>
> The UX team has implemented a number of important practices:
>
> * very frequent sharing of detailed work product, including mock-ups,
> specs, and research results such as user test videos.
> * systematic QA using an external QA firm, browser compatibility
> matrix, etc.; beginnings of automated testing
> * qualitative remote and in-person user testing to assess real-world
> experience with the site
> * moving towards data-driven decision-making using tools such as
> click-tracking, systematic assessment of feedback data, various
> statistics, etc.
> * public and frequent blog updates both in the regular and in the
> technical blog, use of the sitenotice to announce deployments
> * public sandboxes demonstrating all bleeding edge improvements early
> * various community-oriented discussion pages and documentation on
> usability.wikimedia.org informing and supporting design and
> localization
> * largest systematic feedback collection about any software deployment
> in the history of Wikimedia projects
>
> Many of these were "firsts" for the Wikimedia Foundation, and they all
> were firsts on this scale. Some of these practices have clearly
> contributed to letting the community participate _more_ in the overall
> initiative: the larger-scale outreach, the sharing of research
> findings (which led to some community-driven improvements), the
> carefully prepared public testbeds, etc. Some, on the other hand, have
> arguably widened the staff and volunteer gap, both between staff
> developers and volunteer developers, and between staff and larger
> volunteer community.
>
> I don't think Wikimedia's development processes are anywhere near
> those of proprietary web companies, but it's absolutely true that
> they're also not highly comparable to 100% volunteer-driven open
> source projects anymore. I think there are some parallels with both
> the strengths and weaknesses of open source projects with reasonably
> well-funded organizations behind them, both for-profit and non-profit.
>
> I also want to be clear that there's no doubt in my mind that our
> ability to execute projects of this _type_ and at this _scale_ is
> unprecedented, and a historic achievement for the Wikimedia movement.
> MonoBook was implemented in 2004, and the basics of the user interface
> and user experience of Wikipedia have changed very little between then
> and pre-Vector.
>
> The reasons for that are varied. Fundamentally, I believe that online
> volunteer mass collaboration works best when incrementalism and
> self-selection can, over a long period of time, add to an overall
> product that's useful at any given time -- such as Wikipedia or
> Wikimedia Commons. We've seen this work less well where we need to
> collaborate in a short period of time to achieve a result of
> predictably high quality (Wikinews), or where we're working over long
> periods of time on a result with (perceived or actual) limited
> usefulness until it is complete (Wikibooks).
>
> I'm not saying this is inherent in volunteer mass collaboration, or
> that these challenges are insurmountable, just that a) tools and
> practices of volunteer collaboration tend to support self-selected
> long-term incrementalism better than, shall we say, strategic
> megaprojects, b) we're not necessarily even using all the tools and
> practices yet that other volunteer projects successfully employ to
> tackle strategic challenges.
>
> 2) Widening gap between staff and volunteer contributions:
>
> As the Wikimedia Foundation began to professionalize, it initially
> moved its focus away from some of the early experiments in volunteer
> mobilization (such as the 2006 Wikimedia Foundation committees), and
> moved instead towards employing and tasking WMF staff for each
> critical work area. Much of 2008 was spent for this new team to get
> its feet on the ground and get oriented in the world of Wikimedia, to
> the detriment of developing new structures for volunteer engagement.
>
> The strategic planning process launched in 2009 was, from the
> beginning, conceptualized to mobilize the Wikimedia community in
> collecting ideas and planning its own future, and has achieved
> unprecedented volunteer participation in doing so. We're now shifting
> towards using the StrategyWiki to help volunteers in efforts to
> self-organize around particular issues, whether the WMF is giving
> attention to them or not -- we'll be posting more about that soon, and
> I'm pretty excited about it.
>
> At the same time, when tackling large scale strategic projects like
> the usability initiative or the bookshelf, I do believe we haven't yet
> succeeded at forming the most productive collaborative relationship
> possible with the larger volunteer community. I should note that we
> haven't even fully succeeded at closing the gap between staff and
> contractors based in San Francisco, and those who are not -- we have a
> long way to go.
>
> The staff/volunteer gap is evident when looking at the commit history
> of the UsabilityInitiative codebase, for example, which is very much
> dominated by WMF staff and contractors. (The notable exception, as for
> all our code, is localization -- again a great example for
> self-selected incrementalism.) It's also evident in the recent
> conflict between a staff and a volunteer developer regarding the
> language link issue, and in our overall conversation about that same
> issue.
>
> 3) Closing the gap and overcoming cognitive biases:
>
> I do think it's possible and necessary to close that gap. I think
> doing so has to _start_ with discussion, but ultimately to me this
> isn't so much about conversation, but about actual real-world
> collaboration with practical results -- we want to get stuff done
> together, and we're all working towards the same vision.
>
> To do so, I think we have to be wary of our respective cognitive
> biases and operate with the highest possible degree of self-awareness.
> I find the framework of cognitive bias personally very helpful when
> I assess my own actions, and the list on Wikipedia
> ( http://en.wikipedia.org/wiki/List_of_cognitive_biases ) is a good
> starting point. I believe we've seen some examples of biases listed
> there in both WMF staff and volunteer actions, such as:
>
> * belief that the group you're a member of is diverse, whereas other
> groups are not ("the staff does/is X", "the community doesn't have
> the expertise to do Y") ;
> * preferential treatment of people perceived to be members of
> the same group, and the associated us vs. them dynamic;
> * irrational escalation (tendency to defend an investment based on
> cumulative prior commitment) and the bandwagon effect
> (tendency to join the belief of others).
>
> These and other biases contribute to unhealthy spirals of escalation.
> There's a big picture of clear alignment of values and interests:
> we've all committed ourselves to a very important mission, and we
> have seen time and again that we can work together immensely
> successfully in doing so. Fundamentally, I think it's the group
> division that's the most important to overcome -- inside
> the Wikimedia movement, but also towards readers and "outsiders."
> WMF absolutely has been both part of the problem, and part of
> efforts to solve it.
>
> I entirely agree that we need to steer clear of unhealthy power dynamics.
> Wikipedia editing is not a tyranny of the majority or the minority, and nor
> is
> our engineering and design process -- ideally, the best arguments, the
> best code and the best edits should prevail. I'm glad that the BOLD,
> revert, discuss cycle --
> http://en.wikipedia.org/wiki/Wikipedia:BOLD,_revert,_discuss_cycle --
> was invoked in prior threads, which I think is a generally sensible
> framework through which we get stuff done. It's not the only framework,
> but it's one that we (should) stick to wherever reasonably possible.
>
> To the extent that there is an irrational status quo bias, I think the best
> way to overcome it is for us to engage the largest possible number of
> people in bringing about and advocating on behalf of positive changes.
>
> With all this in mind, here are just a few concrete ideas for closing the
> gap:
>
> 1) Embedding teams funded by WMF into larger, publicly visible
> workgroups which include volunteers and which meet regularly e.g. via
> IRC;
> 1 a) Outreach to grow and strengthen those workgroups with the best
> skills present in the Wikimedia volunteer community;
> 2) Publication of mini-projects which we identify and which can be
> tackled by self-organized volunteers, with mentoring by experienced
> WMF staff and volunteers (happening a bit via GSoC, but not as much
> outside of it yet);
> 3) Making development roadmaps fully transparent and open to
> discussion, and sharing justifications for all key priorities;
> 4) Further iteration of tools and processes for rapid volunteer-driven
> bug reporting, cross-browser testing, and submission of simple
> patches;
> 5) Stimulating larger scale contests focusing on specific areas of
> interest;
> 6) More topic-focused meetings and sprints like the multimedia
> usability meeting in Paris.
> 7) Further experimentation with tools like IdeaTorrent for large-scale
> brainstorming and ranking purposes (we have a prototype running at
> http://prototype.wikimedia.org/en-idea/ideatorrent/ ).
>
> The Wikimedia Foundation regards these as crucial questions, and
> we've had many conversations about them, both internally and
> publicly. I hope these conversations will continue to broaden.
> I personally intend to work especially on 3) in the list above, and want
> to help establish 1) as a normal practice in key WMF program activities.
> Howie has started a general brainstorming page here, where I've
> reposted the above:
>
> http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas
>
> I hope you'll join me in adding to that list, and that we continue to
> hold each other to the idea of building true and genuinely open forms
> of collaboration, which is one of the most important cornerstones of
> the Wikimedia movement.
>
> All best,
> Erik
> --
> Erik Möller
> Deputy Director, Wikimedia Foundation
>
> Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
>
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>