Hello all,
the massive thread regarding the default sidebar language link expansion state has surfaced a number of fundamental and significant questions regarding the working relationship between the Wikimedia Foundation and the larger Wikimedia volunteer community. This message represents my snapshot-in-time thinking about these questions, and I hope others will join this thread to meaningfully advance this IMO very important conversation.
Tl;dr version: I believe that the transparency of Wikimedia's engineering processes, and the general quality of these processes, has significantly improved over the last year. At the same time, I agree with those who are observing a widening gap between staff and volunteer contributions, and I think we need to work together to close this gap in full awareness of the cognitive biases present in all of us.
1) Increase in quality and transparency of our engineering processes:
I'm very grateful for the hard work accomplished by the User Experience Team to-date. When we started the usability project, we had never run a discrete engineering effort of comparable scale before, and indeed, most of our larger-scale engineering projects tended to be drowned by day-to-day emergencies and bug fixes. The team consists primarily of people who didn't have a long prior history in Wikimedia projects, nor with each other: they had to grow as a team, understand our enormously complex community, while also delivering results. I want to use this opportunity to explicitly thank Brion, who played a key role in the orientation of the team.
The UX team has implemented a number of important practices:
* very frequent sharing of detailed work product, including mock-ups, specs, and research results such as user test videos. * systematic QA using an external QA firm, browser compatibility matrix, etc.; beginnings of automated testing * qualitative remote and in-person user testing to assess real-world experience with the site * moving towards data-driven decision-making using tools such as click-tracking, systematic assessment of feedback data, various statistics, etc. * public and frequent blog updates both in the regular and in the technical blog, use of the sitenotice to announce deployments * public sandboxes demonstrating all bleeding edge improvements early * various community-oriented discussion pages and documentation on usability.wikimedia.org informing and supporting design and localization * largest systematic feedback collection about any software deployment in the history of Wikimedia projects
Many of these were "firsts" for the Wikimedia Foundation, and they all were firsts on this scale. Some of these practices have clearly contributed to letting the community participate _more_ in the overall initiative: the larger-scale outreach, the sharing of research findings (which led to some community-driven improvements), the carefully prepared public testbeds, etc. Some, on the other hand, have arguably widened the staff and volunteer gap, both between staff developers and volunteer developers, and between staff and larger volunteer community.
I don't think Wikimedia's development processes are anywhere near those of proprietary web companies, but it's absolutely true that they're also not highly comparable to 100% volunteer-driven open source projects anymore. I think there are some parallels with both the strengths and weaknesses of open source projects with reasonably well-funded organizations behind them, both for-profit and non-profit.
I also want to be clear that there's no doubt in my mind that our ability to execute projects of this _type_ and at this _scale_ is unprecedented, and a historic achievement for the Wikimedia movement. MonoBook was implemented in 2004, and the basics of the user interface and user experience of Wikipedia have changed very little between then and pre-Vector.
The reasons for that are varied. Fundamentally, I believe that online volunteer mass collaboration works best when incrementalism and self-selection can, over a long period of time, add to an overall product that's useful at any given time -- such as Wikipedia or Wikimedia Commons. We've seen this work less well where we need to collaborate in a short period of time to achieve a result of predictably high quality (Wikinews), or where we're working over long periods of time on a result with (perceived or actual) limited usefulness until it is complete (Wikibooks).
I'm not saying this is inherent in volunteer mass collaboration, or that these challenges are insurmountable, just that a) tools and practices of volunteer collaboration tend to support self-selected long-term incrementalism better than, shall we say, strategic megaprojects, b) we're not necessarily even using all the tools and practices yet that other volunteer projects successfully employ to tackle strategic challenges.
2) Widening gap between staff and volunteer contributions:
As the Wikimedia Foundation began to professionalize, it initially moved its focus away from some of the early experiments in volunteer mobilization (such as the 2006 Wikimedia Foundation committees), and moved instead towards employing and tasking WMF staff for each critical work area. Much of 2008 was spent for this new team to get its feet on the ground and get oriented in the world of Wikimedia, to the detriment of developing new structures for volunteer engagement.
The strategic planning process launched in 2009 was, from the beginning, conceptualized to mobilize the Wikimedia community in collecting ideas and planning its own future, and has achieved unprecedented volunteer participation in doing so. We're now shifting towards using the StrategyWiki to help volunteers in efforts to self-organize around particular issues, whether the WMF is giving attention to them or not -- we'll be posting more about that soon, and I'm pretty excited about it.
At the same time, when tackling large scale strategic projects like the usability initiative or the bookshelf, I do believe we haven't yet succeeded at forming the most productive collaborative relationship possible with the larger volunteer community. I should note that we haven't even fully succeeded at closing the gap between staff and contractors based in San Francisco, and those who are not -- we have a long way to go.
The staff/volunteer gap is evident when looking at the commit history of the UsabilityInitiative codebase, for example, which is very much dominated by WMF staff and contractors. (The notable exception, as for all our code, is localization -- again a great example for self-selected incrementalism.) It's also evident in the recent conflict between a staff and a volunteer developer regarding the language link issue, and in our overall conversation about that same issue.
3) Closing the gap and overcoming cognitive biases:
I do think it's possible and necessary to close that gap. I think doing so has to _start_ with discussion, but ultimately to me this isn't so much about conversation, but about actual real-world collaboration with practical results -- we want to get stuff done together, and we're all working towards the same vision.
To do so, I think we have to be wary of our respective cognitive biases and operate with the highest possible degree of self-awareness. I find the framework of cognitive bias personally very helpful when I assess my own actions, and the list on Wikipedia ( http://en.wikipedia.org/wiki/List_of_cognitive_biases ) is a good starting point. I believe we've seen some examples of biases listed there in both WMF staff and volunteer actions, such as:
* belief that the group you're a member of is diverse, whereas other groups are not ("the staff does/is X", "the community doesn't have the expertise to do Y") ; * preferential treatment of people perceived to be members of the same group, and the associated us vs. them dynamic; * irrational escalation (tendency to defend an investment based on cumulative prior commitment) and the bandwagon effect (tendency to join the belief of others).
These and other biases contribute to unhealthy spirals of escalation. There's a big picture of clear alignment of values and interests: we've all committed ourselves to a very important mission, and we have seen time and again that we can work together immensely successfully in doing so. Fundamentally, I think it's the group division that's the most important to overcome -- inside the Wikimedia movement, but also towards readers and "outsiders." WMF absolutely has been both part of the problem, and part of efforts to solve it.
I entirely agree that we need to steer clear of unhealthy power dynamics. Wikipedia editing is not a tyranny of the majority or the minority, and nor is our engineering and design process -- ideally, the best arguments, the best code and the best edits should prevail. I'm glad that the BOLD, revert, discuss cycle -- http://en.wikipedia.org/wiki/Wikipedia:BOLD,_revert,_discuss_cycle -- was invoked in prior threads, which I think is a generally sensible framework through which we get stuff done. It's not the only framework, but it's one that we (should) stick to wherever reasonably possible.
To the extent that there is an irrational status quo bias, I think the best way to overcome it is for us to engage the largest possible number of people in bringing about and advocating on behalf of positive changes.
With all this in mind, here are just a few concrete ideas for closing the gap:
1) Embedding teams funded by WMF into larger, publicly visible workgroups which include volunteers and which meet regularly e.g. via IRC; 1 a) Outreach to grow and strengthen those workgroups with the best skills present in the Wikimedia volunteer community; 2) Publication of mini-projects which we identify and which can be tackled by self-organized volunteers, with mentoring by experienced WMF staff and volunteers (happening a bit via GSoC, but not as much outside of it yet); 3) Making development roadmaps fully transparent and open to discussion, and sharing justifications for all key priorities; 4) Further iteration of tools and processes for rapid volunteer-driven bug reporting, cross-browser testing, and submission of simple patches; 5) Stimulating larger scale contests focusing on specific areas of interest; 6) More topic-focused meetings and sprints like the multimedia usability meeting in Paris. 7) Further experimentation with tools like IdeaTorrent for large-scale brainstorming and ranking purposes (we have a prototype running at http://prototype.wikimedia.org/en-idea/ideatorrent/ ).
The Wikimedia Foundation regards these as crucial questions, and we've had many conversations about them, both internally and publicly. I hope these conversations will continue to broaden. I personally intend to work especially on 3) in the list above, and want to help establish 1) as a normal practice in key WMF program activities. Howie has started a general brainstorming page here, where I've reposted the above:
http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas
I hope you'll join me in adding to that list, and that we continue to hold each other to the idea of building true and genuinely open forms of collaboration, which is one of the most important cornerstones of the Wikimedia movement.
All best, Erik