Hello all, the massive thread regarding the default sidebar language link expansion state has surfaced a number of fundamental and significant questions regarding the working relationship between the Wikimedia Foundation and the larger Wikimedia volunteer community. This message represents my snapshot-in-time thinking about these questions, and I hope others will join this thread to meaningfully advance this IMO very important conversation. Tl;dr version: I believe that the transparency of Wikimedia's engineering processes, and the general quality of these processes, has significantly improved over the last year. At the same time, I agree with those who are observing a widening gap between staff and volunteer contributions, and I think we need to work together to close this gap in full awareness of the cognitive biases present in all of us. 1) Increase in quality and transparency of our engineering processes: I'm very grateful for the hard work accomplished by the User Experience Team to-date. When we started the usability project, we had never run a discrete engineering effort of comparable scale before, and indeed, most of our larger-scale engineering projects tended to be drowned by day-to-day emergencies and bug fixes. The team consists primarily of people who didn't have a long prior history in Wikimedia projects, nor with each other: they had to grow as a team, understand our enormously complex community, while also delivering results. I want to use this opportunity to explicitly thank Brion, who played a key role in the orientation of the team. The UX team has implemented a number of important practices: * very frequent sharing of detailed work product, including mock-ups, specs, and research results such as user test videos. * systematic QA using an external QA firm, browser compatibility matrix, etc.; beginnings of automated testing * qualitative remote and in-person user testing to assess real-world experience with the site * moving towards data-driven decision-making using tools such as click-tracking, systematic assessment of feedback data, various statistics, etc. * public and frequent blog updates both in the regular and in the technical blog, use of the sitenotice to announce deployments * public sandboxes demonstrating all bleeding edge improvements early * various community-oriented discussion pages and documentation on usability.wikimedia.org informing and supporting design and localization * largest systematic feedback collection about any software deployment in the history of Wikimedia projects Many of these were "firsts" for the Wikimedia Foundation, and they all were firsts on this scale. Some of these practices have clearly contributed to letting the community participate _more_ in the overall initiative: the larger-scale outreach, the sharing of research findings (which led to some community-driven improvements), the carefully prepared public testbeds, etc. Some, on the other hand, have arguably widened the staff and volunteer gap, both between staff developers and volunteer developers, and between staff and larger volunteer community. I don't think Wikimedia's development processes are anywhere near those of proprietary web companies, but it's absolutely true that they're also not highly comparable to 100% volunteer-driven open source projects anymore. I think there are some parallels with both the strengths and weaknesses of open source projects with reasonably well-funded organizations behind them, both for-profit and non-profit. I also want to be clear that there's no doubt in my mind that our ability to execute projects of this _type_ and at this _scale_ is unprecedented, and a historic achievement for the Wikimedia movement. MonoBook was implemented in 2004, and the basics of the user interface and user experience of Wikipedia have changed very little between then and pre-Vector. The reasons for that are varied. Fundamentally, I believe that online volunteer mass collaboration works best when incrementalism and self-selection can, over a long period of time, add to an overall product that's useful at any given time -- such as Wikipedia or Wikimedia Commons. We've seen this work less well where we need to collaborate in a short period of time to achieve a result of predictably high quality (Wikinews), or where we're working over long periods of time on a result with (perceived or actual) limited usefulness until it is complete (Wikibooks). I'm not saying this is inherent in volunteer mass collaboration, or that these challenges are insurmountable, just that a) tools and practices of volunteer collaboration tend to support self-selected long-term incrementalism better than, shall we say, strategic megaprojects, b) we're not necessarily even using all the tools and practices yet that other volunteer projects successfully employ to tackle strategic challenges. 2) Widening gap between staff and volunteer contributions: As the Wikimedia Foundation began to professionalize, it initially moved its focus away from some of the early experiments in volunteer mobilization (such as the 2006 Wikimedia Foundation committees), and moved instead towards employing and tasking WMF staff for each critical work area. Much of 2008 was spent for this new team to get its feet on the ground and get oriented in the world of Wikimedia, to the detriment of developing new structures for volunteer engagement. The strategic planning process launched in 2009 was, from the beginning, conceptualized to mobilize the Wikimedia community in collecting ideas and planning its own future, and has achieved unprecedented volunteer participation in doing so. We're now shifting towards using the StrategyWiki to help volunteers in efforts to self-organize around particular issues, whether the WMF is giving attention to them or not -- we'll be posting more about that soon, and I'm pretty excited about it. At the same time, when tackling large scale strategic projects like the usability initiative or the bookshelf, I do believe we haven't yet succeeded at forming the most productive collaborative relationship possible with the larger volunteer community. I should note that we haven't even fully succeeded at closing the gap between staff and contractors based in San Francisco, and those who are not -- we have a long way to go. The staff/volunteer gap is evident when looking at the commit history of the UsabilityInitiative codebase, for example, which is very much dominated by WMF staff and contractors. (The notable exception, as for all our code, is localization -- again a great example for self-selected incrementalism.) It's also evident in the recent conflict between a staff and a volunteer developer regarding the language link issue, and in our overall conversation about that same issue. 3) Closing the gap and overcoming cognitive biases: I do think it's possible and necessary to close that gap. I think doing so has to _start_ with discussion, but ultimately to me this isn't so much about conversation, but about actual real-world collaboration with practical results -- we want to get stuff done together, and we're all working towards the same vision. To do so, I think we have to be wary of our respective cognitive biases and operate with the highest possible degree of self-awareness. I find the framework of cognitive bias personally very helpful when I assess my own actions, and the list on Wikipedia ( http://en.wikipedia.org/wiki/List_of_cognitive_biases ) is a good starting point. I believe we've seen some examples of biases listed there in both WMF staff and volunteer actions, such as: * belief that the group you're a member of is diverse, whereas other groups are not ("the staff does/is X", "the community doesn't have the expertise to do Y") ; * preferential treatment of people perceived to be members of the same group, and the associated us vs. them dynamic; * irrational escalation (tendency to defend an investment based on cumulative prior commitment) and the bandwagon effect (tendency to join the belief of others). These and other biases contribute to unhealthy spirals of escalation. There's a big picture of clear alignment of values and interests: we've all committed ourselves to a very important mission, and we have seen time and again that we can work together immensely successfully in doing so. Fundamentally, I think it's the group division that's the most important to overcome -- inside the Wikimedia movement, but also towards readers and "outsiders." WMF absolutely has been both part of the problem, and part of efforts to solve it. I entirely agree that we need to steer clear of unhealthy power dynamics. Wikipedia editing is not a tyranny of the majority or the minority, and nor is our engineering and design process -- ideally, the best arguments, the best code and the best edits should prevail. I'm glad that the BOLD, revert, discuss cycle -- http://en.wikipedia.org/wiki/Wikipedia:BOLD,_revert,_discuss_cycle -- was invoked in prior threads, which I think is a generally sensible framework through which we get stuff done. It's not the only framework, but it's one that we (should) stick to wherever reasonably possible. To the extent that there is an irrational status quo bias, I think the best way to overcome it is for us to engage the largest possible number of people in bringing about and advocating on behalf of positive changes. With all this in mind, here are just a few concrete ideas for closing the gap: 1) Embedding teams funded by WMF into larger, publicly visible workgroups which include volunteers and which meet regularly e.g. via IRC; 1 a) Outreach to grow and strengthen those workgroups with the best skills present in the Wikimedia volunteer community; 2) Publication of mini-projects which we identify and which can be tackled by self-organized volunteers, with mentoring by experienced WMF staff and volunteers (happening a bit via GSoC, but not as much outside of it yet); 3) Making development roadmaps fully transparent and open to discussion, and sharing justifications for all key priorities; 4) Further iteration of tools and processes for rapid volunteer-driven bug reporting, cross-browser testing, and submission of simple patches; 5) Stimulating larger scale contests focusing on specific areas of interest; 6) More topic-focused meetings and sprints like the multimedia usability meeting in Paris. 7) Further experimentation with tools like IdeaTorrent for large-scale brainstorming and ranking purposes (we have a prototype running at http://prototype.wikimedia.org/en-idea/ideatorrent/ ). The Wikimedia Foundation regards these as crucial questions, and we've had many conversations about them, both internally and publicly. I hope these conversations will continue to broaden. I personally intend to work especially on 3) in the list above, and want to help establish 1) as a normal practice in key WMF program activities. Howie has started a general brainstorming page here, where I've reposted the above: http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas I hope you'll join me in adding to that list, and that we continue to hold each other to the idea of building true and genuinely open forms of collaboration, which is one of the most important cornerstones of the Wikimedia movement. All best, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate _______________________________________________ foundation-l mailing list foundation-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

The basic attitude has to be that paid developers are treated identically to volunteers, except that you can tell the former what to do and expect them to put in more time.  There should not be communication between paid developers and the community, paid developers should be an integral *part* of the community rather than a separate group of people.

The basic attitude has to be that paid developers are treated identically to volunteers, except that you can tell the former what to do and expect them to put in more time.  There should not be communication between paid developers and the community, paid developers should be an integral *part* of the community rather than a separate group of people.

On Tue, Jun 8, 2010 at 3:31 PM, Erik Moeller <erik(a)wikimedia.org> wrote: I was super excited to see this go up the other day. Can we move it to, say, strategywiki to try out? Any thoughts on scope? Not having used this much I don't know if you'd want separate torrents by broad topic (ideas for the wikimedia foundation, ideas for mediawiki, ideas for the projects, ideas for outreach) or one big one that could be sorted by topic.

I really agree with this sentiment, but it seems difficult to get staff to really be part of the community unless they're _from_ the community.  The developers I've seen discuss their personal opinions on public fora (especially in ways that are accepted in an open community but not in a business environment—one example would be criticizing their co-workers) have been those who were recruited from the community.  There's nothing wrong with having outsiders as employees, but communication is rather different in the outside world, and I get the sense that a lot of the people hired from elsewhere aren't necessarily familiar with the Wikimedia Way™ of discussing things—and even if they understand that it's there, I'm not sure they always understand that they're supposed to join in.

On Tue, Jun 8, 2010 at 3:31 PM, Erik Moeller &lt;erik(a)wikimedia.org&gt; wrote: <a long, thoughtful, and really important post> I was super excited to see this go up the other day. Can we move it to, say, strategywiki to try out? Any thoughts on scope? Not having used this much I don't know if you'd want separate torrents by broad topic (ideas for the wikimedia foundation, ideas for mediawiki, ideas for the projects, ideas for outreach) or one big one that could be sorted by topic. -- phoebe _______________________________________________ foundation-l mailing list foundation-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

2) Make sure that every paid developer spends time dealing with the community.  This can include giving support to end users, discussing things with volunteers, reviewing patches, etc.  They should be doing this on paid time, and they should be discussing their personal opinions without consulting with anyone else (i.e., not summarizing official positions).  Paid developers and volunteers have to get to know each other and have to be able to discuss MediaWiki together.

It's not specific to Wikimedia, it's practically universal in open-source development. To get it to happen, you need pushing from the top: formally stating it as part of people's job duties (so they don't feel they have to do "real work" instead), and forcing them to engage by only giving them public media to discuss things in with their co-workers. I recall reading that IBM improved its participation in the Linux kernel community by getting rid of all internal communications among its kernel developers, meaning they had to use the public project lists to bounce ideas off anyone. It's also worth pointing out that a good way *not* to engage with the community is to not touch preexisting code that volunteers are familiar with. All the Usability Initiative stuff was created separately: a new skin, and all other functionality in extensions. There's mostly no technical reason for this; at least some could have been integrated with the existing code. Putting most of your code in a directory called "UsabilityInitiative" is a good way of signaling "this is ours, not yours", whether that was the intent or not. If it had touched code that volunteers were familiar with, they would have been more engaged from the start, because they'd have stronger opinions on the changes and no presumption that they shouldn't touch it. _______________________________________________ foundation-l mailing list foundation-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

So, I'll start chipping in my work at the page Erik has started: http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas

One undertone that I've witnessed everywhere is that people in open source communities that have a clear organizational "owner" is that there is a very uneven distribution of people who want a peer-to-peer relationship versus a customer-vendor relationship.  This makes it really difficult to work out in the public, because some people seem to prefer the trappings of a peer-to-peer relationship (let me in on your early thinking, publish your roadmaps, work in the fishbowl), where others prefer the trappings of the customer-vendor relationship (the customer is always right, the customer is the boss).  Some will even go so far as to want a customer-to-peer relationship, which is clearly not sustainable.  To be really clear here, most of my impressions on this topic come from my previous work experience (been doing the corporate open source thing for a while), and only in a limited way with this community, but I've seen hints that the WMF<=>community relationship has some of the same traits. From the vantage point of the "vendor" in this case, the problem is compounded by the cognitive bias Erik pointed to (belief that the group you're a member of is diverse, whereas other groups are not).  The net result of different expectations in the community is that, from the vendor point of viewer, it looks like the community is demanding a customer-to-peer relationship, since that is the "average" opinion of a pretty large and diverse group.  That's why I'm generally pretty careful about using the term "the community", because for those not used to working out in the open, it's really scary to get mixed up in public conversations. One thing to consider about the IBM example is that IBM is a company of about 400,000 employees, and was probably in the middle of their "we're spending $1 billion/year on Linux" year when they instituted that policy. They could probably stand to be a little inefficient in the name of insinuating themselves in the community.  We're not working with that sort of cushion. As someone who currently works from Seattle (and worked on a distributed team in my last job), I also know that long distance collaboration (even in the same timezone as SF) has its disadvantages from an efficiency perspective.  Most people have a strong preference for face-to-face communication for collaboration for good reason...it's high bandwidth.  Even people who are really good at doing it take some time to figure out how to be effective using only email and IRC; forcing people who aren't good at it is really a productivity hit. My recommendation is to strive to make it incredibly compelling for WMF staff to work out in the community.  That means adhering to WP:BITE and WP:GOODFAITH in spades, and reminding each other that we're all on the same team here.  It means making sure that it actually feels like it's increasing our productivity to do it, rather than feeling like a drag.  That's not to say the burden needs to be solely on you all, but I think "forcing" employees to work in the community is some customer-vendor thinking at play. Don't get me wrong: I think it's an incredibly good idea for us to figure out how to all work together better, and clearly a big part of that is going to be strengthening our working relationship with non-employees.  It wasn't that long ago I was a non-employee Wikipedian, and may be one again soon.  I share your goal.  We have an amazingly diverse community with (very importantly) a fantastic volunteer work ethic, and I think we should be able to figure this out.

As you know, any time you want to compel someone to do something, there's always the carrot and the stick.  One thing I don't like about the way you've phrased that is that is that you seem to be advocating the stick.  Am I reading that right?

One undertone that I've witnessed everywhere is that people in open source communities that have a clear organizational "owner" is that there is a very uneven distribution of people who want a peer-to-peer relationship versus a customer-vendor relationship.  This makes it really difficult to work out in the public, because some people seem to prefer the trappings of a peer-to-peer relationship (let me in on your early thinking, publish your roadmaps, work in the fishbowl), where others prefer the trappings of the customer-vendor relationship (the customer is always right, the customer is the boss).

One thing to consider about the IBM example is that IBM is a company of about 400,000 employees, and was probably in the middle of their "we're spending $1 billion/year on Linux" year when they instituted that policy.  They could probably stand to be a little inefficient in the name of insinuating themselves in the community.  We're not working with that sort of cushion.

As someone who currently works from Seattle (and worked on a distributed team in my last job), I also know that long distance collaboration (even in the same timezone as SF) has its disadvantages from an efficiency perspective.  Most people have a strong preference for face-to-face communication for collaboration for good reason...it's high bandwidth.  Even people who are really good at doing it take some time to figure out how to be effective using only email and IRC; forcing people who aren't good at it is really a productivity hit.

My recommendation is to strive to make it incredibly compelling for WMF staff to work out in the community.  That means adhering to WP:BITE and WP:GOODFAITH in spades, and reminding each other that we're all on the same team here.

That's not to say the burden needs to be solely on you all, but I think "forcing" employees to work in the community is some customer-vendor thinking at play.

Um, ok: http://meta.wikimedia.org/wiki/Product_Development_Process_Ideas

On Wed, Jun 9, 2010 at 12:55 AM, Aryeh Gregor &lt;Simetrical+wikilist(a)gmail.com&gt; wrote: I like the "discussing their personal opinions without consulting with anyone else" bit, and you bring up a very good point. I don't think (and I don't mean to imply that anyone else does) that anyone's conspiring to keep the community out, or saying "leave this to the professionals, we know better." When you're hired onto a team, though, you're wary of saying anything that would cause strife or confusion. This isn't necessarily out of fear of retribution from your employer—it's simply conventional professional ethics, and it's usually not even a conscious thing. (It's also not limited to paid staff—the people we put on the Board specifically for their vocal opinions on things often fall into this, for understandable reasons.)

...if for example I was qualified to review a staff member's patch (which I'm not), I might want to think twice about what audience gets that feedback. --Michael Snow

The replies to my comment are missing the point. Sure, the developers themselves need to be able to handle public criticism of their work, just like wiki editors. But I was responding to Austin's comment in particular about board members being cautious with their opinions. In cases like that, there are additional concerns, like the propriety in publicly critiquing someone's work when you also can presumably influence their continued employment. That requires that certain feedback go through other channels, even when the same feedback could be given openly if it were coming from the general public.

...if for example I was qualified to review a staff member's patch (which I'm not), I might want to think twice about what audience gets that feedback. --Michael Snow

Chad, I'm hesitant to reply to your note, because I feel like "defending the staff against the community" is a bad role for me: it tends to polarize and divide, rather than helping us all work together well.  And I think I do, for the most part, agree with you.

(As someone pointed out here the other day, Wikimedia recruits with that in mind: all our job postings specifically call out that people need to be comfortable in an open, collaborative environment, and we aim to only recruit people who can thrive in that context. We've learned the hard way to really probe on that in hiring interviews -- to pose open-ended scenario questions, and to use real-world examples. Practically everyone believes they are inclined to be collaborative, but that doesn't mean their definition is the same as ours.  And we've found, unsurprisingly, that people who are already members of the Wikimedia community are pretty much the only people guaranteed to be risk-free in that regard: to a certain extent, hiring outside the community always carries a certain amount of risk.  Which is fine and unavoidable: we do what we can to pick people we believe can succeed.) But I do want to make one small point that I think is sometimes missed. And that is, the staff can't take wikibreaks.  Volunteers are always free to take a break if they get irritated or discouraged or stressed: their contribution is voluntary, and they can walk away any time.  The staff can't. They need to come in every day and work hard, even on those (fortunately fairly rare) days when they are getting yelled at on mailing lists, when it might be harder than usual to feel motivated.

We try really hard to hire people who are personally resilient, and I think we've succeeded at that reasonably well. Personally though, I think harshness and offence are mostly avoidable, and I think we should avoid them whenever we reasonably can.  (Of course, I am female, and women are socialized to value harmony more than men.  It doesn't stick for us all, but it did stick a fair bit, for me.)  Personally, I think it's mostly possible to be frank without being rude, and I think it's worth trying to do that.  I'm not arguing that people should handle the staff with kid gloves: I would actually argue, and have argued, that an uptick in kindness would be good for everyone.  I realize that not everyone needs that, and it's obvious that not everyone will get it, whether they need it or not.  But I think it's a worthy goal :-)