On Mon, Jun 7, 2010 at 11:52 AM, Eugene Eric Kim eekim@blueoxen.com wrote:
On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
I agree that this comparison is inaccurate, but I disagree that this was Sue's assumption. All she said was that vocal outcry on a mailing list should not be construed as community consensus. I hope that no one disagrees with this.
"All she said", no. She didn't state that. You're putting words in her mouth.
Perhaps I'm guilty of the same crime. But What Sue said was
"The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with."
I read that as contrasting the purposes of the UX team and the people commenting here. I am unable to determine any other reason for bringing these two statements together except for the purpose of drawing the comparison I suggested was being made, even with your proposed alternative.
We all communicate unclearly at times, — and I am more than willing to accept that I saw a comparison there which was not intended.
In the interest of good communication I hope that you will take the time to consider how I could have come to the understanding that I did. I'm sure many other people on this list had the same understanding.
What we have here is a nearly unanimous response with respect to the disposition of the interwiki links. If you'd like me to bring this to the larger community I can do so but my understanding was that the normal community process was already quashed with respect to this change.
While no single forum is indicative of a consensus of the entire community, the broadness of the response here is a strong indicator.
[snip]
The bad thing is the us versus them tone in this and other messages. There is a larger question about ownership and decision-making that is subtle and hard, and we need to continue to work these out. Since I'm going to put myself in the vocal minority and disagree with most of the points in this message, I'll start with what I agree with. :-)
I don't believe that there is anything particularly subtle here. We have many community processes in which foundation staff are welcome to contribute to as peers with a common interest.
When you fail to do so you have created the "us" vs "them" by your own actions.
Rather then trying to draw "us" vs "them" lines in the sand, I am in fact pleading that the foundation discontinue doing so.
In order to do that I must first acknowledge the division which I believe has already formed. [more on this later]
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Absolutely. Assume good faith. I know you and many others feel the same way about the UX team
Absolutely.
I would suggest that the broader community (and not necessarily the participants here) has greater experience than the usability team, and even the portion of the community represented here has a more diverse composition than the UX team.
However, if we were to combine the two— we would have something strictly superior to the component parts. Unfortunately, we're still able to speak about the community and the UX teams as distinct entities. This division will continue so long as the relationship is viewed in the context of "decision"/"feedback" rather than as a dialogue between peers.
That said, keep in mind that most people assume that everyone thinks like they do. This is not an us versus them thing; this is a natural, human thing. I heard a great tip from a psychologist once: If you want to know what people truly think, ask them what they think other people think.
[snip]
While we are on the topic of lessons in human nature, please allow me to introduce the list to the fundamental attribution error: http://en.wikipedia.org/wiki/Fundamental_attribution_error
We all may find it informative.
Given this quirk of human nature, we need more rigorous ways of making decisions than polling people, especially small, self-selecting groups. Being data-driven is one of those ways. But being data-driven is hard, too, because you still have to interpret the data. Which brings me to...
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on.
Agreed. 1% is absolutely a large number of clicks, especially given our overall traffic. Someone in a different message also pointed out that click rates alone don't tell the whole story; if they did, then one could argue that we should eliminate the Edit button.
Good design isn't just about following the user path; it's also about guiding the users in a way that's appropriate to the mission of the work. In that vein, I think the substance of most of the messages in this thread have been very positive. It's resulted in data (such as Max's numbers) that have helped to round out everyone's understanding of the issue, but most importantly, it's resulted in critical context as to why people may be acting a certain way and why these things matter in the first place.
I do not believe that I am understanding the point you are making here.
The group collected on this list appears to generally hold the view that we the interwiki links are something we should be emphasizing beyond the level justified by their usage.
You agree that 1% is a rather large amount of usage, as Tim pointed out— it would compare favourably to the edit feature.
From these facts that we appear to agree on, if we were design from
the perspective of promoting values we would continue to keep the links prominent, and if we were to be data-driven about usage we would also keep the links prominent.
I'm not seeing anything in your above statement to suggest why would decide not to make the links prominent.
If you're not — then why is the discussion continuing? "The usability team made a mistake, thanks are owed to X,Y,Z for catching an fixing it. Sorry we stood in the way of the fixes and implied that you couldn't, that won't happen again." Would be a fine conclusion.
'We will now factor in your critical context and emit new fully formed sausage soon' would not be.
[snip]
I mostly disagree with this. First, let me refer back to Sue's original point. The people on this list are not representative of the community of contributors. It is a self-selecting sample. That doesn't invalidate the substance of what's said on this list, but it does raise questions about claims of consensus when there seems to be a large, vocal group of people agreeing on a point. Passionate discourse on this list is a data point, but it's only one data point, and it needs to be interpreted in the context of many data points.
My prior offer to bring in a larger part of the community still stands, if you really believe that to be an issue here.
In particular, the list of people agreeing on this point includes a number of parties which seldom agree with each other. I have found that to usually be an indication of an issue which will have a fairly broad support in the larger community.
I didn't think bringing on a hundred person pile-on would help— if anything I could see it only increasing distrust of the foundation in the larger community. It would leave me in a position of having to continue to damn the foundation's actions on this and other recent issues when I'd rather just move forward...
Second, most people -- both staff and people on this list -- _are_ disconnected from the needs of the majority of the readers. None of us represent the average reader, and so we need other data points to truly understand what the average reader experiences. And frankly, only a few people in this thread seem to acknowledge this.
The very first message on this subject was requesting additional data.
I think we'd all be interested in seeing more data on the subject.
Though I don't think it's valuable to define the meaning of the data only after you've seen it. That just leads to circular justification, the criteria needs to come first.
Nor do I think we need more data in order to conclude that the promotion of the interwiki links is an important _value_ for us, and that we wish to promote it above and beyond the level justified on a pure ease of use basis.
Accordingly, I think the data which would be most helpful would be indicators that were suggesting that the existence of the exposed interwikis creates a non-trivial harm to the usability of the site, since that is the kind of evidence which would overrule the value based decision to promote the links.
That kind of outcome would be fairly surprising to me, and I think everyone else here as no one has been suggesting that such an outcome is expected.
You're essentially claiming that certain staff members are copping a "we know better" attitude. That may be true (although I honestly don't see it in this thread), and if it's happening, it's wrong. That said, people on this list are copping the exact same attitude. Let's not draw generalities about us versus them based on a few individuals. Let's instead find smarter ways to make decisions that will help us fulfill our mission. You have been one of the best at doing this, and we need more of it.
I think that drawing an equivalence between comments by the staff and community here is an error.
I expect all of us, staff and community alike, to read the comments of others, consult their best available information, and form reasoned viewpoints which they then present and defend vigorously but politely. ... and that as more information and more viewpoints become available that people will have open minds to revising their positions. This is how discussion works.
Nothing about presenting a strong argument should be construed as a fault-worthy "we know better". Even in my strongest statements I am willing to believe that I could be wrong, and I know the same to be true of many others here— It would be hard to sanely hold any other position in the presence of so many experienced and intelligent people. I apologise for sometimes allowing it to sound otherwise.
But this discussion process only works when the participants are peers.
In this case, in spite of a mostly lacking defence and the obvious super-majority holding the counter position, the site continued to display the collapsed interwikis for a prolonged duration.
It is not vigorously defended positions which I am characterizing as "we know better", it is the obvious _absence_ of those positions combined with the reality of the website (at the time I wrote the message). And this characterization is further strengthened by the use of language like "feedback" and the constant reminders of non-representativeness as a fault of the assembled community when the same could equally be applied to the UX team and the staff as a whole.
And once we have been divided into groups, those with the authority to impose without broad agreement or even solid justification, and those without... then I must answer that implicit statement of superior knowledge with the correction: Of the two, it is the community which holds the stronger claim.
What's the success rate for community-run web sites? Drawing a company vs community-run comparison like this isn't just wrong, it's pointless. The level of success of all of the top web sites is a fluke to some extent. There are great, great community sites -- both grassroots and company-run -- that will never reach the scale of Wikimedia or Google.
The success rate for Wikipedia is one for one. It exists, it is successful.
We have a successful modality... but it is one which many people would be surprised that it works at all.
Some people might think it reasonable to try to direct things towards a more traditional mode of operation— with a reasonable eye towards fixing the things we do poorly, but that would ignore the enormous number of things our mode of operation does shockingly well such as existing at all.
So, my points was that traditional organizations are fantastically unsuccessful at accomplishing what we have accomplished and so there is no reason to believe a change would be an improvement and many reasons to believe it would be an enormous failure.
We're facing huge challenges right now. We need everyone to help. The Foundation staff understands this. Most people here understand this. The question is, what's the best way to grapple these challenges?
For starters, we need to come to a better, collective understanding of what it means to come to consensus.
You sound as though you believe that to be a new issue, but it isn't. I see no reason to conclude that the challenge we face now are materially different from the ones community has been grappling with for many years... which has resulted in a multitude of more or less workable compromises.
It's not reasonable to suggest that every single design decision first be tested on foundation-l,
This isn't how our communities usually work in any case.
Bold. Revert. Discuss. is a common modality.
I think that the group assembled here would largely agree that it would have been acceptable for the UX team to make the change— even with little to no public discussion, then not interfere with the community to reverting it when non-trivial objections were raised, then engage in a discussion about the ultimate disposition of the feature.
This pattern allows a significant majority of changes to happen without significant conflict and without the impediment of excessive discussion.
For more critical changes a better process is
Discuss Poll Discuss [ Bold. Revert. Discuss. ] [Bold. Discuss. Revise.] Agree.
Iterating the sections in brackets. We've used processes like this before for other user interface changes.
Through this process we've managed to make non-trivial user interface changes without leaving popular clients out in the cold for weeks at a time.
Sometimes an inclusive process simply takes more time. But the only deadlines we're facing are the ones we impose on ourselves. We can afford some things being slower so long as progress is made, it's a reasonable price to pay.
nor would that lead to the best decisions, even if it were reasonable.
Best is the often enemy of good.
The decisions we make are so highly dimensional that the prospect of ever finding a "best" decision is a laughable joke.
Even if we agreed on a definition of best-ness the chances of actually finding the best solution to anything is infinitesimal. Usually a definition of bestness can't be found because everyone weighs different values differently.
Moreover, a locally best solution isn't desirable. I hope and expect Wikipedia to be around 100 years from now. I hope that the decisions we make today are the ones with the greatest return over that time horizon (or longer). Since no one can know what the future will ask of us, this is just more evidence that a best decision is not possible no matter who makes it.
I fully admit to being clueless. I object to people claiming that have a best solution when no one can.
I think what we generally strive for is "the best long term outcome which doesn't suck too bad in any particular way in the short term".
In any case, Wikipedia is not a good place for people who do not see the value in the inclusive decision and who are too concerned about getting what they believe to be the best result.
In terms of content editing, people who are very concerned with saying the "right" thing are usually the ones who end up blocked.
Developers (including non-staff) need permission to Be Bold. We all
We agree.
need to have permission to try things, to make mistakes, and to learn from them. If you want to draw any lessons from the top web sites on how to grow a successful site, that would be the most important one.
But being bold necessarily implies the entire community process.
If a change is imposed without the rest of the process, that's not "Being bold", thats just simply a forced change.
Sometimes forced changes are reasonable- for example, a change to keep the site operating or a reversion of a user interface change that was blocking access to a large swath of users, but don't confuse an out of process emergency action with the bold actions of the normal process.
It's not clear to me that Trevor should have reverted Aryeh's change. But isn't it remarkable that Aryeh had the power to revert in the first place?
No. This is how we work here, and it's generally how we've worked here for a almost a decade. For many of our contributors Wikipedia and this governance has existed their entire adult life, and it is only way of operating a large website that they've ever participated in.
I think it's remarkable that we've given people the authority to make changes who still find our process surprising or remarkable.
It's only remarkable to people who don't appreciate the historical context... those who view the situation as an organization being gracious enough to let its "users" modify important parts of the site.
Allow me an alternative: what we have is a community of people who have contributed millions of man hours of work towards a common goal, and in the furtherance of that goal we created an office with a staff to handle routine administrative work and the interoperation with external entities (as it's hard to deal with the borg-of-wikipedia without becoming one). We were able to do this because the public supports our goals and the efforts of the community were successful at meeting an important need of the public.
The remarkable thing is the payment you receive. Not the ability of the community to make changes to the site.
... and please don't take that wrong: I am _overjoyed_ that the Foundation has been able to employ so many thoughtful and productive people, but I think it is amazing that it is able to do so on the thin slice of the value that our collective project has produced which comes back in the form of donations.
And that the end result was that Roan re-instated the revert? If the Foundation were on some power trip, would that have been possible?
Power trip isn't something that I said, and I don't think it accurately expresses what I would have alleged.
I think there is a misunderstanding about authority here, not a power-trip. The foundation was never created to "govern" the community. It was created to keep the machines well oiled on the communities behalf, sometimes oiling the machine requires a the execution of authority, sometimes you have to take the elevator out of service in order to improve it, but usually not and usually not much. Without the communities' continued governance the sites could not run at all. The WMF staff is far too small to even dream of replacing the communities role in governance.
None the less, you need to look no further than the facebook privacy policy reversal a year ago for a counter example. Controlling and commanding organizations routinely respond to public outcry.
That it took many days to get that far is a perfect example of the foundation staff not acting as an equal peer with the community.
We have this beautiful opportunity to learn how to do good user experience collectively and at scale. No one has figured out how to do this yet, but everyone is trying. There are going to be bumps along the way. We have to continue to Assume Good Faith and encourage each other to Be Bold.
So, you're suggesting that Wikipedia became a top 10 website, reliably outperforming commercial clones like answers.com that had professional development staffs, all our content, and then some, with an acutely _bad_ user experience?
No?
I think we _had_ a beautiful opportunity to take an already effective and innovative development system and take it to the next level, but instead we tried to supplant it with the same cathedral development model plus 'feedback' which is used by most commercial web-properties.
It wasn't clear to me that what was happening since a lot of community development appears almost fully formed from a single creator or team, the distinction only became clear when the response to concerns was "accepting feedback" rather than cooperation.
The fact that my blackberry _still_ can't load Wikipedia after weeks is clear evidence that a change in how things are run has happened.
I took the time to respond to this thread, and to your message, because I'd like to clear up the confusion and restore the opportunity for cooperation as peers while it is still early enough for such a mild remedy.
Somehow, the community knows how to take the ragtag assembly of its
[snip]
with broad appeal and generally consistent, if somewhat strange, performance.
In some areas yes, in others, no. We all still have a lot to learn, and we need to learn this together.
Absolutely, the community needs to learn how to address all areas. We've been learning how to address all areas for a long time. One thing we learned a long time ago is that asserting hard-authority over the community generally generates more heat than light, that cooperation as equals is usually more productive, and that other means should only be invoked as a last resort.
This isn't the same as accepting the notion of a staff that learns how to accept feedback, and a community that needs to learn to only whine and hope for change instead of taking action. ... the impression that I and others have received here.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_.
I don't see evidence of that here.
[snip]
I agree with this. Again, I don't see evidence of exclusion happening right now. I think the discourse has been positive, I think there are still some things that need to be worked out, and I think the right thing will happen in the end.
I think your inability to see what is obvious to me and many others is an example of the problem.
I'm confident that the right thing will happen in the end, my only concern is how much discomfort is on the road from here to there.
I hope my understanding of the motivations, attitudes, and plans of those involved are incorrect. But these understandings are driven by the actions of the foundation staff, they are positions I'd take with respect to anyone engaging in the same actions. They can be most easily remedied by the staff collaborating with the community as an equal parter, and not mealy as a source of optional feedback. Ten thousand words could not carry the impact of just a few actions.