On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
The interlanguage links can be easily unhidden by anyone who knows about them. The site remembers that you clicked to expand them. That memory is short, but it wouldn't take any real effort to override with personal settings... or people can disable Vector (which is what I've done, because Vector is slow, even though I like it a lot overall). In short, there is little reason for a sophisticated user to complain about this for their own benefit.
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Non-agreement on personal preferences is an entirely different matter than non-agreement about how to best help our readers and how to best express the values and principles behind the operation of our sites.
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on. To hear that it went down considerably with Vector—well, if nothing else, it is a possible objective indication that the change has reduced the usability of the site. It is absolutely clear evidence that this change has made a material impact on how we express ourselves to the world. I think it's clear from my earlier messages, before I knew the actual number, that I would have regarded figures like this as evidence of a clear mistake.
There is a clear attitude from the foundation staff that I, and others, are perceiving in these discussions. The notion that the community of contributors is a particularly whiny batch of customers who must be 'managed', that they express demands unconnected from the needs of the readers... and that it is more meaningful when a couple of office staff retreat to some meeting room and say "we reached a decision". Sadly, this attitude appears to be the worst from the former volunteers on the staff—they are not afraid to speak up in community discussion, and feel a need to distinguish themselves from all the volunteers.
This needs to stop and a point needs to be made clear:
This community is who made the sites. I don't just mean the articles. I mean the user interfaces, the PR statements, the fundraiser material, _everything_. The success rates for companies trying to build large and popular websites is miserable. Every successful one is a fluke, and all the successful ones have a staff and budget orders of magnitude larger than yours.
We have an existence proof that the community is able to manage the operation of the sites at a world class level. Certainly there are many things which could have been done better, more uniformly, more completely, or with better planning... but the community has a proven competence in virtually every area that the foundation is now attempting to be directly involved in. Not every member of the community, of course, but the aggregate.
Wikimedia's ability to do these things is an unknown, but the (lack of) successes of other closed companies running websites—even ones staffed by brilliant people—suggests that it is most likely that you will also be unsuccessful. I don't mean this as a comment on the competence of anyone involved (as I know many of them to be rather fantastic people), it's just the most likely outcome.
Imagine a resume for the community as a unit: * Expertise in every imaginable subject. * Simultaneous background in almost every human culture. * Speaks hundreds of languages. * Wrote the world's largest encyclopedia. * Built one of the world's most popular websites, from the ground up. * Managed to make an encyclopedia somehow interesting enough to be a popular website. * Managed the fundraising campaigns to support the entire operating cost of the above mentioned Top-N website on charitable contributions for many years. * On and on, etc.
(Like all resumes, this does not highlight the negatives--just proclaims what it's been able to accomplish in spite of them.)
Somehow, the community knows how to take the ragtag assembly of its members: the whining, the warped personal preferences, the inspired motivations of individuals and small groups, the collective voice of the uninformed, and a smattering of contributions from world class experts the likes of which we'd never be able to hire and retain, the good and the bad—and fuse it into something which can build output with broad appeal and generally consistent, if somewhat strange, performance.
I've personally been quick to dismiss people who wax philosophic about "the wisdom of crowds"... all of the great community work I've seen is mostly an effort from dedicated individuals and small groups, not some 'crowd'. And yet there clearly is something there, because the community delivers results superior to that of most other small groups and individuals. I guess the real power comes from that fact that every issue can be attacked by a custom small group from a nearly infinite set, plus a little crowd input. Whatever it is, it clearly works.
If Wikimedia itself can't learn how to either develop the same coalition-building skills, participate within the existing community process, or stand out of the way—we'll lose something great.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_. There are areas where the community's coverage is inadequate or inconsistent, and I think that dedicated staff acting as gap-fillers could greatly improve the results. But not if the price of those contributions is to exclude or pigeonhole the great work done by the broad community, either directly by "we reached a decision"-type edicts, or indirectly by removing the personal pride and responsibility that people feel for the complete site.
In this discussion we don't merely have personal preferences, we're arguing principles of design and hypothesizing benefit for the readers. And, excluding the foundation staff, we appear to have a broad, if not complete, consensus that the inter-language links should come back. In the community-operated model this would already be done by now.
I'm also left confused and wondering about one point I consider very important.
If the challenge is to "balance all readers' needs" why is the usability staff currently spending time arguing with the community about some silly sidebar links while the site is still _unviewable_ by a non-trivial portion of our readers (BlackBerry) as a result of the latest usability improvements? In the past the community resolvent these kinds of issues very rapidly, though sometimes by undoing the improvement.
On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
The interlanguage links can be easily unhidden by anyone who knows about them. The site remembers that you clicked to expand them. That memory is short, but it wouldn't take any real effort to override with personal settings... or people can disable Vector (which is what I've done, because Vector is slow, even though I like it a lot overall). In short, there is little reason for a sophisticated user to complain about this for their own benefit.
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Non-agreement on personal preferences is an entirely different matter than non-agreement about how to best help our readers and how to best express the values and principles behind the operation of our sites.
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on. To hear that it went down considerably with Vector—well, if nothing else, it is a possible objective indication that the change has reduced the usability of the site. It is absolutely clear evidence that this change has made a material impact on how we express ourselves to the world. I think it's clear from my earlier messages, before I knew the actual number, that I would have regarded figures like this as evidence of a clear mistake.
There is a clear attitude from the foundation staff that I, and others, are perceiving in these discussions. The notion that the community of contributors is a particularly whiny batch of customers who must be 'managed', that they express demands unconnected from the needs of the readers... and that it is more meaningful when a couple of office staff retreat to some meeting room and say "we reached a decision". Sadly, this attitude appears to be the worst from the former volunteers on the staff—they are not afraid to speak up in community discussion, and feel a need to distinguish themselves from all the volunteers.
This needs to stop and a point needs to be made clear:
This community is who made the sites. I don't just mean the articles. I mean the user interfaces, the PR statements, the fundraiser material, _everything_. The success rates for companies trying to build large and popular websites is miserable. Every successful one is a fluke, and all the successful ones have a staff and budget orders of magnitude larger than yours.
We have an existence proof that the community is able to manage the operation of the sites at a world class level. Certainly there are many things which could have been done better, more uniformly, more completely, or with better planning... but the community has a proven competence in virtually every area that the foundation is now attempting to be directly involved in. Not every member of the community, of course, but the aggregate.
Wikimedia's ability to do these things is an unknown, but the (lack of) successes of other closed companies running websites—even ones staffed by brilliant people—suggests that it is most likely that you will also be unsuccessful. I don't mean this as a comment on the competence of anyone involved (as I know many of them to be rather fantastic people), it's just the most likely outcome.
Imagine a resume for the community as a unit:
- Expertise in every imaginable subject.
- Simultaneous background in almost every human culture.
- Speaks hundreds of languages.
- Wrote the world's largest encyclopedia.
- Built one of the world's most popular websites, from the ground up.
- Managed to make an encyclopedia somehow interesting enough to be a
popular website.
- Managed the fundraising campaigns to support the entire operating
cost of the above mentioned Top-N website on charitable contributions for many years.
- On and on, etc.
(Like all resumes, this does not highlight the negatives--just proclaims what it's been able to accomplish in spite of them.)
Somehow, the community knows how to take the ragtag assembly of its members: the whining, the warped personal preferences, the inspired motivations of individuals and small groups, the collective voice of the uninformed, and a smattering of contributions from world class experts the likes of which we'd never be able to hire and retain, the good and the bad—and fuse it into something which can build output with broad appeal and generally consistent, if somewhat strange, performance.
I've personally been quick to dismiss people who wax philosophic about "the wisdom of crowds"... all of the great community work I've seen is mostly an effort from dedicated individuals and small groups, not some 'crowd'. And yet there clearly is something there, because the community delivers results superior to that of most other small groups and individuals. I guess the real power comes from that fact that every issue can be attacked by a custom small group from a nearly infinite set, plus a little crowd input. Whatever it is, it clearly works.
If Wikimedia itself can't learn how to either develop the same coalition-building skills, participate within the existing community process, or stand out of the way—we'll lose something great.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_. There are areas where the community's coverage is inadequate or inconsistent, and I think that dedicated staff acting as gap-fillers could greatly improve the results. But not if the price of those contributions is to exclude or pigeonhole the great work done by the broad community, either directly by "we reached a decision"-type edicts, or indirectly by removing the personal pride and responsibility that people feel for the complete site.
+ 1,000,000.
On a day to day level, the Wikimedia community has a challenge to solve: what are those gaps, how to fill them, and how to make those solutions part of the community process. And this is really the entire community's challenge, whether there's a specific Foundation involved or not.
-- phoebe
Gregory Maxwell wrote:
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on. To hear that it went down considerably with Vector—well, if nothing else, it is a possible objective indication that the change has reduced the usability of the site.
To clarify, no current statistics have been released. The "0.28%" figure refers to use of Vector before the official roll-out (when the interwiki links became collapsed by default, if I'm not mistaken). The disparity is attributable to the fact that most Vector users were participants in an opt-in, English-language beta test.
For the record, I agree with everything else that you wrote.
David Levy
On Sat, Jun 5, 2010 at 10:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
In short, there is little reason for a sophisticated user to complain about this for their own benefit.
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
I don't mean to detract from Greg's truly excellent e-mail by replying to just part of it, but I know that this is the case for me—I still use the Classic theme, restyled with my own CSS and Javascript, and all of the interwiki links are right where they were before. Vector doesn't affect me personally, but I see its impact on people around me all day.
For the love of all that is virtuous, please at least read everything this man says.
Austin
The foundation's programmers have the technical power to define the experience of all aspects of the site however they please. They cannot be prevented from having this power, but they nonetheless must not use it, except for the most mundane details of day to day maintenance. Their role is to carry out the wishes of the community to the extent it is feasible. They will obviously need to figure out how to accommodate different and conflicting wishes, but it is not up to them to establish the priorities.
This is true also of the specialists, such as the interface team: their role is to advise the community, not determine the results, and they should accept that their advice however excellent will nonetheless not always be followed. This is especially true for the specialists who do not have prior experience with WP, and can therefore not be expected to know the customs and way of thinking that prevails, and that sets the limits for what any individual can do by their own decision. Certainly they can be expected to learn it, but they must expect their understanding of it to be always corrected by the actual community.
For example, they seem to have operated on the assumption that 1% use of a feature, or the use of an uncommon platform, is something that can be ignored. This may be a common assumption in many settings, including some I am quite familiar with, but it is not in WP.
On Sat, Jun 5, 2010 at 5:10 PM, Austin Hair adhair@gmail.com wrote:
On Sat, Jun 5, 2010 at 10:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
In short, there is little reason for a sophisticated user to complain about this for their own benefit.
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
I don't mean to detract from Greg's truly excellent e-mail by replying to just part of it, but I know that this is the case for me—I still use the Classic theme, restyled with my own CSS and Javascript, and all of the interwiki links are right where they were before. Vector doesn't affect me personally, but I see its impact on people around me all day.
For the love of all that is virtuous, please at least read everything this man says.
Austin
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
On Sat, Jun 5, 2010 at 4:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
In this discussion we don't merely have personal preferences, we're arguing principles of design and hypothesizing benefit for the readers. And, excluding the foundation staff, we appear to have a broad, if not complete, consensus that the inter-language links should come back. In the community-operated model this would already be done by now.
I'd like to echo Phoebe's +1,000,000 first of all. I agree with everything you've said.
I'd like to touch on this one particular point. The community HAS spoken and clearly wants it back the way it was. A volunteer even did so [0] but was reverted [1] with the message that UI changes to Vector are off-limits without some sort of prior discussion and approval.
This sits with me _very_ badly. I don't disagree (in principle) that changes to our user experience should be discussed and not implemented via fiat. But when you've got overwhelming consensus that this is the right course of action, reverting the change and declaring it off-limits to our committers is just wrong. Our volunteer developers do a pretty good job of judging and implementing community consensus, and saying that some things aren't negotiable sets a bad precedence.
Of course I don't suggest we start a revert war in SVN over it, but I do think that Trevor's revert should be backed out and the full list restored until a better long term solution is thought out (per Erik's e-mail).
-Chad
[0] http://www.mediawiki.org/wiki/Special:Code/MediaWiki/67281 [1] http://www.mediawiki.org/wiki/Special:Code/MediaWiki/67299
Chad <innocentkiller@...> writes:
I'd like to touch on this one particular point. The community HAS spoken and clearly wants it back the way it was. A volunteer even did so [0] but was reverted [1] with the message that UI changes to Vector are off-limits without some sort of prior discussion and approval.
This sits with me _very_ badly. I don't disagree (in principle) that changes to our user experience should be discussed and not implemented via fiat. But when you've got overwhelming consensus that this is the right course of action, reverting the change and declaring it off-limits to our committers is just wrong. Our volunteer developers do a pretty good job of judging and implementing community consensus, and saying that some things aren't negotiable sets a bad precedence.
I completely agree with this. Although the people that made and executed this decision are my friends and coworkers, I increasingly feel the need to call them out on this particular action. We, the usability team, exist to improve the appearance and usability of the site, not to own or monopolize these topics. This revert, particularly the tone (and, to a lesser degree, the substance) of the revert summary, sends the message that we do in fact claim that monopoly; that any decision about usability goes through us; that "our" code is a sacred work that may only be touched with prior approval of a staff member, and that any mortal who dares violate these sacred commandments will experience the Wrath of the Immediate Revert.
There is no doubt in my mind that all members of the usability team, as well as other people involved with our work, will reject these notions instantly upon reading them. I am convinced that every single one of them has the genuine desire to work with the community in mutual respect rather than to impose their views upon them. However, they have failed to be cooperative, having appeared rather authoritarian in both their actions and their (mostly unconscious) messaging. I am certain this was not their intention, but that doesn't mean it wasn't inappropriate.
About the issue at hand: there seems to be an overwhelming consensus that the collapsing of the language links should be reverted, be it permanently or in anticipation of a different solution. The Foundation has been neglecting to do this for too long now. Unless someone stops me, I will reinstate Plationdes' revert and deploy it to the live site tomorrow morning (PDT).
Roan Kattouw (Catrope)
P.S.: Except for the last sentence, this post expresses my opinions as a community member, not as a contractor for the Foundation.
On Mon, Jun 7, 2010 at 6:23 AM, Roan Kattouw roan.kattouw@gmail.com wrote:
Chad <innocentkiller@...> writes:
I'd like to touch on this one particular point. The community HAS spoken and clearly wants it back the way it was. A volunteer even did so [0] but was reverted [1] with the message that UI changes to Vector are off-limits without some sort of prior discussion and approval.
This sits with me _very_ badly. I don't disagree (in principle) that changes to our user experience should be discussed and not implemented via fiat. But when you've got overwhelming consensus that this is the right course of action, reverting the change and declaring it off-limits to our committers is just wrong. Our volunteer developers do a pretty good job of judging and implementing community consensus, and saying that some things aren't negotiable sets a bad precedence.
I completely agree with this. Although the people that made and executed this decision are my friends and coworkers, I increasingly feel the need to call them out on this particular action. We, the usability team, exist to improve the appearance and usability of the site, not to own or monopolize these topics. This revert, particularly the tone (and, to a lesser degree, the substance) of the revert summary, sends the message that we do in fact claim that monopoly; that any decision about usability goes through us; that "our" code is a sacred work that may only be touched with prior approval of a staff member, and that any mortal who dares violate these sacred commandments will experience the Wrath of the Immediate Revert.
I will say to be fair that the best response to what you perceive as a poor design choice in somebody else's code is not to revert them and say "There, I fixed it for you. Thank me later.", but perhaps to discuss it with them first and find a compromise. There's an imperative to listen and respond to community feedback, but quietly changing somebody else's code against their explicit wishes is not a good way to make your point.
Andrew Garrett <agarrett@...> writes:
I will say to be fair that the best response to what you perceive as a poor design choice in somebody else's code is not to revert them and say "There, I fixed it for you. Thank me later.", but perhaps to discuss it with them first and find a compromise. There's an imperative to listen and respond to community feedback, but quietly changing somebody else's code against their explicit wishes is not a good way to make your point.
This is true. In that sense, I do feel that the revert itself was justified for the exact reasons you state, but that the message sent by the revert summary was harsh and authoritarian, as I said. Reverting that change with a gentler and more helpful summary, or even just leaving it in for some time while a compromise is being worked on (the latter should not be done as a rule, of course, to discourage the point-making by disruption you speak of), would've been a better course of action.
Roan Kattouw (Catrope)
Andrew Garrett wrote:
I will say to be fair that the best response to what you perceive as a poor design choice in somebody else's code is not to revert them and say "There, I fixed it for you. Thank me later.", but perhaps to discuss it with them first and find a compromise. There's an imperative to listen and respond to community feedback, but quietly changing somebody else's code against their explicit wishes is not a good way to make your point.
Have you looked at r67281? That was not a revert. Given the phrase "collapse all navs but the first" it changed it to "collapse all navs but the first or the interwiki one". That was a bug fix, you might even call it "fine tuning collapsiblenavs". And it was not *me* considering it a bug. It was backed up by the community. I wasn't trying to make a point, just trying to finally fix it and stop the mourning. I wasn't too successful :)
However, it wasn't against their explicit wishes, since they hadn't expressed their wishes. Had they wontfixed bug 23497, I wouldn't have done that. Or expressed that in the bug, or this thread... It's worth noting the lack of feedback from the team here. There were a couple of replies by Howie after the fact, but other than those, the only coding staff replies were from Roan and you, which incidentally come both from the community.
It was later revealed that Trevor had been on vacation for the last 2 weeks. That can partly explain the silence. I didn't know it. Although that could be taken a reason /for/ changing the code, too.
We are editing each other code all the time. Extensions are more individual than eg. core, but still a one-line patch shouldn't be an issue. It was even stated later: "Anyone is welcome to touch usability code", I think the problem was that it was that it was "against their wishes", which I should have somehow guessed from being non-responsive.
Greg,
This makes two home runs in one month -- you get a prize.
On Sat, Jun 5, 2010 at 4:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers.
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Yes. We are skilled at these trade-offs as a community (by which I mean, those who care to do the work that must be done -- build up all parts of the sites, tell the world about them, greet the folks who join in, plan for the needs of those who read, write, draw, film, code, tag, and share). We may not yet have a deep bench of UI gods, but we have much to be proud of, and care a lot about such things.
And we do not fear change -- we love new points of view. This list and those like it are some of the best groups I know of to vet and smooth such work once it is done.
a point needs to be made clear:
This community is who made the sites. I don't just mean the articles. I mean the user interfaces, the PR statements, the fundraiser material, _everything_. ...
Just so. And the community runs the sites each day, and sees to most change that hits the site. Case in point: the new skin is one part of UI work, and a big chunk rolled out all at once, but much more is done each week step by step, niche by niche, with no fuss.
We have an existence proof that the community is able to manage the operation of the sites at a world class level. Certainly there are many things which could have been done better, more uniformly, more completely, or with better planning... but the community has a proven competence in virtually every area that the foundation is now attempting to be directly involved in.
The foundation can serve as a sure core of work and a hub for large tasks, but it is small next to the community as a whole. More raw work still gets done (in press and grants and style-work and thoughts on how to reach new groups) through the community and chapters.
I guess the real power comes from that fact that every issue can be attacked by a custom small group from a nearly infinite set, plus a little crowd input. Whatever it is, it clearly works.
It works as long as those small groups feel they can/should dive in and claim that work as their own. This takes love, trust (for the skills new groups bring, and for their own lore and views and sense of the world), and the will to share (a call to share the joy of work on a big task, not just to "say one's piece" and move on).
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_.
It seems to be hard at times, and a cinch at times. We talk most about the times when it is hard -- as we should; they need the most work. But we can learn from both.
There are areas where the community's coverage is inadequate or inconsistent, and I think that dedicated staff acting as gap-fillers could greatly improve the results. But not if the price of those contributions is to exclude or pigeonhole the great work done by the broad community, either directly by "we reached a decision"-type edicts, or indirectly by removing the personal pride and responsibility that people feel for the complete site.
Right.
Erik writes:
I would suggest the following approach:
< 1) That we return to the default-expanded state for now... < 2) That we prototype the system above...
< by implementing 1), we can do 2) on a timeline that makes sense without a false urgency.
+1
SJ
-- Samuel Klein identi.ca:sj w:user:sj Kat: tag, you're it
On Sat, Jun 5, 2010 at 4:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
There is a clear attitude from the foundation staff that I, and others, are perceiving in these discussions. The notion that the community of contributors is a particularly whiny batch of customers who must be 'managed', that they express demands unconnected from the needs of the readers... and that it is more meaningful when a couple of office staff retreat to some meeting room and say "we reached a decision".
I agree entirely with this, and most of the rest of your post. Wikimedia has not yet figured out how to manage paid employees alongside a community. I've been meaning to write up some comments on this, and hopefully will get to that soonish. If important volunteer contributors raise complaints about a change, it's not acceptable to simply say "This is what we've decided". You need to seriously engage their arguments in detail, and provide the data used to make the decision. Yes, this costs employees time that they could be using to work on other things, but a strong community will repay the time invested tenfold.
Separately, I think that many (not all) of the people objecting to the change in this case presented weak arguments, as I detailed in previous posts. But this does not mean that they should be ignored, only refuted. (And *some* of the arguments against the change were not obviously incorrect.)
2010/6/6 Aryeh Gregor Simetrical+wikilist@gmail.com:
On Sat, Jun 5, 2010 at 4:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
There is a clear attitude from the foundation staff that I, and others, are perceiving in these discussions. The notion that the community of contributors is a particularly whiny batch of customers who must be 'managed', that they express demands unconnected from the needs of the readers... and that it is more meaningful when a couple of office staff retreat to some meeting room and say "we reached a decision".
I agree entirely with this, and most of the rest of your post. Wikimedia has not yet figured out how to manage paid employees alongside a community. I've been meaning to write up some comments on this, and hopefully will get to that soonish. If important volunteer contributors raise complaints about a change, it's not acceptable to simply say "This is what we've decided". You need to seriously engage their arguments in detail, and provide the data used to make the decision. Yes, this costs employees time that they could be using to work on other things, but a strong community will repay the time invested tenfold.
I meant to write something about this, too, but i feared that i would sound rude and touch sensitive issues :)
So yes, it's a major organizational problem. This affair should be remembered in the future when such significant technical decisions are made.
Separately, I think that many (not all) of the people objecting to the change in this case presented weak arguments, as I detailed in previous posts. But this does not mean that they should be ignored, only refuted. (And *some* of the arguments against the change were not obviously incorrect.)
... And you made very interesting and challenging counterarguments indeed.
I think that most of us can agree now that the decision to make the change was made by a small number of people and based on assumptions which were not necessarily wrong, but the discussion about and the testing of which was minimal.
The Interlanguage links feature is not much smaller than Flagged Revisions, which is being thoroughly tested and discussed for many months now.
On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
I agree that this comparison is inaccurate, but I disagree that this was Sue's assumption. All she said was that vocal outcry on a mailing list should not be construed as community consensus. I hope that no one disagrees with this.
I see a good thing and a bad thing happening in this thread. The good thing is that this discussion is airing a lot of legitimate concerns about the specific change in question. I expect that the resulting design changes will be better after this whole thing plays out, which is The Right Thing.
The bad thing is the us versus them tone in this and other messages. There is a larger question about ownership and decision-making that is subtle and hard, and we need to continue to work these out. Since I'm going to put myself in the vocal minority and disagree with most of the points in this message, I'll start with what I agree with. :-)
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Absolutely. Assume good faith. I know you and many others feel the same way about the UX team
That said, keep in mind that most people assume that everyone thinks like they do. This is not an us versus them thing; this is a natural, human thing. I heard a great tip from a psychologist once: If you want to know what people truly think, ask them what they think other people think.
Given this quirk of human nature, we need more rigorous ways of making decisions than polling people, especially small, self-selecting groups. Being data-driven is one of those ways. But being data-driven is hard, too, because you still have to interpret the data. Which brings me to...
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on.
Agreed. 1% is absolutely a large number of clicks, especially given our overall traffic. Someone in a different message also pointed out that click rates alone don't tell the whole story; if they did, then one could argue that we should eliminate the Edit button.
Good design isn't just about following the user path; it's also about guiding the users in a way that's appropriate to the mission of the work. In that vein, I think the substance of most of the messages in this thread have been very positive. It's resulted in data (such as Max's numbers) that have helped to round out everyone's understanding of the issue, but most importantly, it's resulted in critical context as to why people may be acting a certain way and why these things matter in the first place.
There is a clear attitude from the foundation staff that I, and others, are perceiving in these discussions. The notion that the community of contributors is a particularly whiny batch of customers who must be 'managed', that they express demands unconnected from the needs of the readers... and that it is more meaningful when a couple of office staff retreat to some meeting room and say "we reached a decision". Sadly, this attitude appears to be the worst from the former volunteers on the staff—they are not afraid to speak up in community discussion, and feel a need to distinguish themselves from all the volunteers.
This needs to stop and a point needs to be made clear:
I mostly disagree with this. First, let me refer back to Sue's original point. The people on this list are not representative of the community of contributors. It is a self-selecting sample. That doesn't invalidate the substance of what's said on this list, but it does raise questions about claims of consensus when there seems to be a large, vocal group of people agreeing on a point. Passionate discourse on this list is a data point, but it's only one data point, and it needs to be interpreted in the context of many data points.
Second, most people -- both staff and people on this list -- _are_ disconnected from the needs of the majority of the readers. None of us represent the average reader, and so we need other data points to truly understand what the average reader experiences. And frankly, only a few people in this thread seem to acknowledge this.
You're essentially claiming that certain staff members are copping a "we know better" attitude. That may be true (although I honestly don't see it in this thread), and if it's happening, it's wrong. That said, people on this list are copping the exact same attitude. Let's not draw generalities about us versus them based on a few individuals. Let's instead find smarter ways to make decisions that will help us fulfill our mission. You have been one of the best at doing this, and we need more of it.
This community is who made the sites. I don't just mean the articles. I mean the user interfaces, the PR statements, the fundraiser material, _everything_. The success rates for companies trying to build large and popular websites is miserable. Every successful one is a fluke, and all the successful ones have a staff and budget orders of magnitude larger than yours.
What's the success rate for community-run web sites?
Drawing a company vs community-run comparison like this isn't just wrong, it's pointless. The level of success of all of the top web sites is a fluke to some extent. There are great, great community sites -- both grassroots and company-run -- that will never reach the scale of Wikimedia or Google.
We're facing huge challenges right now. We need everyone to help. The Foundation staff understands this. Most people here understand this. The question is, what's the best way to grapple these challenges?
For starters, we need to come to a better, collective understanding of what it means to come to consensus. It's not reasonable to suggest that every single design decision first be tested on foundation-l, nor would that lead to the best decisions, even if it were reasonable. Developers (including non-staff) need permission to Be Bold. We all need to have permission to try things, to make mistakes, and to learn from them. If you want to draw any lessons from the top web sites on how to grow a successful site, that would be the most important one.
It's not clear to me that Trevor should have reverted Aryeh's change. But isn't it remarkable that Aryeh had the power to revert in the first place? And that the end result was that Roan re-instated the revert? If the Foundation were on some power trip, would that have been possible?
We have this beautiful opportunity to learn how to do good user experience collectively and at scale. No one has figured out how to do this yet, but everyone is trying. There are going to be bumps along the way. We have to continue to Assume Good Faith and encourage each other to Be Bold.
Somehow, the community knows how to take the ragtag assembly of its members: the whining, the warped personal preferences, the inspired motivations of individuals and small groups, the collective voice of the uninformed, and a smattering of contributions from world class experts the likes of which we'd never be able to hire and retain, the good and the bad—and fuse it into something which can build output with broad appeal and generally consistent, if somewhat strange, performance.
In some areas yes, in others, no. We all still have a lot to learn, and we need to learn this together.
If Wikimedia itself can't learn how to either develop the same coalition-building skills, participate within the existing community process, or stand out of the way—we'll lose something great.
I assume you mean the Foundation here. This, I agree with, although I'll also note that the community includes the Foundation. The community process is itself dynamic, and it needs to evolve in positive ways.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_.
I don't see evidence of that here.
There are areas where the community's coverage is inadequate or inconsistent, and I think that dedicated staff acting as gap-fillers could greatly improve the results. But not if the price of those contributions is to exclude or pigeonhole the great work done by the broad community, either directly by "we reached a decision"-type edicts, or indirectly by removing the personal pride and responsibility that people feel for the complete site.
I agree with this. Again, I don't see evidence of exclusion happening right now. I think the discourse has been positive, I think there are still some things that need to be worked out, and I think the right thing will happen in the end.
=Eugene
On 7 June 2010 16:52, Eugene Eric Kim eekim@blueoxen.com wrote:
Good design isn't just about following the user path; it's also about guiding the users in a way that's appropriate to the mission of the work.
This appears to sum up the problem with this change: the usability team focused on some ideal of usability, and ignored the fact that they switched off an important path which people used to "freely share in the sum of all knowledge. That's our commitment."
Was the Foundation mission statement expressly part of the usability initiative? Was the obvious conflict with the mission statement considered when the list was switched to collapsed by default?
(That's a "yes or no" question, and I'd love to hear the answer.)
- d.
On Mon, Jun 7, 2010 at 11:52 AM, Eugene Eric Kim eekim@blueoxen.com wrote:
On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
I agree that this comparison is inaccurate, but I disagree that this was Sue's assumption. All she said was that vocal outcry on a mailing list should not be construed as community consensus. I hope that no one disagrees with this.
"All she said", no. She didn't state that. You're putting words in her mouth.
Perhaps I'm guilty of the same crime. But What Sue said was
"The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with."
I read that as contrasting the purposes of the UX team and the people commenting here. I am unable to determine any other reason for bringing these two statements together except for the purpose of drawing the comparison I suggested was being made, even with your proposed alternative.
We all communicate unclearly at times, — and I am more than willing to accept that I saw a comparison there which was not intended.
In the interest of good communication I hope that you will take the time to consider how I could have come to the understanding that I did. I'm sure many other people on this list had the same understanding.
What we have here is a nearly unanimous response with respect to the disposition of the interwiki links. If you'd like me to bring this to the larger community I can do so but my understanding was that the normal community process was already quashed with respect to this change.
While no single forum is indicative of a consensus of the entire community, the broadness of the response here is a strong indicator.
[snip]
The bad thing is the us versus them tone in this and other messages. There is a larger question about ownership and decision-making that is subtle and hard, and we need to continue to work these out. Since I'm going to put myself in the vocal minority and disagree with most of the points in this message, I'll start with what I agree with. :-)
I don't believe that there is anything particularly subtle here. We have many community processes in which foundation staff are welcome to contribute to as peers with a common interest.
When you fail to do so you have created the "us" vs "them" by your own actions.
Rather then trying to draw "us" vs "them" lines in the sand, I am in fact pleading that the foundation discontinue doing so.
In order to do that I must first acknowledge the division which I believe has already formed. [more on this later]
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Absolutely. Assume good faith. I know you and many others feel the same way about the UX team
Absolutely.
I would suggest that the broader community (and not necessarily the participants here) has greater experience than the usability team, and even the portion of the community represented here has a more diverse composition than the UX team.
However, if we were to combine the two— we would have something strictly superior to the component parts. Unfortunately, we're still able to speak about the community and the UX teams as distinct entities. This division will continue so long as the relationship is viewed in the context of "decision"/"feedback" rather than as a dialogue between peers.
That said, keep in mind that most people assume that everyone thinks like they do. This is not an us versus them thing; this is a natural, human thing. I heard a great tip from a psychologist once: If you want to know what people truly think, ask them what they think other people think.
[snip]
While we are on the topic of lessons in human nature, please allow me to introduce the list to the fundamental attribution error: http://en.wikipedia.org/wiki/Fundamental_attribution_error
We all may find it informative.
Given this quirk of human nature, we need more rigorous ways of making decisions than polling people, especially small, self-selecting groups. Being data-driven is one of those ways. But being data-driven is hard, too, because you still have to interpret the data. Which brings me to...
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on.
Agreed. 1% is absolutely a large number of clicks, especially given our overall traffic. Someone in a different message also pointed out that click rates alone don't tell the whole story; if they did, then one could argue that we should eliminate the Edit button.
Good design isn't just about following the user path; it's also about guiding the users in a way that's appropriate to the mission of the work. In that vein, I think the substance of most of the messages in this thread have been very positive. It's resulted in data (such as Max's numbers) that have helped to round out everyone's understanding of the issue, but most importantly, it's resulted in critical context as to why people may be acting a certain way and why these things matter in the first place.
I do not believe that I am understanding the point you are making here.
The group collected on this list appears to generally hold the view that we the interwiki links are something we should be emphasizing beyond the level justified by their usage.
You agree that 1% is a rather large amount of usage, as Tim pointed out— it would compare favourably to the edit feature.
From these facts that we appear to agree on, if we were design from
the perspective of promoting values we would continue to keep the links prominent, and if we were to be data-driven about usage we would also keep the links prominent.
I'm not seeing anything in your above statement to suggest why would decide not to make the links prominent.
If you're not — then why is the discussion continuing? "The usability team made a mistake, thanks are owed to X,Y,Z for catching an fixing it. Sorry we stood in the way of the fixes and implied that you couldn't, that won't happen again." Would be a fine conclusion.
'We will now factor in your critical context and emit new fully formed sausage soon' would not be.
[snip]
I mostly disagree with this. First, let me refer back to Sue's original point. The people on this list are not representative of the community of contributors. It is a self-selecting sample. That doesn't invalidate the substance of what's said on this list, but it does raise questions about claims of consensus when there seems to be a large, vocal group of people agreeing on a point. Passionate discourse on this list is a data point, but it's only one data point, and it needs to be interpreted in the context of many data points.
My prior offer to bring in a larger part of the community still stands, if you really believe that to be an issue here.
In particular, the list of people agreeing on this point includes a number of parties which seldom agree with each other. I have found that to usually be an indication of an issue which will have a fairly broad support in the larger community.
I didn't think bringing on a hundred person pile-on would help— if anything I could see it only increasing distrust of the foundation in the larger community. It would leave me in a position of having to continue to damn the foundation's actions on this and other recent issues when I'd rather just move forward...
Second, most people -- both staff and people on this list -- _are_ disconnected from the needs of the majority of the readers. None of us represent the average reader, and so we need other data points to truly understand what the average reader experiences. And frankly, only a few people in this thread seem to acknowledge this.
The very first message on this subject was requesting additional data.
I think we'd all be interested in seeing more data on the subject.
Though I don't think it's valuable to define the meaning of the data only after you've seen it. That just leads to circular justification, the criteria needs to come first.
Nor do I think we need more data in order to conclude that the promotion of the interwiki links is an important _value_ for us, and that we wish to promote it above and beyond the level justified on a pure ease of use basis.
Accordingly, I think the data which would be most helpful would be indicators that were suggesting that the existence of the exposed interwikis creates a non-trivial harm to the usability of the site, since that is the kind of evidence which would overrule the value based decision to promote the links.
That kind of outcome would be fairly surprising to me, and I think everyone else here as no one has been suggesting that such an outcome is expected.
You're essentially claiming that certain staff members are copping a "we know better" attitude. That may be true (although I honestly don't see it in this thread), and if it's happening, it's wrong. That said, people on this list are copping the exact same attitude. Let's not draw generalities about us versus them based on a few individuals. Let's instead find smarter ways to make decisions that will help us fulfill our mission. You have been one of the best at doing this, and we need more of it.
I think that drawing an equivalence between comments by the staff and community here is an error.
I expect all of us, staff and community alike, to read the comments of others, consult their best available information, and form reasoned viewpoints which they then present and defend vigorously but politely. ... and that as more information and more viewpoints become available that people will have open minds to revising their positions. This is how discussion works.
Nothing about presenting a strong argument should be construed as a fault-worthy "we know better". Even in my strongest statements I am willing to believe that I could be wrong, and I know the same to be true of many others here— It would be hard to sanely hold any other position in the presence of so many experienced and intelligent people. I apologise for sometimes allowing it to sound otherwise.
But this discussion process only works when the participants are peers.
In this case, in spite of a mostly lacking defence and the obvious super-majority holding the counter position, the site continued to display the collapsed interwikis for a prolonged duration.
It is not vigorously defended positions which I am characterizing as "we know better", it is the obvious _absence_ of those positions combined with the reality of the website (at the time I wrote the message). And this characterization is further strengthened by the use of language like "feedback" and the constant reminders of non-representativeness as a fault of the assembled community when the same could equally be applied to the UX team and the staff as a whole.
And once we have been divided into groups, those with the authority to impose without broad agreement or even solid justification, and those without... then I must answer that implicit statement of superior knowledge with the correction: Of the two, it is the community which holds the stronger claim.
What's the success rate for community-run web sites? Drawing a company vs community-run comparison like this isn't just wrong, it's pointless. The level of success of all of the top web sites is a fluke to some extent. There are great, great community sites -- both grassroots and company-run -- that will never reach the scale of Wikimedia or Google.
The success rate for Wikipedia is one for one. It exists, it is successful.
We have a successful modality... but it is one which many people would be surprised that it works at all.
Some people might think it reasonable to try to direct things towards a more traditional mode of operation— with a reasonable eye towards fixing the things we do poorly, but that would ignore the enormous number of things our mode of operation does shockingly well such as existing at all.
So, my points was that traditional organizations are fantastically unsuccessful at accomplishing what we have accomplished and so there is no reason to believe a change would be an improvement and many reasons to believe it would be an enormous failure.
We're facing huge challenges right now. We need everyone to help. The Foundation staff understands this. Most people here understand this. The question is, what's the best way to grapple these challenges?
For starters, we need to come to a better, collective understanding of what it means to come to consensus.
You sound as though you believe that to be a new issue, but it isn't. I see no reason to conclude that the challenge we face now are materially different from the ones community has been grappling with for many years... which has resulted in a multitude of more or less workable compromises.
It's not reasonable to suggest that every single design decision first be tested on foundation-l,
This isn't how our communities usually work in any case.
Bold. Revert. Discuss. is a common modality.
I think that the group assembled here would largely agree that it would have been acceptable for the UX team to make the change— even with little to no public discussion, then not interfere with the community to reverting it when non-trivial objections were raised, then engage in a discussion about the ultimate disposition of the feature.
This pattern allows a significant majority of changes to happen without significant conflict and without the impediment of excessive discussion.
For more critical changes a better process is
Discuss Poll Discuss [ Bold. Revert. Discuss. ] [Bold. Discuss. Revise.] Agree.
Iterating the sections in brackets. We've used processes like this before for other user interface changes.
Through this process we've managed to make non-trivial user interface changes without leaving popular clients out in the cold for weeks at a time.
Sometimes an inclusive process simply takes more time. But the only deadlines we're facing are the ones we impose on ourselves. We can afford some things being slower so long as progress is made, it's a reasonable price to pay.
nor would that lead to the best decisions, even if it were reasonable.
Best is the often enemy of good.
The decisions we make are so highly dimensional that the prospect of ever finding a "best" decision is a laughable joke.
Even if we agreed on a definition of best-ness the chances of actually finding the best solution to anything is infinitesimal. Usually a definition of bestness can't be found because everyone weighs different values differently.
Moreover, a locally best solution isn't desirable. I hope and expect Wikipedia to be around 100 years from now. I hope that the decisions we make today are the ones with the greatest return over that time horizon (or longer). Since no one can know what the future will ask of us, this is just more evidence that a best decision is not possible no matter who makes it.
I fully admit to being clueless. I object to people claiming that have a best solution when no one can.
I think what we generally strive for is "the best long term outcome which doesn't suck too bad in any particular way in the short term".
In any case, Wikipedia is not a good place for people who do not see the value in the inclusive decision and who are too concerned about getting what they believe to be the best result.
In terms of content editing, people who are very concerned with saying the "right" thing are usually the ones who end up blocked.
Developers (including non-staff) need permission to Be Bold. We all
We agree.
need to have permission to try things, to make mistakes, and to learn from them. If you want to draw any lessons from the top web sites on how to grow a successful site, that would be the most important one.
But being bold necessarily implies the entire community process.
If a change is imposed without the rest of the process, that's not "Being bold", thats just simply a forced change.
Sometimes forced changes are reasonable- for example, a change to keep the site operating or a reversion of a user interface change that was blocking access to a large swath of users, but don't confuse an out of process emergency action with the bold actions of the normal process.
It's not clear to me that Trevor should have reverted Aryeh's change. But isn't it remarkable that Aryeh had the power to revert in the first place?
No. This is how we work here, and it's generally how we've worked here for a almost a decade. For many of our contributors Wikipedia and this governance has existed their entire adult life, and it is only way of operating a large website that they've ever participated in.
I think it's remarkable that we've given people the authority to make changes who still find our process surprising or remarkable.
It's only remarkable to people who don't appreciate the historical context... those who view the situation as an organization being gracious enough to let its "users" modify important parts of the site.
Allow me an alternative: what we have is a community of people who have contributed millions of man hours of work towards a common goal, and in the furtherance of that goal we created an office with a staff to handle routine administrative work and the interoperation with external entities (as it's hard to deal with the borg-of-wikipedia without becoming one). We were able to do this because the public supports our goals and the efforts of the community were successful at meeting an important need of the public.
The remarkable thing is the payment you receive. Not the ability of the community to make changes to the site.
... and please don't take that wrong: I am _overjoyed_ that the Foundation has been able to employ so many thoughtful and productive people, but I think it is amazing that it is able to do so on the thin slice of the value that our collective project has produced which comes back in the form of donations.
And that the end result was that Roan re-instated the revert? If the Foundation were on some power trip, would that have been possible?
Power trip isn't something that I said, and I don't think it accurately expresses what I would have alleged.
I think there is a misunderstanding about authority here, not a power-trip. The foundation was never created to "govern" the community. It was created to keep the machines well oiled on the communities behalf, sometimes oiling the machine requires a the execution of authority, sometimes you have to take the elevator out of service in order to improve it, but usually not and usually not much. Without the communities' continued governance the sites could not run at all. The WMF staff is far too small to even dream of replacing the communities role in governance.
None the less, you need to look no further than the facebook privacy policy reversal a year ago for a counter example. Controlling and commanding organizations routinely respond to public outcry.
That it took many days to get that far is a perfect example of the foundation staff not acting as an equal peer with the community.
We have this beautiful opportunity to learn how to do good user experience collectively and at scale. No one has figured out how to do this yet, but everyone is trying. There are going to be bumps along the way. We have to continue to Assume Good Faith and encourage each other to Be Bold.
So, you're suggesting that Wikipedia became a top 10 website, reliably outperforming commercial clones like answers.com that had professional development staffs, all our content, and then some, with an acutely _bad_ user experience?
No?
I think we _had_ a beautiful opportunity to take an already effective and innovative development system and take it to the next level, but instead we tried to supplant it with the same cathedral development model plus 'feedback' which is used by most commercial web-properties.
It wasn't clear to me that what was happening since a lot of community development appears almost fully formed from a single creator or team, the distinction only became clear when the response to concerns was "accepting feedback" rather than cooperation.
The fact that my blackberry _still_ can't load Wikipedia after weeks is clear evidence that a change in how things are run has happened.
I took the time to respond to this thread, and to your message, because I'd like to clear up the confusion and restore the opportunity for cooperation as peers while it is still early enough for such a mild remedy.
Somehow, the community knows how to take the ragtag assembly of its
[snip]
with broad appeal and generally consistent, if somewhat strange, performance.
In some areas yes, in others, no. We all still have a lot to learn, and we need to learn this together.
Absolutely, the community needs to learn how to address all areas. We've been learning how to address all areas for a long time. One thing we learned a long time ago is that asserting hard-authority over the community generally generates more heat than light, that cooperation as equals is usually more productive, and that other means should only be invoked as a last resort.
This isn't the same as accepting the notion of a staff that learns how to accept feedback, and a community that needs to learn to only whine and hope for change instead of taking action. ... the impression that I and others have received here.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_.
I don't see evidence of that here.
[snip]
I agree with this. Again, I don't see evidence of exclusion happening right now. I think the discourse has been positive, I think there are still some things that need to be worked out, and I think the right thing will happen in the end.
I think your inability to see what is obvious to me and many others is an example of the problem.
I'm confident that the right thing will happen in the end, my only concern is how much discomfort is on the road from here to there.
I hope my understanding of the motivations, attitudes, and plans of those involved are incorrect. But these understandings are driven by the actions of the foundation staff, they are positions I'd take with respect to anyone engaging in the same actions. They can be most easily remedied by the staff collaborating with the community as an equal parter, and not mealy as a source of optional feedback. Ten thousand words could not carry the impact of just a few actions.
One major problem I have with this entire initiative, at least as I understand it, is that data was only collected from en.wp and mostly from native English speakers. Wikipedia is not monolingual, although many of our users are... and it's important to remember that many of these people are monolingual in languages other than English. Without further study, we have no way of knowing how many of the conclusions we may have drawn from the data will still hold true for non-English-speaking audiences.
I had hoped that the internationalism of our organization would grow, rather than any sort of increasing centralization and the treatment of en.wp as a "flagship" project, the capital of the Wikimedia empire. It is not.
-m.
On Mon, Jun 7, 2010 at 12:15 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Mon, Jun 7, 2010 at 11:52 AM, Eugene Eric Kim eekim@blueoxen.com wrote:
On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM, susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
I agree that this comparison is inaccurate, but I disagree that this was Sue's assumption. All she said was that vocal outcry on a mailing list should not be construed as community consensus. I hope that no one disagrees with this.
"All she said", no. She didn't state that. You're putting words in her mouth.
Perhaps I'm guilty of the same crime. But What Sue said was
"The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with."
I read that as contrasting the purposes of the UX team and the people commenting here. I am unable to determine any other reason for bringing these two statements together except for the purpose of drawing the comparison I suggested was being made, even with your proposed alternative.
We all communicate unclearly at times, — and I am more than willing to accept that I saw a comparison there which was not intended.
In the interest of good communication I hope that you will take the time to consider how I could have come to the understanding that I did. I'm sure many other people on this list had the same understanding.
What we have here is a nearly unanimous response with respect to the disposition of the interwiki links. If you'd like me to bring this to the larger community I can do so but my understanding was that the normal community process was already quashed with respect to this change.
While no single forum is indicative of a consensus of the entire community, the broadness of the response here is a strong indicator.
[snip]
The bad thing is the us versus them tone in this and other messages. There is a larger question about ownership and decision-making that is subtle and hard, and we need to continue to work these out. Since I'm going to put myself in the vocal minority and disagree with most of the points in this message, I'll start with what I agree with. :-)
I don't believe that there is anything particularly subtle here. We have many community processes in which foundation staff are welcome to contribute to as peers with a common interest.
When you fail to do so you have created the "us" vs "them" by your own actions.
Rather then trying to draw "us" vs "them" lines in the sand, I am in fact pleading that the foundation discontinue doing so.
In order to do that I must first acknowledge the division which I believe has already formed. [more on this later]
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Absolutely. Assume good faith. I know you and many others feel the same way about the UX team
Absolutely.
I would suggest that the broader community (and not necessarily the participants here) has greater experience than the usability team, and even the portion of the community represented here has a more diverse composition than the UX team.
However, if we were to combine the two— we would have something strictly superior to the component parts. Unfortunately, we're still able to speak about the community and the UX teams as distinct entities. This division will continue so long as the relationship is viewed in the context of "decision"/"feedback" rather than as a dialogue between peers.
That said, keep in mind that most people assume that everyone thinks like they do. This is not an us versus them thing; this is a natural, human thing. I heard a great tip from a psychologist once: If you want to know what people truly think, ask them what they think other people think.
[snip]
While we are on the topic of lessons in human nature, please allow me to introduce the list to the fundamental attribution error: http://en.wikipedia.org/wiki/Fundamental_attribution_error
We all may find it informative.
Given this quirk of human nature, we need more rigorous ways of making decisions than polling people, especially small, self-selecting groups. Being data-driven is one of those ways. But being data-driven is hard, too, because you still have to interpret the data. Which brings me to...
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on.
Agreed. 1% is absolutely a large number of clicks, especially given our overall traffic. Someone in a different message also pointed out that click rates alone don't tell the whole story; if they did, then one could argue that we should eliminate the Edit button.
Good design isn't just about following the user path; it's also about guiding the users in a way that's appropriate to the mission of the work. In that vein, I think the substance of most of the messages in this thread have been very positive. It's resulted in data (such as Max's numbers) that have helped to round out everyone's understanding of the issue, but most importantly, it's resulted in critical context as to why people may be acting a certain way and why these things matter in the first place.
I do not believe that I am understanding the point you are making here.
The group collected on this list appears to generally hold the view that we the interwiki links are something we should be emphasizing beyond the level justified by their usage.
You agree that 1% is a rather large amount of usage, as Tim pointed out— it would compare favourably to the edit feature.
From these facts that we appear to agree on, if we were design from the perspective of promoting values we would continue to keep the links prominent, and if we were to be data-driven about usage we would also keep the links prominent.
I'm not seeing anything in your above statement to suggest why would decide not to make the links prominent.
If you're not — then why is the discussion continuing? "The usability team made a mistake, thanks are owed to X,Y,Z for catching an fixing it. Sorry we stood in the way of the fixes and implied that you couldn't, that won't happen again." Would be a fine conclusion.
'We will now factor in your critical context and emit new fully formed sausage soon' would not be.
[snip]
I mostly disagree with this. First, let me refer back to Sue's original point. The people on this list are not representative of the community of contributors. It is a self-selecting sample. That doesn't invalidate the substance of what's said on this list, but it does raise questions about claims of consensus when there seems to be a large, vocal group of people agreeing on a point. Passionate discourse on this list is a data point, but it's only one data point, and it needs to be interpreted in the context of many data points.
My prior offer to bring in a larger part of the community still stands, if you really believe that to be an issue here.
In particular, the list of people agreeing on this point includes a number of parties which seldom agree with each other. I have found that to usually be an indication of an issue which will have a fairly broad support in the larger community.
I didn't think bringing on a hundred person pile-on would help— if anything I could see it only increasing distrust of the foundation in the larger community. It would leave me in a position of having to continue to damn the foundation's actions on this and other recent issues when I'd rather just move forward...
Second, most people -- both staff and people on this list -- _are_ disconnected from the needs of the majority of the readers. None of us represent the average reader, and so we need other data points to truly understand what the average reader experiences. And frankly, only a few people in this thread seem to acknowledge this.
The very first message on this subject was requesting additional data.
I think we'd all be interested in seeing more data on the subject.
Though I don't think it's valuable to define the meaning of the data only after you've seen it. That just leads to circular justification, the criteria needs to come first.
Nor do I think we need more data in order to conclude that the promotion of the interwiki links is an important _value_ for us, and that we wish to promote it above and beyond the level justified on a pure ease of use basis.
Accordingly, I think the data which would be most helpful would be indicators that were suggesting that the existence of the exposed interwikis creates a non-trivial harm to the usability of the site, since that is the kind of evidence which would overrule the value based decision to promote the links.
That kind of outcome would be fairly surprising to me, and I think everyone else here as no one has been suggesting that such an outcome is expected.
You're essentially claiming that certain staff members are copping a "we know better" attitude. That may be true (although I honestly don't see it in this thread), and if it's happening, it's wrong. That said, people on this list are copping the exact same attitude. Let's not draw generalities about us versus them based on a few individuals. Let's instead find smarter ways to make decisions that will help us fulfill our mission. You have been one of the best at doing this, and we need more of it.
I think that drawing an equivalence between comments by the staff and community here is an error.
I expect all of us, staff and community alike, to read the comments of others, consult their best available information, and form reasoned viewpoints which they then present and defend vigorously but politely. ... and that as more information and more viewpoints become available that people will have open minds to revising their positions. This is how discussion works.
Nothing about presenting a strong argument should be construed as a fault-worthy "we know better". Even in my strongest statements I am willing to believe that I could be wrong, and I know the same to be true of many others here— It would be hard to sanely hold any other position in the presence of so many experienced and intelligent people. I apologise for sometimes allowing it to sound otherwise.
But this discussion process only works when the participants are peers.
In this case, in spite of a mostly lacking defence and the obvious super-majority holding the counter position, the site continued to display the collapsed interwikis for a prolonged duration.
It is not vigorously defended positions which I am characterizing as "we know better", it is the obvious _absence_ of those positions combined with the reality of the website (at the time I wrote the message). And this characterization is further strengthened by the use of language like "feedback" and the constant reminders of non-representativeness as a fault of the assembled community when the same could equally be applied to the UX team and the staff as a whole.
And once we have been divided into groups, those with the authority to impose without broad agreement or even solid justification, and those without... then I must answer that implicit statement of superior knowledge with the correction: Of the two, it is the community which holds the stronger claim.
What's the success rate for community-run web sites? Drawing a company vs community-run comparison like this isn't just wrong, it's pointless. The level of success of all of the top web sites is a fluke to some extent. There are great, great community sites -- both grassroots and company-run -- that will never reach the scale of Wikimedia or Google.
The success rate for Wikipedia is one for one. It exists, it is successful.
We have a successful modality... but it is one which many people would be surprised that it works at all.
Some people might think it reasonable to try to direct things towards a more traditional mode of operation— with a reasonable eye towards fixing the things we do poorly, but that would ignore the enormous number of things our mode of operation does shockingly well such as existing at all.
So, my points was that traditional organizations are fantastically unsuccessful at accomplishing what we have accomplished and so there is no reason to believe a change would be an improvement and many reasons to believe it would be an enormous failure.
We're facing huge challenges right now. We need everyone to help. The Foundation staff understands this. Most people here understand this. The question is, what's the best way to grapple these challenges?
For starters, we need to come to a better, collective understanding of what it means to come to consensus.
You sound as though you believe that to be a new issue, but it isn't. I see no reason to conclude that the challenge we face now are materially different from the ones community has been grappling with for many years... which has resulted in a multitude of more or less workable compromises.
It's not reasonable to suggest that every single design decision first be tested on foundation-l,
This isn't how our communities usually work in any case.
Bold. Revert. Discuss. is a common modality.
I think that the group assembled here would largely agree that it would have been acceptable for the UX team to make the change— even with little to no public discussion, then not interfere with the community to reverting it when non-trivial objections were raised, then engage in a discussion about the ultimate disposition of the feature.
This pattern allows a significant majority of changes to happen without significant conflict and without the impediment of excessive discussion.
For more critical changes a better process is
Discuss Poll Discuss [ Bold. Revert. Discuss. ] [Bold. Discuss. Revise.] Agree.
Iterating the sections in brackets. We've used processes like this before for other user interface changes.
Through this process we've managed to make non-trivial user interface changes without leaving popular clients out in the cold for weeks at a time.
Sometimes an inclusive process simply takes more time. But the only deadlines we're facing are the ones we impose on ourselves. We can afford some things being slower so long as progress is made, it's a reasonable price to pay.
nor would that lead to the best decisions, even if it were reasonable.
Best is the often enemy of good.
The decisions we make are so highly dimensional that the prospect of ever finding a "best" decision is a laughable joke.
Even if we agreed on a definition of best-ness the chances of actually finding the best solution to anything is infinitesimal. Usually a definition of bestness can't be found because everyone weighs different values differently.
Moreover, a locally best solution isn't desirable. I hope and expect Wikipedia to be around 100 years from now. I hope that the decisions we make today are the ones with the greatest return over that time horizon (or longer). Since no one can know what the future will ask of us, this is just more evidence that a best decision is not possible no matter who makes it.
I fully admit to being clueless. I object to people claiming that have a best solution when no one can.
I think what we generally strive for is "the best long term outcome which doesn't suck too bad in any particular way in the short term".
In any case, Wikipedia is not a good place for people who do not see the value in the inclusive decision and who are too concerned about getting what they believe to be the best result.
In terms of content editing, people who are very concerned with saying the "right" thing are usually the ones who end up blocked.
Developers (including non-staff) need permission to Be Bold. We all
We agree.
need to have permission to try things, to make mistakes, and to learn from them. If you want to draw any lessons from the top web sites on how to grow a successful site, that would be the most important one.
But being bold necessarily implies the entire community process.
If a change is imposed without the rest of the process, that's not "Being bold", thats just simply a forced change.
Sometimes forced changes are reasonable- for example, a change to keep the site operating or a reversion of a user interface change that was blocking access to a large swath of users, but don't confuse an out of process emergency action with the bold actions of the normal process.
It's not clear to me that Trevor should have reverted Aryeh's change. But isn't it remarkable that Aryeh had the power to revert in the first place?
No. This is how we work here, and it's generally how we've worked here for a almost a decade. For many of our contributors Wikipedia and this governance has existed their entire adult life, and it is only way of operating a large website that they've ever participated in.
I think it's remarkable that we've given people the authority to make changes who still find our process surprising or remarkable.
It's only remarkable to people who don't appreciate the historical context... those who view the situation as an organization being gracious enough to let its "users" modify important parts of the site.
Allow me an alternative: what we have is a community of people who have contributed millions of man hours of work towards a common goal, and in the furtherance of that goal we created an office with a staff to handle routine administrative work and the interoperation with external entities (as it's hard to deal with the borg-of-wikipedia without becoming one). We were able to do this because the public supports our goals and the efforts of the community were successful at meeting an important need of the public.
The remarkable thing is the payment you receive. Not the ability of the community to make changes to the site.
... and please don't take that wrong: I am _overjoyed_ that the Foundation has been able to employ so many thoughtful and productive people, but I think it is amazing that it is able to do so on the thin slice of the value that our collective project has produced which comes back in the form of donations.
And that the end result was that Roan re-instated the revert? If the Foundation were on some power trip, would that have been possible?
Power trip isn't something that I said, and I don't think it accurately expresses what I would have alleged.
I think there is a misunderstanding about authority here, not a power-trip. The foundation was never created to "govern" the community. It was created to keep the machines well oiled on the communities behalf, sometimes oiling the machine requires a the execution of authority, sometimes you have to take the elevator out of service in order to improve it, but usually not and usually not much. Without the communities' continued governance the sites could not run at all. The WMF staff is far too small to even dream of replacing the communities role in governance.
None the less, you need to look no further than the facebook privacy policy reversal a year ago for a counter example. Controlling and commanding organizations routinely respond to public outcry.
That it took many days to get that far is a perfect example of the foundation staff not acting as an equal peer with the community.
We have this beautiful opportunity to learn how to do good user experience collectively and at scale. No one has figured out how to do this yet, but everyone is trying. There are going to be bumps along the way. We have to continue to Assume Good Faith and encourage each other to Be Bold.
So, you're suggesting that Wikipedia became a top 10 website, reliably outperforming commercial clones like answers.com that had professional development staffs, all our content, and then some, with an acutely _bad_ user experience?
No?
I think we _had_ a beautiful opportunity to take an already effective and innovative development system and take it to the next level, but instead we tried to supplant it with the same cathedral development model plus 'feedback' which is used by most commercial web-properties.
It wasn't clear to me that what was happening since a lot of community development appears almost fully formed from a single creator or team, the distinction only became clear when the response to concerns was "accepting feedback" rather than cooperation.
The fact that my blackberry _still_ can't load Wikipedia after weeks is clear evidence that a change in how things are run has happened.
I took the time to respond to this thread, and to your message, because I'd like to clear up the confusion and restore the opportunity for cooperation as peers while it is still early enough for such a mild remedy.
Somehow, the community knows how to take the ragtag assembly of its
[snip]
with broad appeal and generally consistent, if somewhat strange, performance.
In some areas yes, in others, no. We all still have a lot to learn, and we need to learn this together.
Absolutely, the community needs to learn how to address all areas. We've been learning how to address all areas for a long time. One thing we learned a long time ago is that asserting hard-authority over the community generally generates more heat than light, that cooperation as equals is usually more productive, and that other means should only be invoked as a last resort.
This isn't the same as accepting the notion of a staff that learns how to accept feedback, and a community that needs to learn to only whine and hope for change instead of taking action. ... the impression that I and others have received here.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_.
I don't see evidence of that here.
[snip]
I agree with this. Again, I don't see evidence of exclusion happening right now. I think the discourse has been positive, I think there are still some things that need to be worked out, and I think the right thing will happen in the end.
I think your inability to see what is obvious to me and many others is an example of the problem.
I'm confident that the right thing will happen in the end, my only concern is how much discomfort is on the road from here to there.
I hope my understanding of the motivations, attitudes, and plans of those involved are incorrect. But these understandings are driven by the actions of the foundation staff, they are positions I'd take with respect to anyone engaging in the same actions. They can be most easily remedied by the staff collaborating with the community as an equal parter, and not mealy as a source of optional feedback. Ten thousand words could not carry the impact of just a few actions.
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
I agree that the User Experience Team and the community are still learning how to most effectively work together to do product development. Things aren't perfect, and it make take some time before we get to a comfortable point. I think the best thing we can do is to continue learning from each experience, and hopefully we'll come to a better process as we do more and more projects. We'll make mistakes from time to time and that's just part of the learning process. The important point is to acknowledge where we can do better and learn from those experiences.
Based on this feature, I think there are a number of areas where we could improve. First, I do agree with the various statements that the tone of the revert was overly harsh and authoritarian. I can assure you that that was not the intent of the revert -- we've always endeavored to be open in our collaboration, even in (and hopefully especially in) times of disagreement. We've also been iterative in our approach, and decisions are rarely, if ever, considered final as the web affords us the luxury of continual improvement. Nevertheless, words are important, and the ones used in the revert should have conveyed more openness to discussion, which they did not. We'll make sure that these intentions are clear in the future. We also recognize that the compromise solution that we proposed may not have met the needs of some within the community. But I hope people understand that the shortcomings of this proposal did not arise from a failure to listen. There have been many, many insightful and helpful comments which we took into consideration when developing the proposal.
Another area in which we can improve is data. The debate on the data, especially what path of action the data would suggest, is a very legitimate debate (incidentally, I think the data may have been partly misinterpreted [1], but the debate is nonetheless valid). This is a case where, because the data is imperfect, we will require additional perspectives to make a reasoned decision (someone used the phrase “life isn’t so simple”). And as several posters have pointed out, the members on this mailing list represent a small, self-selecting sample of users, as do the members of the User Experience Team. We carry our own biases into these decisions. One way to overcome the inherent biases, as Eugene mentioned, is to be more data driven. In this particular case, I am an advocate of using A/B testing as a way to better understand the impacts of a change in the user interface. In general, I am very much in favor of having better analytics so that the balance of decision-making tilts even more towards data.
So in terms of a path forward, here is a proposal: 1. Immediate revert so that all languages are exposed by default. 2. We will continue work on a compromise solution. The current interface is probably not perfect, so we’ll be continuing to look for ways to improve it. We welcome your ideas -- please direct them to [2] so we can keep track. 3. We will A/B-test proposed solutions against the default. We'll involve the community to design the A/B test (e.g., what % of traffic, what threshold we use for decisions, etc). In response to the comments about the data being only on enwp, we should be open to the fact that different Wikipedias may require different implementations.
In fairness, I should say clearly that while #1 will happen immediately, #2 and #3 will probably not be immediate. Our team will need to balance this feature against other commitments (e.g., the next phase of the default rollout later this week, which will roll out with the inter-wiki links expanded).
From a broader standpoint, I think there needs to be a direct discussion around how to involve the community in these types of decisions. I’ve created a page on the usability wiki to discuss ideas [3] and look forward to the discussion.
Please let us know what you think of the proposed path forward.
Howie
[1] Based on the replies to the post, it seemed as though some people might have thought that the ~1% figure for Monobook referred to all clicks. To be clear, the ~1% refers to ~1% of total left-nav clicks, not clicks overall. But better data is still needed to inform this decision. [2] http://usability.wikimedia.org/wiki/Interwiki_Link_Proposals [3] http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas
On 6/7/10 1:21 PM, Mark Williamson wrote:
One major problem I have with this entire initiative, at least as I understand it, is that data was only collected from en.wp and mostly from native English speakers. Wikipedia is not monolingual, although many of our users are... and it's important to remember that many of these people are monolingual in languages other than English. Without further study, we have no way of knowing how many of the conclusions we may have drawn from the data will still hold true for non-English-speaking audiences.
I had hoped that the internationalism of our organization would grow, rather than any sort of increasing centralization and the treatment of en.wp as a "flagship" project, the capital of the Wikimedia empire. It is not.
-m.
On Mon, Jun 7, 2010 at 12:15 PM, Gregory Maxwellgmaxwell@gmail.com wrote:
On Mon, Jun 7, 2010 at 11:52 AM, Eugene Eric Kimeekim@blueoxen.com wrote:
On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwellgmaxwell@gmail.com wrote:
On Sat, Jun 5, 2010 at 2:03 PM,susanpgardner@gmail.com wrote:
Sorry for top-posting.
Austin, think about who "everyone" is. The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
Sue, you appear to be making the assumption that the folks here are writing from a position of their personal preferences while the usability team is working on the behalf of the best interests of the project.
I don't believe this comparison to be accurate.
I agree that this comparison is inaccurate, but I disagree that this was Sue's assumption. All she said was that vocal outcry on a mailing list should not be construed as community consensus. I hope that no one disagrees with this.
"All she said", no. She didn't state that. You're putting words in her mouth.
Perhaps I'm guilty of the same crime. But What Sue said was
"The folks here on foundation-l are not representative of readers. The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with."
I read that as contrasting the purposes of the UX team and the people commenting here. I am unable to determine any other reason for bringing these two statements together except for the purpose of drawing the comparison I suggested was being made, even with your proposed alternative.
We all communicate unclearly at times, — and I am more than willing to accept that I saw a comparison there which was not intended.
In the interest of good communication I hope that you will take the time to consider how I could have come to the understanding that I did. I'm sure many other people on this list had the same understanding.
What we have here is a nearly unanimous response with respect to the disposition of the interwiki links. If you'd like me to bring this to the larger community I can do so but my understanding was that the normal community process was already quashed with respect to this change.
While no single forum is indicative of a consensus of the entire community, the broadness of the response here is a strong indicator.
[snip]
The bad thing is the us versus them tone in this and other messages. There is a larger question about ownership and decision-making that is subtle and hard, and we need to continue to work these out. Since I'm going to put myself in the vocal minority and disagree with most of the points in this message, I'll start with what I agree with. :-)
I don't believe that there is anything particularly subtle here. We have many community processes in which foundation staff are welcome to contribute to as peers with a common interest.
When you fail to do so you have created the "us" vs "them" by your own actions.
Rather then trying to draw "us" vs "them" lines in the sand, I am in fact pleading that the foundation discontinue doing so.
In order to do that I must first acknowledge the division which I believe has already formed. [more on this later]
I think the people here are speaking up for the sake of the readers, and for the sake of preserving the best of the existing design principles used on the site. I know I am.
Absolutely. Assume good faith. I know you and many others feel the same way about the UX team
Absolutely.
I would suggest that the broader community (and not necessarily the participants here) has greater experience than the usability team, and even the portion of the community represented here has a more diverse composition than the UX team.
However, if we were to combine the two— we would have something strictly superior to the component parts. Unfortunately, we're still able to speak about the community and the UX teams as distinct entities. This division will continue so long as the relationship is viewed in the context of "decision"/"feedback" rather than as a dialogue between peers.
That said, keep in mind that most people assume that everyone thinks like they do. This is not an us versus them thing; this is a natural, human thing. I heard a great tip from a psychologist once: If you want to know what people truly think, ask them what they think other people think.
[snip]
While we are on the topic of lessons in human nature, please allow me to introduce the list to the fundamental attribution error: http://en.wikipedia.org/wiki/Fundamental_attribution_error
We all may find it informative.
Given this quirk of human nature, we need more rigorous ways of making decisions than polling people, especially small, self-selecting groups. Being data-driven is one of those ways. But being data-driven is hard, too, because you still have to interpret the data. Which brings me to...
I was alarmed when I heard the click rates: 1%. That's an enormous number of clicks, considerably higher than I expected with the large number of things available for folks to click on.
Agreed. 1% is absolutely a large number of clicks, especially given our overall traffic. Someone in a different message also pointed out that click rates alone don't tell the whole story; if they did, then one could argue that we should eliminate the Edit button.
Good design isn't just about following the user path; it's also about guiding the users in a way that's appropriate to the mission of the work. In that vein, I think the substance of most of the messages in this thread have been very positive. It's resulted in data (such as Max's numbers) that have helped to round out everyone's understanding of the issue, but most importantly, it's resulted in critical context as to why people may be acting a certain way and why these things matter in the first place.
I do not believe that I am understanding the point you are making here.
The group collected on this list appears to generally hold the view that we the interwiki links are something we should be emphasizing beyond the level justified by their usage.
You agree that 1% is a rather large amount of usage, as Tim pointed out— it would compare favourably to the edit feature.
From these facts that we appear to agree on, if we were design from the perspective of promoting values we would continue to keep the links prominent, and if we were to be data-driven about usage we would also keep the links prominent.
I'm not seeing anything in your above statement to suggest why would decide not to make the links prominent.
If you're not — then why is the discussion continuing? "The usability team made a mistake, thanks are owed to X,Y,Z for catching an fixing it. Sorry we stood in the way of the fixes and implied that you couldn't, that won't happen again." Would be a fine conclusion.
'We will now factor in your critical context and emit new fully formed sausage soon' would not be.
[snip]
I mostly disagree with this. First, let me refer back to Sue's original point. The people on this list are not representative of the community of contributors. It is a self-selecting sample. That doesn't invalidate the substance of what's said on this list, but it does raise questions about claims of consensus when there seems to be a large, vocal group of people agreeing on a point. Passionate discourse on this list is a data point, but it's only one data point, and it needs to be interpreted in the context of many data points.
My prior offer to bring in a larger part of the community still stands, if you really believe that to be an issue here.
In particular, the list of people agreeing on this point includes a number of parties which seldom agree with each other. I have found that to usually be an indication of an issue which will have a fairly broad support in the larger community.
I didn't think bringing on a hundred person pile-on would help— if anything I could see it only increasing distrust of the foundation in the larger community. It would leave me in a position of having to continue to damn the foundation's actions on this and other recent issues when I'd rather just move forward...
Second, most people -- both staff and people on this list -- _are_ disconnected from the needs of the majority of the readers. None of us represent the average reader, and so we need other data points to truly understand what the average reader experiences. And frankly, only a few people in this thread seem to acknowledge this.
The very first message on this subject was requesting additional data.
I think we'd all be interested in seeing more data on the subject.
Though I don't think it's valuable to define the meaning of the data only after you've seen it. That just leads to circular justification, the criteria needs to come first.
Nor do I think we need more data in order to conclude that the promotion of the interwiki links is an important _value_ for us, and that we wish to promote it above and beyond the level justified on a pure ease of use basis.
Accordingly, I think the data which would be most helpful would be indicators that were suggesting that the existence of the exposed interwikis creates a non-trivial harm to the usability of the site, since that is the kind of evidence which would overrule the value based decision to promote the links.
That kind of outcome would be fairly surprising to me, and I think everyone else here as no one has been suggesting that such an outcome is expected.
You're essentially claiming that certain staff members are copping a "we know better" attitude. That may be true (although I honestly don't see it in this thread), and if it's happening, it's wrong. That said, people on this list are copping the exact same attitude. Let's not draw generalities about us versus them based on a few individuals. Let's instead find smarter ways to make decisions that will help us fulfill our mission. You have been one of the best at doing this, and we need more of it.
I think that drawing an equivalence between comments by the staff and community here is an error.
I expect all of us, staff and community alike, to read the comments of others, consult their best available information, and form reasoned viewpoints which they then present and defend vigorously but politely. ... and that as more information and more viewpoints become available that people will have open minds to revising their positions. This is how discussion works.
Nothing about presenting a strong argument should be construed as a fault-worthy "we know better". Even in my strongest statements I am willing to believe that I could be wrong, and I know the same to be true of many others here— It would be hard to sanely hold any other position in the presence of so many experienced and intelligent people. I apologise for sometimes allowing it to sound otherwise.
But this discussion process only works when the participants are peers.
In this case, in spite of a mostly lacking defence and the obvious super-majority holding the counter position, the site continued to display the collapsed interwikis for a prolonged duration.
It is not vigorously defended positions which I am characterizing as "we know better", it is the obvious _absence_ of those positions combined with the reality of the website (at the time I wrote the message). And this characterization is further strengthened by the use of language like "feedback" and the constant reminders of non-representativeness as a fault of the assembled community when the same could equally be applied to the UX team and the staff as a whole.
And once we have been divided into groups, those with the authority to impose without broad agreement or even solid justification, and those without... then I must answer that implicit statement of superior knowledge with the correction: Of the two, it is the community which holds the stronger claim.
What's the success rate for community-run web sites? Drawing a company vs community-run comparison like this isn't just wrong, it's pointless. The level of success of all of the top web sites is a fluke to some extent. There are great, great community sites -- both grassroots and company-run -- that will never reach the scale of Wikimedia or Google.
The success rate for Wikipedia is one for one. It exists, it is successful.
We have a successful modality... but it is one which many people would be surprised that it works at all.
Some people might think it reasonable to try to direct things towards a more traditional mode of operation— with a reasonable eye towards fixing the things we do poorly, but that would ignore the enormous number of things our mode of operation does shockingly well such as existing at all.
So, my points was that traditional organizations are fantastically unsuccessful at accomplishing what we have accomplished and so there is no reason to believe a change would be an improvement and many reasons to believe it would be an enormous failure.
We're facing huge challenges right now. We need everyone to help. The Foundation staff understands this. Most people here understand this. The question is, what's the best way to grapple these challenges?
For starters, we need to come to a better, collective understanding of what it means to come to consensus.
You sound as though you believe that to be a new issue, but it isn't. I see no reason to conclude that the challenge we face now are materially different from the ones community has been grappling with for many years... which has resulted in a multitude of more or less workable compromises.
It's not reasonable to suggest that every single design decision first be tested on foundation-l,
This isn't how our communities usually work in any case.
Bold. Revert. Discuss. is a common modality.
I think that the group assembled here would largely agree that it would have been acceptable for the UX team to make the change— even with little to no public discussion, then not interfere with the community to reverting it when non-trivial objections were raised, then engage in a discussion about the ultimate disposition of the feature.
This pattern allows a significant majority of changes to happen without significant conflict and without the impediment of excessive discussion.
For more critical changes a better process is
Discuss Poll Discuss [ Bold. Revert. Discuss. ] [Bold. Discuss. Revise.] Agree.
Iterating the sections in brackets. We've used processes like this before for other user interface changes.
Through this process we've managed to make non-trivial user interface changes without leaving popular clients out in the cold for weeks at a time.
Sometimes an inclusive process simply takes more time. But the only deadlines we're facing are the ones we impose on ourselves. We can afford some things being slower so long as progress is made, it's a reasonable price to pay.
nor would that lead to the best decisions, even if it were reasonable.
Best is the often enemy of good.
The decisions we make are so highly dimensional that the prospect of ever finding a "best" decision is a laughable joke.
Even if we agreed on a definition of best-ness the chances of actually finding the best solution to anything is infinitesimal. Usually a definition of bestness can't be found because everyone weighs different values differently.
Moreover, a locally best solution isn't desirable. I hope and expect Wikipedia to be around 100 years from now. I hope that the decisions we make today are the ones with the greatest return over that time horizon (or longer). Since no one can know what the future will ask of us, this is just more evidence that a best decision is not possible no matter who makes it.
I fully admit to being clueless. I object to people claiming that have a best solution when no one can.
I think what we generally strive for is "the best long term outcome which doesn't suck too bad in any particular way in the short term".
In any case, Wikipedia is not a good place for people who do not see the value in the inclusive decision and who are too concerned about getting what they believe to be the best result.
In terms of content editing, people who are very concerned with saying the "right" thing are usually the ones who end up blocked.
Developers (including non-staff) need permission to Be Bold. We all
We agree.
need to have permission to try things, to make mistakes, and to learn from them. If you want to draw any lessons from the top web sites on how to grow a successful site, that would be the most important one.
But being bold necessarily implies the entire community process.
If a change is imposed without the rest of the process, that's not "Being bold", thats just simply a forced change.
Sometimes forced changes are reasonable- for example, a change to keep the site operating or a reversion of a user interface change that was blocking access to a large swath of users, but don't confuse an out of process emergency action with the bold actions of the normal process.
It's not clear to me that Trevor should have reverted Aryeh's change. But isn't it remarkable that Aryeh had the power to revert in the first place?
No. This is how we work here, and it's generally how we've worked here for a almost a decade. For many of our contributors Wikipedia and this governance has existed their entire adult life, and it is only way of operating a large website that they've ever participated in.
I think it's remarkable that we've given people the authority to make changes who still find our process surprising or remarkable.
It's only remarkable to people who don't appreciate the historical context... those who view the situation as an organization being gracious enough to let its "users" modify important parts of the site.
Allow me an alternative: what we have is a community of people who have contributed millions of man hours of work towards a common goal, and in the furtherance of that goal we created an office with a staff to handle routine administrative work and the interoperation with external entities (as it's hard to deal with the borg-of-wikipedia without becoming one). We were able to do this because the public supports our goals and the efforts of the community were successful at meeting an important need of the public.
The remarkable thing is the payment you receive. Not the ability of the community to make changes to the site.
... and please don't take that wrong: I am _overjoyed_ that the Foundation has been able to employ so many thoughtful and productive people, but I think it is amazing that it is able to do so on the thin slice of the value that our collective project has produced which comes back in the form of donations.
And that the end result was that Roan re-instated the revert? If the Foundation were on some power trip, would that have been possible?
Power trip isn't something that I said, and I don't think it accurately expresses what I would have alleged.
I think there is a misunderstanding about authority here, not a power-trip. The foundation was never created to "govern" the community. It was created to keep the machines well oiled on the communities behalf, sometimes oiling the machine requires a the execution of authority, sometimes you have to take the elevator out of service in order to improve it, but usually not and usually not much. Without the communities' continued governance the sites could not run at all. The WMF staff is far too small to even dream of replacing the communities role in governance.
None the less, you need to look no further than the facebook privacy policy reversal a year ago for a counter example. Controlling and commanding organizations routinely respond to public outcry.
That it took many days to get that far is a perfect example of the foundation staff not acting as an equal peer with the community.
We have this beautiful opportunity to learn how to do good user experience collectively and at scale. No one has figured out how to do this yet, but everyone is trying. There are going to be bumps along the way. We have to continue to Assume Good Faith and encourage each other to Be Bold.
So, you're suggesting that Wikipedia became a top 10 website, reliably outperforming commercial clones like answers.com that had professional development staffs, all our content, and then some, with an acutely _bad_ user experience?
No?
I think we _had_ a beautiful opportunity to take an already effective and innovative development system and take it to the next level, but instead we tried to supplant it with the same cathedral development model plus 'feedback' which is used by most commercial web-properties.
It wasn't clear to me that what was happening since a lot of community development appears almost fully formed from a single creator or team, the distinction only became clear when the response to concerns was "accepting feedback" rather than cooperation.
The fact that my blackberry _still_ can't load Wikipedia after weeks is clear evidence that a change in how things are run has happened.
I took the time to respond to this thread, and to your message, because I'd like to clear up the confusion and restore the opportunity for cooperation as peers while it is still early enough for such a mild remedy.
Somehow, the community knows how to take the ragtag assembly of its
[snip]
with broad appeal and generally consistent, if somewhat strange, performance.
In some areas yes, in others, no. We all still have a lot to learn, and we need to learn this together.
Absolutely, the community needs to learn how to address all areas. We've been learning how to address all areas for a long time. One thing we learned a long time ago is that asserting hard-authority over the community generally generates more heat than light, that cooperation as equals is usually more productive, and that other means should only be invoked as a last resort.
This isn't the same as accepting the notion of a staff that learns how to accept feedback, and a community that needs to learn to only whine and hope for change instead of taking action. ... the impression that I and others have received here.
I think it's unfortunate that the foundation has an apparent difficulty in _contributing_ without _commanding_.
I don't see evidence of that here.
[snip]
I agree with this. Again, I don't see evidence of exclusion happening right now. I think the discourse has been positive, I think there are still some things that need to be worked out, and I think the right thing will happen in the end.
I think your inability to see what is obvious to me and many others is an example of the problem.
I'm confident that the right thing will happen in the end, my only concern is how much discomfort is on the road from here to there.
I hope my understanding of the motivations, attitudes, and plans of those involved are incorrect. But these understandings are driven by the actions of the foundation staff, they are positions I'd take with respect to anyone engaging in the same actions. They can be most easily remedied by the staff collaborating with the community as an equal parter, and not mealy as a source of optional feedback. Ten thousand words could not carry the impact of just a few actions.
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
On 7 June 2010 23:16, Howie Fung hfung@wikimedia.org wrote:
So in terms of a path forward, here is a proposal:
- Immediate revert so that all languages are exposed by default.
- We will continue work on a compromise solution. The current
interface is probably not perfect, so we’ll be continuing to look for ways to improve it. We welcome your ideas -- please direct them to [2] so we can keep track. 3. We will A/B-test proposed solutions against the default. We'll involve the community to design the A/B test (e.g., what % of traffic, what threshold we use for decisions, etc). In response to the comments about the data being only on enwp, we should be open to the fact that different Wikipedias may require different implementations.
You need also to take into account that almost 25% of the visits to enwp are from non-English speaking countries. Among these readers are users that might be recruited to read or even perhaps edit in their own language by noticing that there was a link visible. Wikipedia is more than a set of separate language projects. It is a connected set of language projects. The iw-links are what connects these projects. Hiding them on en-wp damages the other projects by cutting connections.
Hans Rosbach
On Mon, Jun 7, 2010 at 3:15 PM, Gregory Maxwell gmaxwell@gmail.com wrote:
Unfortunately, we're still able to speak about the community and the UX teams as distinct entities. This division will continue so long as the relationship is viewed in the context of "decision"/"feedback" rather than as a dialogue between peers.
. . .
This isn't how our communities usually work in any case.
Bold. Revert. Discuss. is a common modality.
I think that the group assembled here would largely agree that it would have been acceptable for the UX team to make the change— even with little to no public discussion, then not interfere with the community to reverting it when non-trivial objections were raised, then engage in a discussion about the ultimate disposition of the feature.
This pattern allows a significant majority of changes to happen without significant conflict and without the impediment of excessive discussion.
. . .
I think it's remarkable that we've given people the authority to make changes who still find our process surprising or remarkable.
I emphatically agree with all of the above.
On Mon, Jun 7, 2010 at 6:31 PM, David Levy lifeisunfair@gmail.com wrote:
Agreed. So why are you dismissing people's arguments on the basis that they stem from such judgement (while simultaneously passing similar judgement of your own)? You can't have it both ways.
I am not. It would be perfectly fine if objections to the change stemmed from judgment, provided the judgment was sound. In some cases (not all), I don't think it was. I provided arguments for why I thought the result of my own judgment was better.
To make an analogy to a more clear-cut case, suppose someone proposes a database schema change to fix a particular bug. Suppose Domas says "that wouldn't give acceptable performance". That one statement of Domas, with no further support, would outweigh quite a lot of countervailing evidence and argument, although not a limitless amount. As soon as he says that, the burden of evidence falls very strongly on anyone who disagrees. Why? Because Domas has worked professionally as a MySQL database engineer for years, and has proven by his actions as a community member that his expertise in that field is sound. His judgment outweighs other people's, because he's simply more qualified.
Now, let me point out two key differences between that case and this:
1) Domas has a track record of years of contributions in this field within the community, and his expertise is well known to anyone who's familiar with MediaWiki development. He has built a deserved reputation within the community. The usability team might be expert in usability, but typical community members can't tell, because its operations and evidence are mostly hidden in practice. (Maybe they're posted somewhere, but nobody sees them.) As such, the community will rightfully demand better explanations when it's overruled by the usability team than by Domas.
2) If a lot of people objected, some kind of clear-cut explanation for Domas' decision would be given. He might not give details to people who asked him stupid questions, but some other developers would ask him more intelligent questions if they didn't understand, and he'd answer them. Everyone qualified to understand the issue in the first place would be able to get a fairly good explanation in the end. As a result, the community would be more informed and would better understand future decisions, and their respect for Domas would increase. In this case, we ended up with no good explanations from the usability team, so no one came out any wiser, and the community has not gained any further respect for the usability team's expertise.
I don't mean to imply that my judgment here (or anyone's) counts nearly as much as Domas' judgment on MySQL issues. My point with this analogy is to show that in some cases, it's totally legitimate for someone to pass judgment while rejecting other people's judgment -- this objection of yours is not valid by itself.
It's entirely reasonable to vigorously disagree with others' arguments, of course. But when the rationale is "they are not backed by data," it's unreasonable to exempt the user experience team and yourself from this standard.
Compare to the analogy I gave above to Domas. If Domas says something about the DB, then yes, his judgment overrules almost anyone else unless they have data or other strong evidence. Not everyone's judgment counts the same. (But in this case, the usability team should be justifying its judgment explicitly and at length, so that the community comes to respect its judgment.)
As previously noted, perceived clutter draws complaints.
I cannot think of any time when I've seen someone propose that MediaWiki interface elements be removed unless they're patently useless, except for a few developers who occasionally go out of their way to cut down on clutter. If there have been any, they were probably right after the clutter was added, so people complained mostly because it changed and gave clutter as a justification.
Of course not. But for reasons explained throughout the discussion, many of us regard this feature as immensely important and feel that it should not be demoted in the absence of data indicating that the change is beneficial.
I agree with the "be bold, revert, discuss" methodology that's being employed now. However, I don't think that data in favor of a change should be an absolute requirement in the face of objections (even strong ones), unless there's data against the change. (Which there isn't really here.) The judgment of sufficiently trusted parties is a legitimate substitute for data. Obviously the community does not view the usability team as sufficiently trusted in this case, and that's fine, but I want to be clear about the general principles.
Aryeh Gregor wrote:
It would be perfectly fine if objections to the change stemmed from judgment, provided the judgment was sound. In some cases (not all), I don't think it was. I provided arguments for why I thought the result of my own judgment was better.
And that's fine (as I previously noted). But you didn't merely seek to refute arguments; you also dismissed them due to "the absence of further data." You then introduced arguments that were every bit as speculative in nature, if not more so.
If Domas says something about the DB, then yes, his judgment overrules almost anyone else unless they have data or other strong evidence. Not everyone's judgment counts the same. (But in this case, the usability team should be justifying its judgment explicitly and at length, so that the community comes to respect its judgment.)
I agree, but I don't think that anyone is challenging their expertise. Rather, we're questioning the application thereof. It appears that they made two key mistakes:
1. They assigned undue weight to the interlanguage link click rate. The statistics were gathered strictly from the English Wikipedia, which is not necessarily representative of Wikipedias in general. And even if it is, a mathematical measure of the links' usage does not convey their actual importance within the project. (As others have noted, the "edit" link also receives relatively little use, but Wikipedia could not function without it.)
This is what led them to believe that the change would not be detrimental.
2. They applied a general design principle without considering atypical circumstances that might render it inapplicable.
This is what led them to believe that the change would be beneficial.
In summary, the issue is not that the team lacks expertise; it's that it was exercised in a robotic manner.
I agree with the "be bold, revert, discuss" methodology that's being employed now. However, I don't think that data in favor of a change should be an absolute requirement in the face of objections (even strong ones), unless there's data against the change. (Which there isn't really here.)
I strongly disagree.
Firstly, much of the necessary data (such as usage statistics from non-English Wikipedias) simply hasn't been gathered.
Secondly, it's been explained that the links serve a mission-critical purpose not fully reflected in raw numbers.
Legally blind people compose a very small percentage of our users. (The rate within the U.S. population is ~0.3%, so the rate among Wikipedia users worldwide likely is similar or perhaps lower.) So an interface change theorized to "reduce clutter" for sighted users (while creating a stumbling block for the tiny number of legally blind users) could be considered mathematically justified. But from a social standpoint, it would be unacceptable.
By all accounts, the interlanguage links receive non-zero use, and there is clear consensus within the community that the importance thereof far exceeds that which can be gleaned from numbers alone.
Unless and until a net benefit (and not a purely mathematical one) is demonstrated, no reduction in the feature's prominence should occur.
David Levy
wikimedia-l@lists.wikimedia.org