[Foundation-l] hiding interlanguage links by default is a Bad Idea, part 2

Howie Fung hfung at wikimedia.org
Mon Jun 7 21:16:50 UTC 2010

I agree that the User Experience Team and the community are still 
learning how to most effectively work together to do product 
development.  Things aren't perfect, and it make take some time before 
we get to a comfortable point.  I think the best thing we can do is to 
continue learning from each experience, and hopefully we'll come to a 
better process as we do more and more projects.  We'll make mistakes 
from time to time and that's just part of the learning process.   The 
important point is to acknowledge where we can do better and learn from 
those experiences.

Based on this feature, I think there are a number of areas where we 
could improve.  First, I do agree with the various statements that the 
tone of the revert was overly harsh and authoritarian.  I can assure you 
that that was not the intent of the revert -- we've always endeavored to 
be open in our collaboration, even in (and hopefully especially in) 
times of disagreement.  We've also been iterative in our approach, and 
decisions are rarely, if ever, considered final as the web affords us 
the luxury of continual improvement.  Nevertheless, words are important, 
and the ones used in the revert should have conveyed more openness to 
discussion, which they did not.  We'll make sure that these intentions 
are clear in the future.  We also recognize that the compromise solution 
that we proposed may not have met the needs of some within the 
community.  But I hope people understand that the shortcomings of this 
proposal did not arise from a failure to listen.  There have been many, 
many insightful and helpful comments which we took into consideration 
when developing the proposal.

Another area in which we can improve is data.  The debate on the data, 
especially what path of action the data would suggest, is a very 
legitimate debate (incidentally, I think the data may have been partly 
misinterpreted [1], but the debate is nonetheless valid).  This is a 
case where, because the data is imperfect, we will require additional 
perspectives to make a reasoned decision (someone used the phrase “life 
isn’t so simple”).  And as several posters have pointed out, the members 
on this mailing list represent a small, self-selecting sample of users, 
as do the members of the User Experience Team.  We carry our own biases 
into these decisions.  One way to overcome the inherent biases, as 
Eugene mentioned, is to be more data driven.  In this particular case, I 
am an advocate of using A/B testing as a way to better understand the 
impacts of a change in the user interface.  In general, I am very much 
in favor of having better analytics so that the balance of 
decision-making tilts even more towards data.

So in terms of a path forward, here is a proposal:
1.  Immediate revert so that all languages are exposed by default.
2.  We will continue work on a compromise solution.  The current 
interface is probably not perfect, so we’ll be continuing to look for 
ways to improve it.  We welcome your ideas -- please direct them to [2] 
so we can keep track.
3.  We will A/B-test proposed solutions against the default.  We'll 
involve the community to design the A/B test (e.g., what % of traffic, 
what threshold we use for decisions, etc).  In response to the comments 
about the data being only on enwp, we should be open to the fact that 
different Wikipedias may require different implementations.

In fairness, I should say clearly that while #1 will happen immediately, 
#2 and #3 will probably not be immediate.  Our team will need to balance 
this feature against other commitments (e.g., the next phase of the 
default rollout later this week, which will roll out with the inter-wiki 
links expanded).

 From a broader standpoint, I think there needs to be a direct 
discussion around how to involve the community in these types of 
decisions.  I’ve created a page on the usability wiki to discuss ideas 
[3] and look forward to the discussion.

Please let us know what you think of the proposed path forward.


[1] Based on the replies to the post, it seemed as though some people 
might have thought that the ~1% figure for Monobook referred to all 
clicks.  To be clear, the ~1% refers to ~1% of total left-nav clicks, 
not clicks overall.  But better data is still needed to inform this 
[2] http://usability.wikimedia.org/wiki/Interwiki_Link_Proposals
[3] http://usability.wikimedia.org/wiki/Product_Development_Process_Ideas

On 6/7/10 1:21 PM, Mark Williamson wrote:
> One major problem I have with this entire initiative, at least as I
> understand it, is that data was only collected from en.wp and mostly
> from native English speakers. Wikipedia is not monolingual, although
> many of our users are... and it's important to remember that many of
> these people are monolingual in languages other than English. Without
> further study, we have no way of knowing how many of the conclusions
> we may have drawn from the data will still hold true for
> non-English-speaking audiences.
> I had hoped that the internationalism of our organization would grow,
> rather than any sort of increasing centralization and the treatment of
> en.wp as a "flagship" project, the capital of the Wikimedia empire. It
> is not.
> -m.
> On Mon, Jun 7, 2010 at 12:15 PM, Gregory Maxwell<gmaxwell at gmail.com>  wrote:
>> On Mon, Jun 7, 2010 at 11:52 AM, Eugene Eric Kim<eekim at blueoxen.com>  wrote:
>>> On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell<gmaxwell at gmail.com>  wrote:
>>>> On Sat, Jun 5, 2010 at 2:03 PM,<susanpgardner at gmail.com>  wrote:
>>>>> Sorry for top-posting.
>>>>> Austin, think about who "everyone" is.  The folks here on foundation-l are not representative of readers.  The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
>>>> Sue, you appear to be making the assumption that the folks here are
>>>> writing from a position of their personal preferences while the
>>>> usability team is working on the behalf of the best interests of the
>>>> project.
>>>> I don't believe this comparison to be accurate.
>>> I agree that this comparison is inaccurate, but I disagree that this
>>> was Sue's assumption. All she said was that vocal outcry on a mailing
>>> list should not be construed as community consensus. I hope that no
>>> one disagrees with this.
>> "All she said", no. She didn't state that. You're putting words in her mouth.
>> Perhaps I'm guilty of the same crime. But What Sue said was
>> "The folks here on foundation-l are not representative of readers.
>> The job of the user experience team is to try to balance all readers'
>> needs, which is not easy, and will sometimes involve making decisions
>> that not everyone agrees with."
>> I read that as contrasting the purposes of the UX team and the people
>> commenting here.  I am unable to determine any other reason for
>> bringing these two statements together except for the purpose of
>> drawing the comparison I suggested was being made, even with your
>> proposed alternative.
>> We all communicate unclearly at times, — and I am more than willing to
>> accept that I saw a comparison there which was not intended.
>> In the interest of good communication I hope that you will take the
>> time to consider how I could have come to the understanding that I
>> did.   I'm sure many other people on this list had the same
>> understanding.
>> What we have here is a nearly unanimous response with respect to the
>> disposition of the interwiki links. If you'd like me to bring this to
>> the larger community I can do so but my understanding was that the
>> normal community process was already quashed with respect to this
>> change.
>> While no single forum is indicative of a consensus of the entire
>> community, the broadness of the response here is a strong indicator.
>> [snip]
>>> The bad thing is the us versus them tone in this and other messages.
>>> There is a larger question about ownership and decision-making that is
>>> subtle and hard, and we need to continue to work these out. Since I'm
>>> going to put myself in the vocal minority and disagree with most of
>>> the points in this message, I'll start with what I agree with. :-)
>> I don't believe that there is anything particularly subtle here.  We
>> have many community processes in which foundation staff are welcome to
>> contribute to as peers with a common interest.
>> When you fail to do so you have created the "us" vs "them" by your own actions.
>> Rather then trying to draw "us" vs "them" lines in the sand, I am in
>> fact pleading that the foundation discontinue doing so.
>> In order to do that I must first acknowledge the division which I
>> believe has already formed. [more on this later]
>>>> I think the people here are speaking up for the sake of the readers,
>>>> and for the sake of preserving the best of the existing design
>>>> principles used on the site.  I know I am.
>>> Absolutely. Assume good faith. I know you and many others feel the
>>> same way about the UX team
>> Absolutely.
>> I would suggest that the broader community (and not necessarily the
>> participants here) has greater experience than the usability team, and
>> even the portion of the community represented here has a more diverse
>> composition than the UX team.
>> However, if we were to combine the two— we would have something
>> strictly superior to the component parts.  Unfortunately, we're still
>> able to speak about the community and the UX teams as distinct
>> entities.  This division will continue so long as the relationship is
>> viewed in the context of "decision"/"feedback" rather than as a
>> dialogue between peers.
>>> That said, keep in mind that most people assume that everyone thinks
>>> like they do. This is not an us versus them thing; this is a natural,
>>> human thing. I heard a great tip from a psychologist once: If you want
>>> to know what people truly think, ask them what they think other people
>>> think.
>> [snip]
>> While we are on the topic of lessons in human nature, please allow me
>> to introduce the list to the fundamental attribution error:
>> http://en.wikipedia.org/wiki/Fundamental_attribution_error
>> We all may find it informative.
>>> Given this quirk of human nature, we need more rigorous ways of making
>>> decisions than polling people, especially small, self-selecting
>>> groups. Being data-driven is one of those ways. But being data-driven
>>> is hard, too, because you still have to interpret the data. Which
>>> brings me to...
>>>> I was alarmed when I heard the click rates: 1%.  That's an enormous
>>>> number of clicks, considerably higher than I expected with the large
>>>> number of things available for folks to click on.
>>> Agreed. 1% is absolutely a large number of clicks, especially given
>>> our overall traffic. Someone in a different message also pointed out
>>> that click rates alone don't tell the whole story; if they did, then
>>> one could argue that we should eliminate the Edit button.
>>> Good design isn't just about following the user path; it's also about
>>> guiding the users in a way that's appropriate to the mission of the
>>> work. In that vein, I think the substance of most of the messages in
>>> this thread have been very positive. It's resulted in data (such as
>>> Max's numbers) that have helped to round out everyone's understanding
>>> of the issue, but most importantly, it's resulted in critical context
>>> as to why people may be acting a certain way and why these things
>>> matter in the first place.
>> I do not believe that I am understanding the point you are making here.
>> The group collected on this list appears to generally hold the view
>> that we the interwiki links are something we should be emphasizing
>> beyond the level justified by their usage.
>> You agree that 1% is a rather large amount of usage, as Tim pointed
>> out— it would compare favourably to the edit feature.
>>  From these facts that we appear to agree on, if we were design from
>> the perspective of promoting values we would continue to keep the
>> links prominent, and if we were to be data-driven about usage we would
>> also keep the links prominent.
>> I'm not seeing anything in your above statement to suggest why would
>> decide not to make the links prominent.
>> If you're not — then why is the discussion continuing?  "The usability
>> team made a mistake, thanks are owed to X,Y,Z for catching an fixing
>> it. Sorry we stood in the way of the fixes and implied that you
>> couldn't, that won't happen again." Would be a fine conclusion.
>> 'We will now factor in your critical context and emit new fully formed
>> sausage soon' would not be.
>> [snip]
>>> I mostly disagree with this. First, let me refer back to Sue's
>>> original point. The people on this list are not representative of the
>>> community of contributors. It is a self-selecting sample. That doesn't
>>> invalidate the substance of what's said on this list, but it does
>>> raise questions about claims of consensus when there seems to be a
>>> large, vocal group of people agreeing on a point. Passionate discourse
>>> on this list is a data point, but it's only one data point, and it
>>> needs to be interpreted in the context of many data points.
>> My prior offer to bring in a larger part of the community still
>> stands, if you really believe that to be an issue here.
>> In particular, the list of people agreeing on this point includes a
>> number of parties which seldom agree with each other. I have found
>> that to usually be an indication of an issue which will have a fairly
>> broad support in the larger community.
>> I didn't think bringing on a hundred person pile-on would help— if
>> anything I could see it only increasing distrust of the foundation in
>> the larger community.   It would leave me in a position of having to
>> continue to damn the foundation's actions on this and other recent
>> issues when I'd rather just move forward...
>>> Second, most people -- both staff and people on this list -- _are_
>>> disconnected from the needs of the majority of the readers. None of us
>>> represent the average reader, and so we need other data points to
>>> truly understand what the average reader experiences. And frankly,
>>> only a few people in this thread seem to acknowledge this.
>> The very first message on this subject was requesting additional data.
>> I think we'd all be interested in seeing more data on the subject.
>> Though I don't think it's valuable to define the meaning of the data
>> only after you've seen it. That just leads to circular justification,
>> the criteria needs to come first.
>> Nor do I think we need more data in order to conclude that the
>> promotion of the interwiki links is an important _value_ for us, and
>> that we wish to promote it above and beyond the level justified on a
>> pure ease of use basis.
>> Accordingly, I think the data which would be most helpful would be
>> indicators that were suggesting that the existence of the exposed
>> interwikis creates a non-trivial harm to the usability of the site,
>> since that is the kind of evidence which would overrule the value
>> based decision to promote the links.
>> That kind of outcome would be fairly surprising to me, and I think
>> everyone else here as no one has been suggesting that such an outcome
>> is expected.
>>> You're essentially claiming that certain staff members are copping a
>>> "we know better" attitude. That may be true (although I honestly don't
>>> see it in this thread), and if it's happening, it's wrong. That said,
>>> people on this list are copping the exact same attitude. Let's not
>>> draw generalities about us versus them based on a few individuals.
>>> Let's instead find smarter ways to make decisions that will help us
>>> fulfill our mission. You have been one of the best at doing this, and
>>> we need more of it.
>> I think that drawing an equivalence between comments by the staff and
>> community here is an error.
>> I expect all of us, staff and community alike, to read the comments of
>> others, consult their best available information, and form reasoned
>> viewpoints which they then present and defend vigorously but politely.
>> ... and that as more information and more viewpoints become available
>> that people will have open minds to revising their positions.   This
>> is how discussion works.
>> Nothing about presenting a strong argument should be construed as a
>> fault-worthy "we know better".  Even in my strongest statements I am
>> willing to believe that I could be wrong, and I know the same to be
>> true of many others here—   It would be hard to sanely hold any other
>> position in the presence of so many experienced and intelligent
>> people.  I apologise for sometimes allowing it to sound otherwise.
>> But this discussion process only works when the participants are peers.
>> In this case, in spite of a mostly lacking defence and the obvious
>> super-majority holding the counter position, the site continued to
>> display the collapsed interwikis for a prolonged duration.
>> It is not vigorously defended positions which I am characterizing as
>> "we know better",  it is the obvious _absence_ of those positions
>> combined with the reality of the website (at the time I wrote the
>> message). And this characterization is further strengthened by the use
>> of language like "feedback" and the constant reminders of
>> non-representativeness as a fault of the assembled community when the
>> same could equally be applied to the UX team and the staff as a whole.
>> And once we have been divided into groups, those with the authority to
>> impose without broad agreement or even solid justification, and those
>> without... then I must answer that implicit statement of superior
>> knowledge with the correction: Of the two, it is the community which
>> holds the stronger claim.
>>> What's the success rate for community-run web sites?
>>> Drawing a company vs community-run comparison like this isn't just
>>> wrong, it's pointless. The level of success of all of the top web
>>> sites is a fluke to some extent. There are great, great community
>>> sites -- both grassroots and company-run -- that will never reach the
>>> scale of Wikimedia or Google.
>> The success rate for Wikipedia is one for one. It exists, it is successful.
>> We have a successful modality... but it is one which many people would
>> be surprised that it works at all.
>> Some people might think it reasonable to try to direct things towards
>> a more traditional mode of operation— with a reasonable eye towards
>> fixing the things we do poorly, but that would ignore the enormous
>> number of things our mode of operation does shockingly well such as
>> existing at all.
>> So, my points was that traditional organizations are fantastically
>> unsuccessful at accomplishing what we have accomplished and so there
>> is no reason to believe a change would be an improvement and many
>> reasons to believe it would be an enormous failure.
>>> We're facing huge challenges right now. We need everyone to help. The
>>> Foundation staff understands this. Most people here understand this.
>>> The question is, what's the best way to grapple these challenges?
>>> For starters, we need to come to a better, collective understanding of
>>> what it means to come to consensus.
>> You sound as though you believe that to be a new issue, but it isn't.
>> I see no reason to conclude that the challenge we face now are
>> materially different from the ones community has been grappling with
>> for many years... which has resulted in a multitude of more or less
>> workable compromises.
>>> It's not reasonable to suggest
>>> that every single design decision first be tested on foundation-l,
>> This isn't how our communities usually work in any case.
>> Bold. Revert. Discuss.  is a common modality.
>> I think that the group assembled here would largely agree that it
>> would have been acceptable for the UX team to make the change— even
>> with little to no public discussion, then not interfere with the
>> community to reverting it when non-trivial objections were raised,
>> then engage in a discussion about the ultimate disposition of the
>> feature.
>> This pattern allows a significant majority of changes to happen
>> without significant conflict and without the impediment of excessive
>> discussion.
>> For more critical changes a better process is
>> Discuss Poll Discuss [ Bold. Revert. Discuss. ] [Bold. Discuss.
>> Revise.]  Agree.
>> Iterating the sections in brackets. We've used processes like this
>> before for other user interface changes.
>> Through this process we've managed to make non-trivial user interface
>> changes without leaving popular clients out in the cold for weeks at a
>> time.
>> Sometimes an inclusive process simply takes more time.  But the only
>> deadlines we're facing are the ones we impose on ourselves. We can
>> afford some things being slower so long as progress is made, it's a
>> reasonable price to pay.
>>> nor
>>> would that lead to the best decisions, even if it were reasonable.
>> Best is the often enemy of good.
>> The decisions we make are so highly dimensional that the prospect of
>> ever finding a "best" decision is a laughable joke.
>> Even if we agreed on a definition of best-ness the chances of actually
>> finding the best solution to anything is infinitesimal.   Usually a
>> definition of bestness can't be found because everyone weighs
>> different values differently.
>> Moreover, a locally best solution isn't desirable.   I hope and expect
>> Wikipedia to be around 100 years from now. I hope that the decisions
>> we make today are the ones with the greatest return over that time
>> horizon (or longer).  Since no one can know what the future will ask
>> of us, this is just more evidence that a best decision is not possible
>> no matter who makes it.
>> I fully admit to being clueless. I object to people claiming that have
>> a best solution when no one can.
>> I think what we generally strive for is  "the best long term outcome
>> which doesn't suck too bad in any particular way in the short term".
>> In any case, Wikipedia is not a good place for people who do not see
>> the value in the inclusive decision and who are too concerned about
>> getting what they believe to be the best result.
>> In terms of content editing, people who are very concerned with saying
>> the "right" thing are usually the ones who end up blocked.
>>> Developers (including non-staff) need permission to Be Bold. We all
>> We agree.
>>> need to have permission to try things, to make mistakes, and to learn
>>> from them. If you want to draw any lessons from the top web sites on
>>> how to grow a successful site, that would be the most important one.
>> But being bold necessarily implies the entire community process.
>> If a change is imposed without the rest of the process, that's not
>> "Being bold", thats just simply a forced change.
>> Sometimes forced changes are reasonable- for example, a change to keep
>> the site operating or a reversion of a user interface change that was
>> blocking access to a large swath of users, but don't confuse an out of
>> process emergency action with the bold actions of the normal process.
>>> It's not clear to me that Trevor should have reverted Aryeh's change.
>>> But isn't it remarkable that Aryeh had the power to revert in the
>>> first place?
>> No. This is how we work here, and it's generally how we've worked here
>> for a almost a decade. For many of our contributors Wikipedia and this
>> governance has existed their entire adult life, and it is only way of
>> operating a large website that they've ever participated in.
>> I think it's remarkable that we've given people the authority to make
>> changes who still find our process surprising or remarkable.
>> It's only remarkable to people who don't appreciate the historical
>> context... those who view the situation as an organization being
>> gracious enough to let its "users" modify important parts of the site.
>> Allow me an alternative:  what we have is a community of people who
>> have contributed millions of man hours of work towards a common goal,
>> and in the furtherance of that goal we created an office with a staff
>> to handle routine administrative work and the interoperation with
>> external entities (as it's hard to deal with the borg-of-wikipedia
>> without becoming one).  We were able to do this because the public
>> supports our goals and the efforts of the community were successful at
>> meeting an important need of the public.
>> The remarkable thing is the payment you receive. Not the ability of
>> the community to make changes to the site.
>> ... and please don't take that wrong:  I am _overjoyed_ that the
>> Foundation has been able to employ so many thoughtful and productive
>> people, but I think it is amazing that it is able to do so on the thin
>> slice of the value that our collective project has produced which
>> comes back in the form of donations.
>>> And that the end result was that Roan re-instated the
>>> revert? If the Foundation were on some power trip, would that have
>>> been possible?
>> Power trip isn't something that I said, and I don't think it
>> accurately expresses what I would have alleged.
>> I think there is a misunderstanding about authority here, not a
>> power-trip.  The foundation was never created to "govern" the
>> community.  It was created to keep the machines well oiled on the
>> communities behalf, sometimes oiling the machine requires a the
>> execution of authority, sometimes you have to take the elevator out of
>> service in order to improve it, but usually not and usually not much.
>>   Without the communities' continued governance the sites could not run
>> at all.  The WMF staff is far too small to even dream of replacing the
>> communities role in governance.
>> None the less, you need to look no further than the facebook privacy
>> policy reversal a year ago for a counter example.  Controlling and
>> commanding organizations routinely respond to public outcry.
>> That it took many days to get that far is a perfect example of the
>> foundation staff not acting as an equal peer with the community.
>>> We have this beautiful opportunity to learn how to do good user
>>> experience collectively and at scale. No one has figured out how to do
>>> this yet, but everyone is trying. There are going to be bumps along
>>> the way. We have to continue to Assume Good Faith and encourage each
>>> other to Be Bold.
>> So, you're suggesting that Wikipedia became a top 10 website, reliably
>> outperforming commercial clones like answers.com that had professional
>> development staffs, all our content, and then some,  with an acutely
>> _bad_ user experience?
>> No?
>> I think we _had_ a beautiful opportunity to take an already effective
>> and innovative development system and take it to the next level,  but
>> instead we tried to supplant it with the same cathedral development
>> model plus 'feedback' which is used by most commercial web-properties.
>> It wasn't clear to me that what was happening since a lot of community
>> development appears almost fully formed from a single creator or team,
>>   the distinction only became clear when the response to concerns was
>> "accepting feedback" rather than cooperation.
>> The fact that my blackberry _still_ can't load Wikipedia after weeks
>> is clear evidence that a change in how things are run has happened.
>> I took the time to respond to this thread, and to your message,
>> because I'd like to clear up the confusion and restore the opportunity
>> for cooperation as peers while it is still early enough for such a
>> mild remedy.
>>>> Somehow, the community knows how to take the ragtag assembly of its
>> [snip]
>>>> with broad appeal and generally consistent, if somewhat strange,
>>>> performance.
>>> In some areas yes, in others, no. We all still have a lot to learn,
>>> and we need to learn this together.
>> Absolutely, the community needs to learn how to address all areas.
>> We've been learning how to address all areas for a long time.  One
>> thing we learned a long time ago is that asserting hard-authority over
>> the community generally generates more heat than light, that
>> cooperation as equals is usually more productive, and that other means
>> should only be invoked as a last resort.
>> This isn't the same as accepting the notion of a staff that learns how
>> to accept feedback, and a community that needs to learn to only whine
>> and hope for change instead of taking action. ... the impression that
>> I and others have received here.
>>>> I think it's unfortunate that the foundation has an apparent
>>>> difficulty in _contributing_ without _commanding_.
>>> I don't see evidence of that here.
>> [snip]
>>> I agree with this. Again, I don't see evidence of exclusion happening
>>> right now. I think the discourse has been positive, I think there are
>>> still some things that need to be worked out, and I think the right
>>> thing will happen in the end.
>> I think your inability to see what is obvious to me and many others is
>> an example of the problem.
>> I'm confident that the right thing will happen in the end, my only
>> concern is how much discomfort is on the road from here to there.
>> I hope my understanding of the motivations, attitudes, and plans of
>> those involved are incorrect.  But these understandings are driven by
>> the actions of the foundation staff, they are positions I'd take with
>> respect to anyone engaging in the same actions.  They can be most
>> easily remedied by the staff collaborating with the community as an
>> equal parter, and not mealy as a source of optional feedback.  Ten
>> thousand words could not carry the impact of just a few actions.
>> _______________________________________________
>> foundation-l mailing list
>> foundation-l at lists.wikimedia.org
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l

More information about the wikimedia-l mailing list