[Foundation-l] hiding interlanguage links by default is a Bad Idea, part 2

[Mark Williamson]

One major problem I have with this entire initiative, at least as I
understand it, is that data was only collected from en.wp and mostly
from native English speakers. Wikipedia is not monolingual, although
many of our users are... and it's important to remember that many of
these people are monolingual in languages other than English. Without
further study, we have no way of knowing how many of the conclusions
we may have drawn from the data will still hold true for
non-English-speaking audiences.

I had hoped that the internationalism of our organization would grow,
rather than any sort of increasing centralization and the treatment of
en.wp as a "flagship" project, the capital of the Wikimedia empire. It
is not.

-m.


On Mon, Jun 7, 2010 at 12:15 PM, Gregory Maxwell <gmaxwell at gmail.com> wrote:
> On Mon, Jun 7, 2010 at 11:52 AM, Eugene Eric Kim <eekim at blueoxen.com> wrote:
>> On Sat, Jun 5, 2010 at 1:00 PM, Gregory Maxwell <gmaxwell at gmail.com> wrote:
>>> On Sat, Jun 5, 2010 at 2:03 PM,  <susanpgardner at gmail.com> wrote:
>>>> Sorry for top-posting.
>>>>
>>>> Austin, think about who "everyone" is.  The folks here on foundation-l are not representative of readers.  The job of the user experience team is to try to balance all readers' needs, which is not easy, and will sometimes involve making decisions that not everyone agrees with. People here have given some useful input, but I think it's far from obvious that the user experience team has made a "mistake.". (I'm not really intending to weigh in on this particular issue -- I'm speaking generally.)
>>>
>>> Sue, you appear to be making the assumption that the folks here are
>>> writing from a position of their personal preferences while the
>>> usability team is working on the behalf of the best interests of the
>>> project.
>>>
>>> I don't believe this comparison to be accurate.
>>
>> I agree that this comparison is inaccurate, but I disagree that this
>> was Sue's assumption. All she said was that vocal outcry on a mailing
>> list should not be construed as community consensus. I hope that no
>> one disagrees with this.
>
> "All she said", no. She didn't state that. You're putting words in her mouth.
>
> Perhaps I'm guilty of the same crime. But What Sue said was
>
> "The folks here on foundation-l are not representative of readers.
> The job of the user experience team is to try to balance all readers'
> needs, which is not easy, and will sometimes involve making decisions
> that not everyone agrees with."
>
> I read that as contrasting the purposes of the UX team and the people
> commenting here.  I am unable to determine any other reason for
> bringing these two statements together except for the purpose of
> drawing the comparison I suggested was being made, even with your
> proposed alternative.
>
> We all communicate unclearly at times, — and I am more than willing to
> accept that I saw a comparison there which was not intended.
>
> In the interest of good communication I hope that you will take the
> time to consider how I could have come to the understanding that I
> did.   I'm sure many other people on this list had the same
> understanding.
>
> What we have here is a nearly unanimous response with respect to the
> disposition of the interwiki links. If you'd like me to bring this to
> the larger community I can do so but my understanding was that the
> normal community process was already quashed with respect to this
> change.
>
> While no single forum is indicative of a consensus of the entire
> community, the broadness of the response here is a strong indicator.
>
>
> [snip]
>> The bad thing is the us versus them tone in this and other messages.
>> There is a larger question about ownership and decision-making that is
>> subtle and hard, and we need to continue to work these out. Since I'm
>> going to put myself in the vocal minority and disagree with most of
>> the points in this message, I'll start with what I agree with. :-)
>
> I don't believe that there is anything particularly subtle here.  We
> have many community processes in which foundation staff are welcome to
> contribute to as peers with a common interest.
>
> When you fail to do so you have created the "us" vs "them" by your own actions.
>
> Rather then trying to draw "us" vs "them" lines in the sand, I am in
> fact pleading that the foundation discontinue doing so.
>
> In order to do that I must first acknowledge the division which I
> believe has already formed. [more on this later]
>
>>> I think the people here are speaking up for the sake of the readers,
>>> and for the sake of preserving the best of the existing design
>>> principles used on the site.  I know I am.
>>
>> Absolutely. Assume good faith. I know you and many others feel the
>> same way about the UX team
>
> Absolutely.
>
> I would suggest that the broader community (and not necessarily the
> participants here) has greater experience than the usability team, and
> even the portion of the community represented here has a more diverse
> composition than the UX team.
>
> However, if we were to combine the two— we would have something
> strictly superior to the component parts.  Unfortunately, we're still
> able to speak about the community and the UX teams as distinct
> entities.  This division will continue so long as the relationship is
> viewed in the context of "decision"/"feedback" rather than as a
> dialogue between peers.
>
>
>> That said, keep in mind that most people assume that everyone thinks
>> like they do. This is not an us versus them thing; this is a natural,
>> human thing. I heard a great tip from a psychologist once: If you want
>> to know what people truly think, ask them what they think other people
>> think.
> [snip]
>
>
> While we are on the topic of lessons in human nature, please allow me
> to introduce the list to the fundamental attribution error:
> http://en.wikipedia.org/wiki/Fundamental_attribution_error
>
> We all may find it informative.
>
>
>> Given this quirk of human nature, we need more rigorous ways of making
>> decisions than polling people, especially small, self-selecting
>> groups. Being data-driven is one of those ways. But being data-driven
>> is hard, too, because you still have to interpret the data. Which
>> brings me to...
>>
>>> I was alarmed when I heard the click rates: 1%.  That's an enormous
>>> number of clicks, considerably higher than I expected with the large
>>> number of things available for folks to click on.
>>
>> Agreed. 1% is absolutely a large number of clicks, especially given
>> our overall traffic. Someone in a different message also pointed out
>> that click rates alone don't tell the whole story; if they did, then
>> one could argue that we should eliminate the Edit button.
>>
>> Good design isn't just about following the user path; it's also about
>> guiding the users in a way that's appropriate to the mission of the
>> work. In that vein, I think the substance of most of the messages in
>> this thread have been very positive. It's resulted in data (such as
>> Max's numbers) that have helped to round out everyone's understanding
>> of the issue, but most importantly, it's resulted in critical context
>> as to why people may be acting a certain way and why these things
>> matter in the first place.
>
>
> I do not believe that I am understanding the point you are making here.
>
>
> The group collected on this list appears to generally hold the view
> that we the interwiki links are something we should be emphasizing
> beyond the level justified by their usage.
>
> You agree that 1% is a rather large amount of usage, as Tim pointed
> out— it would compare favourably to the edit feature.
>
> From these facts that we appear to agree on, if we were design from
> the perspective of promoting values we would continue to keep the
> links prominent, and if we were to be data-driven about usage we would
> also keep the links prominent.
>
> I'm not seeing anything in your above statement to suggest why would
> decide not to make the links prominent.
>
> If you're not — then why is the discussion continuing?  "The usability
> team made a mistake, thanks are owed to X,Y,Z for catching an fixing
> it. Sorry we stood in the way of the fixes and implied that you
> couldn't, that won't happen again." Would be a fine conclusion.
>
> 'We will now factor in your critical context and emit new fully formed
> sausage soon' would not be.
>
>
> [snip]
>> I mostly disagree with this. First, let me refer back to Sue's
>> original point. The people on this list are not representative of the
>> community of contributors. It is a self-selecting sample. That doesn't
>> invalidate the substance of what's said on this list, but it does
>> raise questions about claims of consensus when there seems to be a
>> large, vocal group of people agreeing on a point. Passionate discourse
>> on this list is a data point, but it's only one data point, and it
>> needs to be interpreted in the context of many data points.
>
> My prior offer to bring in a larger part of the community still
> stands, if you really believe that to be an issue here.
>
> In particular, the list of people agreeing on this point includes a
> number of parties which seldom agree with each other. I have found
> that to usually be an indication of an issue which will have a fairly
> broad support in the larger community.
>
> I didn't think bringing on a hundred person pile-on would help— if
> anything I could see it only increasing distrust of the foundation in
> the larger community.   It would leave me in a position of having to
> continue to damn the foundation's actions on this and other recent
> issues when I'd rather just move forward...
>
>
>> Second, most people -- both staff and people on this list -- _are_
>> disconnected from the needs of the majority of the readers. None of us
>> represent the average reader, and so we need other data points to
>> truly understand what the average reader experiences. And frankly,
>> only a few people in this thread seem to acknowledge this.
>
> The very first message on this subject was requesting additional data.
>
> I think we'd all be interested in seeing more data on the subject.
>
> Though I don't think it's valuable to define the meaning of the data
> only after you've seen it. That just leads to circular justification,
> the criteria needs to come first.
>
> Nor do I think we need more data in order to conclude that the
> promotion of the interwiki links is an important _value_ for us, and
> that we wish to promote it above and beyond the level justified on a
> pure ease of use basis.
>
> Accordingly, I think the data which would be most helpful would be
> indicators that were suggesting that the existence of the exposed
> interwikis creates a non-trivial harm to the usability of the site,
> since that is the kind of evidence which would overrule the value
> based decision to promote the links.
>
> That kind of outcome would be fairly surprising to me, and I think
> everyone else here as no one has been suggesting that such an outcome
> is expected.
>
>> You're essentially claiming that certain staff members are copping a
>> "we know better" attitude. That may be true (although I honestly don't
>> see it in this thread), and if it's happening, it's wrong. That said,
>> people on this list are copping the exact same attitude. Let's not
>> draw generalities about us versus them based on a few individuals.
>> Let's instead find smarter ways to make decisions that will help us
>> fulfill our mission. You have been one of the best at doing this, and
>> we need more of it.
>
> I think that drawing an equivalence between comments by the staff and
> community here is an error.
>
> I expect all of us, staff and community alike, to read the comments of
> others, consult their best available information, and form reasoned
> viewpoints which they then present and defend vigorously but politely.
> ... and that as more information and more viewpoints become available
> that people will have open minds to revising their positions.   This
> is how discussion works.
>
> Nothing about presenting a strong argument should be construed as a
> fault-worthy "we know better".  Even in my strongest statements I am
> willing to believe that I could be wrong, and I know the same to be
> true of many others here—   It would be hard to sanely hold any other
> position in the presence of so many experienced and intelligent
> people.  I apologise for sometimes allowing it to sound otherwise.
>
> But this discussion process only works when the participants are peers.
>
> In this case, in spite of a mostly lacking defence and the obvious
> super-majority holding the counter position, the site continued to
> display the collapsed interwikis for a prolonged duration.
>
> It is not vigorously defended positions which I am characterizing as
> "we know better",  it is the obvious _absence_ of those positions
> combined with the reality of the website (at the time I wrote the
> message). And this characterization is further strengthened by the use
> of language like "feedback" and the constant reminders of
> non-representativeness as a fault of the assembled community when the
> same could equally be applied to the UX team and the staff as a whole.
>
> And once we have been divided into groups, those with the authority to
> impose without broad agreement or even solid justification, and those
> without... then I must answer that implicit statement of superior
> knowledge with the correction: Of the two, it is the community which
> holds the stronger claim.
>
>> What's the success rate for community-run web sites?
>> Drawing a company vs community-run comparison like this isn't just
>> wrong, it's pointless. The level of success of all of the top web
>> sites is a fluke to some extent. There are great, great community
>> sites -- both grassroots and company-run -- that will never reach the
>> scale of Wikimedia or Google.
>
> The success rate for Wikipedia is one for one. It exists, it is successful.
>
> We have a successful modality... but it is one which many people would
> be surprised that it works at all.
>
> Some people might think it reasonable to try to direct things towards
> a more traditional mode of operation— with a reasonable eye towards
> fixing the things we do poorly, but that would ignore the enormous
> number of things our mode of operation does shockingly well such as
> existing at all.
>
> So, my points was that traditional organizations are fantastically
> unsuccessful at accomplishing what we have accomplished and so there
> is no reason to believe a change would be an improvement and many
> reasons to believe it would be an enormous failure.
>
>> We're facing huge challenges right now. We need everyone to help. The
>> Foundation staff understands this. Most people here understand this.
>> The question is, what's the best way to grapple these challenges?
>>
>> For starters, we need to come to a better, collective understanding of
>> what it means to come to consensus.
>
> You sound as though you believe that to be a new issue, but it isn't.
> I see no reason to conclude that the challenge we face now are
> materially different from the ones community has been grappling with
> for many years... which has resulted in a multitude of more or less
> workable compromises.
>
>> It's not reasonable to suggest
>> that every single design decision first be tested on foundation-l,
>
> This isn't how our communities usually work in any case.
>
> Bold. Revert. Discuss.  is a common modality.
>
> I think that the group assembled here would largely agree that it
> would have been acceptable for the UX team to make the change— even
> with little to no public discussion, then not interfere with the
> community to reverting it when non-trivial objections were raised,
> then engage in a discussion about the ultimate disposition of the
> feature.
>
> This pattern allows a significant majority of changes to happen
> without significant conflict and without the impediment of excessive
> discussion.
>
> For more critical changes a better process is
>
> Discuss Poll Discuss [ Bold. Revert. Discuss. ] [Bold. Discuss.
> Revise.]  Agree.
>
> Iterating the sections in brackets. We've used processes like this
> before for other user interface changes.
>
> Through this process we've managed to make non-trivial user interface
> changes without leaving popular clients out in the cold for weeks at a
> time.
>
> Sometimes an inclusive process simply takes more time.  But the only
> deadlines we're facing are the ones we impose on ourselves. We can
> afford some things being slower so long as progress is made, it's a
> reasonable price to pay.
>
>
>> nor
>> would that lead to the best decisions, even if it were reasonable.
>
> Best is the often enemy of good.
>
> The decisions we make are so highly dimensional that the prospect of
> ever finding a "best" decision is a laughable joke.
>
> Even if we agreed on a definition of best-ness the chances of actually
> finding the best solution to anything is infinitesimal.   Usually a
> definition of bestness can't be found because everyone weighs
> different values differently.
>
> Moreover, a locally best solution isn't desirable.   I hope and expect
> Wikipedia to be around 100 years from now. I hope that the decisions
> we make today are the ones with the greatest return over that time
> horizon (or longer).  Since no one can know what the future will ask
> of us, this is just more evidence that a best decision is not possible
> no matter who makes it.
>
> I fully admit to being clueless. I object to people claiming that have
> a best solution when no one can.
>
> I think what we generally strive for is  "the best long term outcome
> which doesn't suck too bad in any particular way in the short term".
>
> In any case, Wikipedia is not a good place for people who do not see
> the value in the inclusive decision and who are too concerned about
> getting what they believe to be the best result.
>
> In terms of content editing, people who are very concerned with saying
> the "right" thing are usually the ones who end up blocked.
>
>> Developers (including non-staff) need permission to Be Bold. We all
>
> We agree.
>
>> need to have permission to try things, to make mistakes, and to learn
>> from them. If you want to draw any lessons from the top web sites on
>> how to grow a successful site, that would be the most important one.
>
> But being bold necessarily implies the entire community process.
>
> If a change is imposed without the rest of the process, that's not
> "Being bold", thats just simply a forced change.
>
> Sometimes forced changes are reasonable- for example, a change to keep
> the site operating or a reversion of a user interface change that was
> blocking access to a large swath of users, but don't confuse an out of
> process emergency action with the bold actions of the normal process.
>
>> It's not clear to me that Trevor should have reverted Aryeh's change.
>> But isn't it remarkable that Aryeh had the power to revert in the
>> first place?
>
> No. This is how we work here, and it's generally how we've worked here
> for a almost a decade. For many of our contributors Wikipedia and this
> governance has existed their entire adult life, and it is only way of
> operating a large website that they've ever participated in.
>
> I think it's remarkable that we've given people the authority to make
> changes who still find our process surprising or remarkable.
>
> It's only remarkable to people who don't appreciate the historical
> context... those who view the situation as an organization being
> gracious enough to let its "users" modify important parts of the site.
>
> Allow me an alternative:  what we have is a community of people who
> have contributed millions of man hours of work towards a common goal,
> and in the furtherance of that goal we created an office with a staff
> to handle routine administrative work and the interoperation with
> external entities (as it's hard to deal with the borg-of-wikipedia
> without becoming one).  We were able to do this because the public
> supports our goals and the efforts of the community were successful at
> meeting an important need of the public.
>
> The remarkable thing is the payment you receive. Not the ability of
> the community to make changes to the site.
>
> ... and please don't take that wrong:  I am _overjoyed_ that the
> Foundation has been able to employ so many thoughtful and productive
> people, but I think it is amazing that it is able to do so on the thin
> slice of the value that our collective project has produced which
> comes back in the form of donations.
>
>
>> And that the end result was that Roan re-instated the
>> revert? If the Foundation were on some power trip, would that have
>> been possible?
>
> Power trip isn't something that I said, and I don't think it
> accurately expresses what I would have alleged.
>
> I think there is a misunderstanding about authority here, not a
> power-trip.  The foundation was never created to "govern" the
> community.  It was created to keep the machines well oiled on the
> communities behalf, sometimes oiling the machine requires a the
> execution of authority, sometimes you have to take the elevator out of
> service in order to improve it, but usually not and usually not much.
>  Without the communities' continued governance the sites could not run
> at all.  The WMF staff is far too small to even dream of replacing the
> communities role in governance.
>
> None the less, you need to look no further than the facebook privacy
> policy reversal a year ago for a counter example.  Controlling and
> commanding organizations routinely respond to public outcry.
>
> That it took many days to get that far is a perfect example of the
> foundation staff not acting as an equal peer with the community.
>
>> We have this beautiful opportunity to learn how to do good user
>> experience collectively and at scale. No one has figured out how to do
>> this yet, but everyone is trying. There are going to be bumps along
>> the way. We have to continue to Assume Good Faith and encourage each
>> other to Be Bold.
>
> So, you're suggesting that Wikipedia became a top 10 website, reliably
> outperforming commercial clones like answers.com that had professional
> development staffs, all our content, and then some,  with an acutely
> _bad_ user experience?
>
> No?
>
> I think we _had_ a beautiful opportunity to take an already effective
> and innovative development system and take it to the next level,  but
> instead we tried to supplant it with the same cathedral development
> model plus 'feedback' which is used by most commercial web-properties.
>
> It wasn't clear to me that what was happening since a lot of community
> development appears almost fully formed from a single creator or team,
>  the distinction only became clear when the response to concerns was
> "accepting feedback" rather than cooperation.
>
> The fact that my blackberry _still_ can't load Wikipedia after weeks
> is clear evidence that a change in how things are run has happened.
>
> I took the time to respond to this thread, and to your message,
> because I'd like to clear up the confusion and restore the opportunity
> for cooperation as peers while it is still early enough for such a
> mild remedy.
>
>>> Somehow, the community knows how to take the ragtag assembly of its
> [snip]
>>> with broad appeal and generally consistent, if somewhat strange,
>>> performance.
>>
>> In some areas yes, in others, no. We all still have a lot to learn,
>> and we need to learn this together.
>
> Absolutely, the community needs to learn how to address all areas.
> We've been learning how to address all areas for a long time.  One
> thing we learned a long time ago is that asserting hard-authority over
> the community generally generates more heat than light, that
> cooperation as equals is usually more productive, and that other means
> should only be invoked as a last resort.
>
> This isn't the same as accepting the notion of a staff that learns how
> to accept feedback, and a community that needs to learn to only whine
> and hope for change instead of taking action. ... the impression that
> I and others have received here.
>
>>> I think it's unfortunate that the foundation has an apparent
>>> difficulty in _contributing_ without _commanding_.
>>
>> I don't see evidence of that here.
> [snip]
>> I agree with this. Again, I don't see evidence of exclusion happening
>> right now. I think the discourse has been positive, I think there are
>> still some things that need to be worked out, and I think the right
>> thing will happen in the end.
>
> I think your inability to see what is obvious to me and many others is
> an example of the problem.
>
> I'm confident that the right thing will happen in the end, my only
> concern is how much discomfort is on the road from here to there.
>
> I hope my understanding of the motivations, attitudes, and plans of
> those involved are incorrect.  But these understandings are driven by
> the actions of the foundation staff, they are positions I'd take with
> respect to anyone engaging in the same actions.  They can be most
> easily remedied by the staff collaborating with the community as an
> equal parter, and not mealy as a source of optional feedback.  Ten
> thousand words could not carry the impact of just a few actions.
>
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>