[Foundation-l] Personal Image Filter results announced

[Thomas Dalton]

On 15 September 2011 07:31, phoebe ayers <phoebe.wiki at gmail.com> wrote:
> I've been away for a week offline, so am trying to catch up. I'm
> picking a random point in the thread to try and answer lots of
> questions at once, from my own viewpoint.

Thank you for this email. I'm going to pick just a few portions of it
to respond to, since a lot of what I would say has already been
covered at length by me and others.


> c) In hindsight I would have done more to clarify the role of the
> board in this process. The board didn't ask for the referendum to be
> conducted; Sue did, as part of being directed to implement the board's
> resolution. The board has naturally been sent the results, and I acted
> as board liaison to the referendum committee, and helped think through
> the questions -- but the referendum wasn't specifically a board
> project. (The board did ask for the feature to be built in the first
> place, however).

The resolution did mandate Sue to consult with the community. Sue
chose to do that using this "referendum", but she was required to do
something.

> d) In hindsight I would have made sure that we had more careful review
> of the questions for their utility as survey instruments, perhaps
> running them past the research committee. There's not much precedent
> for that, but we could start!

This is a recurring problem. The Foundation has a tendancy to do
surveys and polls without actually thinking about what they are trying
to find out and how they are going to analyse it before hand. You
should know exactly what analysis you are going to do on the answers
before you ask the questions, otherwise you have no chance of knowing
what questions to ask.

> e) The big question -- should we have asked "yes or no" or not? I
> pushed for not asking this directly because of the premise that we
> were asking for broad-scale community input on design, and because the
> board had already asked for the thing to be built, and because
> "importance" felt like a more subtle measure of where people stood. In
> hindsight, given all the controversy and the number of people who if
> they were consulted at all wanted to be asked simply yes or no, that
> was likely a mistake. People certainly made their views known in the
> comments and talk pages though, and I am glad we have that rich input.

The big problem with "importance" is that it is a relative concept and
we weren't given anything to compare it to. That makes the whole thing
meaningless.

> f) It's not a surprise to me, or the Board, that this is
> controversial; from what the referendum did measure, it seems clear
> that the community is fairly split. I am glad that we had the
> referendum though, because it did reveal that split to be bimodal and
> complex. I have reviewed a sampling of the comments, and along with
> the negatives and those opposed on practical and philosophical grounds
> there are many positives, and many arguments for why such a feature is
> needed. And remember, we did broaden the net so that both long-term
> heavy editors and occasional, mostly-reader editors had a chance to
> say their piece, which I think was a success in getting much wider and
> diverse input that we generally do just here on foundation-l or on
> meta talk pages.

The bimodality could easily be an artifact of the flawed methodology.
The lack of a "yes/no" question probably resulted in a lot of people
choosing the interpret the first question as a "yes/no" question with
1=no and 10=yes, ignoring 2-9. If you had asked a combined
yes/no-importance question (eg. "Should we do this? Yes, it's very
important/Yes, but only if it doesn't cost too much/I don't know/No,
but it won't really hurt if you do/No, it's very important that you
don't") then I would guess you wouldn't get the same level of
bimodality.