[Foundation-l] Editor Survey, 2011

[MZMcBride]

Mani Pande wrote:
> It is with great pleasure that I would like to inform you that we are in
> the process of the launching Wikipedia's second editor survey. The
> survey is a redo of the UNU-Merit Survey that the foundation had
> conducted last year. The survey covers a variety of topics, but its
> primary goal is to understand the needs and participation of the editing
> community. You can read more about the objectives of the survey in the
> FAQ section in strategy wiki.  The survey will launch in the first week
> of April.

I looked at the survey that's currently being translated:
http://meta.wikimedia.org/wiki/Wikipedia_Editors_Survey_2011/Translation/en

How was this survey written? Who wrote it? Do the author(s) have any
background in surveying?

It seems very long and overly complex. Some of the questions it asks seem
unnecessary given that you could query the information based on the user
providing the input. Will the username of the person being surveyed be
recorded and stored? If so, for how long? (I know you talked about releasing
only anonymized data, but I imagine plenty of people would like to know how
many people such as sysadmins or those administering the survey will have
access to this data and for how long, if it's being stored at all.)

The survey also seems to use some language that won't translate very easily
(if at all) into other languages. Terminology and phrasing are particularly
important in surveying, so this seems more important than it typically would
be.

If a user starts the survey, gets bored, and doesn't finish it, will the
results be partially saved?

Can a user choose not to answer particular questions? For example, if a user
did not want to answer the gender question, can it simply be skipped? If it
can be skipped, is this recorded as a skip ("I choose not to answer")?

What survey software is going to be used to conduct the survey (and where
will it be hosted)? I remember one of the past surveys used some
particularly bad software that wouldn't allow simple user behavior, such as
hitting the back button on your browser.

Is the survey software smart enough to not ask questions if a previous
question has been answered in a particular way? For example, if a user
answers "no" to participating in future surveys, will the software still ask
for an e-mail address?

Why is it an option to choose "unregistered user" if the survey is only
being provided to registered users?

Certain terms in the draft are in bold (e.g., "Global South"). Will these be
in bold/highlighted in the published survey? If so, why?

A question about a user's sexual orientation is conspicuously missing (given
that several other questions reference sexual orientation). Was this an
intentional omission? If so, why?

Is there a concern that a question such as "Do you know whether the
Wikimedia Foundation that runs Wikipedia is a nonprofit or for-profit
organization?" might have biased results given that the survey introduction
specifically notes that the Wikimedia Foundation is a non-profit
[organization]?

Some questions will presumably have a long list of possible answers (e.g.,
"What is/are your primary language(s)?"). Will the order of these possible
answers be alphabetical, based on referring wiki (put English at the top for
users who come from the English Wikipedia), or something else?

Will referring site be tracked (assuming this survey is conducted on a
separate domain)?

Is there a reason only Wikipedia is being targeted? It seems to me that
figuring out why other projects have such lower rates of participation would
be pretty important/valuable information, for example. And is there a reason
the page at the strategy wiki isn't more clear about the fact that this is
limited to a specific wiki family (i.e., "Editor survey feedback" vs.
"Wikipedia editor survey feedback")?

Who will be in charge of determining which data is released and how? If a
data trend is embarrassing to the Wikimedia Foundation, there might be an
incentive to not release that data. Is there a way to combat this? Who has
final say over what information is released?

Apologies for the slew of questions. I skimmed the FAQ, but didn't see any
of these answered. If I've simply missed some of these answers and they're
posted elsewhere, feel free to just drop a link as a reply. :-)

MZMcBride