[Wikimedia-l] Upcoming Survey, Feedback requested, and Office Hour

Tue Aug 21 13:32:44 UTC 2012

On Tue, Aug 21, 2012 at 5:04 AM, Delphine Ménard <notafishz at gmail.com> wrote:
> Hi, sorry to fish out a very old message, but since the survey is
> about to go live, I would like to share some concerns I have about it.
> Unfortunately, I was on holidays as this announcement came out, so
> couldn't do it earlier.
>
> I find the idea of an editor's survey to be extremely important, since
> it is (among other) a good indicator of how the editing community
> perceives the atmosphere in the projects, the evolution of the
> software and such things. However, I feel this survey is a bit of a
> missed opportunity on different aspects.
>
> There is a mix of "feedback about the projects and the community" and
> "satisfaction about the WMF", which does not, in my opinion, quite fit
> together. I find we should separate those things so as to keep people
> free of personal opinions about what the organisation may or may not
> do for/with them and let them focus better on their editor's
> experience as such. Moreover, this would allow for more questions
> about editing, maybe a short presentation of new tools, rating them
> etc. which seems to be quite absent from this questionnaire. For
> example, I would love to see a question in the technology part about
> whether people want/edit from their mobile device, or if they are
> familiar with the mobile apps and use them, that kind of stuff.
> (rationale given for taking these questions out was length of the
> survey, but I think these things are much more relevant to the
> well-editing of the contributors than how well they rate the WMF
> work). Not to mention that trying to get some feedback from sister
> projects would be good also (Commons is already a good first step).
>
> If, however, we're going to mix editor's experience and satisfaction
> about Wikimedia, I am cruelly missing any kind of feedback question
> about the work of the chapters and/or other organisations or groups in
> the Wikimedia Universe that would give people the right scope about
> what is happening in a more "offline" kind of way. Of course, we could
> do a separate survey for chapters, but if we're truly an international
> movement, then all Wikimedia entities that support/interact with the
> community probably would benefit from being put in the same bag in
> order to fine tune their support and help for the Wikimedia
> communities.
>
> Sue in Washington at Wikimania pulled out the results of one question
> that was asked in the editor's survey about whether people were
> satisfied about the work of the Foundation and the work of the
> chapters. She underlined herself that the results to this question
> were probably difficult to interpret out of the box since at no point
> in the survey was there a question about whether people were aware
> that there was a chapter in their country, which would have qulified
> the results a bit.
I'm not sure she actually said that. In any case, it is wrong -
question D6 in the last survey asked if there was a Wikimedia chapter
in the country where the respondent lived
(https://meta.wikimedia.org/w/index.php?title=File:December_2011_Wikipedia_Editor_Survey_topline.pdf&page=5
).

> This was already quite criticized last time, yet
> the question is still there, unchanged, and without more context than
> it had last time. (under FINAL THOUGHTS).
We tried to avoid modifying questions in order to preserve consistence
and be able  to do some longitudinal analysis (as it was already begun
for that question in http://blog.wikimedia.org/?p=14296 ).

Still, I was aware that there had been some objections to that
question by chapter representatives (which I don't assume have to do
with the fact that respondents rated chapters' performance lower than
that of other entities in the two previous surveys), and looked into
these concerns while the present questionnaire was prepared; I also
reached out to one of the critics in person at Wikimania. But I still
haven't seen a compelling argument why the way the question is asked
should be biased against chapters. The argument that the opinion of
Wikimedians who live in countries without chapter should not count
seems weak to me, e.g. because the projects that the work of chapters
aims to support are international, and because the question asked
about chapters in general, not one particular chapter.

In any case, in parallel to preparing the upcoming survey, we have
recently been compiling the public dataset with the anonymized
responses from the last survey, which was uploaded yesterday here:
http://dumps.wikimedia.org/other/surveys/editorsurveynov-dec2011/
Using this, anyone can do the analysis you suggest and check if
ratings differed significantly in chapter/non-chapter countries.

> In short people are asked to
> rate the Foundation about everything it does, while the chapters are
> never mentionned, and then people are asked to rate the work of both.
> Interesting way to look at it.
>
> We're doing much better, but there is still some English Wikipedia
> centrism in Question F2 for example. ;-)
> : How well do you believe the Foundation supports:
> English Wikipedia?
> Wikipedia sites in other languages?
>
>
> Finally, what I regret most, is that o little time was allocated to
> reviewing this survey collaboratively. There was about 20 days to
> review, comment, give feedback, translate etc. and this in the mist of
> the summer holiday for a big part of our community. When I see such
> processes as the ToS revision [1] conducted successfully allowing for
> 120 days of discussion, and such surveys rushed through the summer,
> when their data will probably be used to decide much of the strategic
> orientation of the next year, I'm thinking we're missing out on trying
> to collect data that is truly relevant to the work we do and that
> helps us to review our orientations and adjust how well we support our
> editing community.
Of course there are lots of interesting and important questions that
had to be left out of this survey. As said earlier, the idea is to run
the editor survey more frequently from now on, probably quarterly and
in a more lightweight version, with a different focus in each.
>
> Best,
>
> Delphine
>
>
>
> [1] http://meta.wikimedia.org/wiki/Talk:Terms_of_use
>
> On Sat, Jul 28, 2012 at 7:28 AM, Christine Moellenberndt
> <christinemoe at gmail.com> wrote:
>> *Hi everyone,
>>
>> It's been a bit since I last emailed this list (or any list, for that
>> matter!)... you may remember me, I worked at the Foundation last year in the
>> Community Department, working with Philippe on any number of issues, as well
>> as with the OTRS team.  I've come back to work on a short term project with
>> the Foundation, and I have to say it's great to be back! (and a great break
>> from my Master's thesis!)
>>
>> We're getting ready to run the next version of the Editor Survey, for August
>> 2012. This will be the third incarnation we've run since 2011.  As with the
>> prior incarnations of the survey, we'll be looking at a variety of topics,
>> this time with the goal of not only understanding your needs and pressing
>> issues while interacting with fellow editors, but also focusing on editors'
>> satisfaction with the work of the Foundation.
>>
>> The last time we ran an editor survey, it was completed by over 6,000
>> respondents.  When you break that down, it means that each minute of time
>> demanded by the survey corresponds to 100 hours of Wikipedians' time.  We
>> want to make sure that this time is spent wisely, ensuring that the
>> questions we have are worded clearly, don't cause confusion, and will
>> generate meaningful answers.  So we'd like to ask you to take a look at the
>> survey, and give us feedback on the questions.  You can find them here:
>>
>> https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012/Questions
>> <https://meta.wikimedia.org/w/index.php?title=Research:Wikipedia_Editor_Survey_August_2012/Questions>
>> ... and please leave your feedback on the talk page there so we can keep the
>> discussion in one place :)
>>
>> You can find out more information about the survey here:
>>
>> https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012
>>
>> Also, we are planning an IRC Office Hour on the survey, this **Tuesday, July
>> 31 at 1700 UTC.** (See https://meta.wikimedia.org/wiki/IRC_office_hoursfor
>> general information about IRC Office hours)
>>
>> I know there has been some discussion about offering Office Hours in a
>> broader range of times, and I know this time may not be the greatest for
>> some... but this was the best time we could find currently.
>>
>> Thanks everyone!!
>>
>> -Christine
>> Wikimedia Foundation*
>> _______________________________________________
>> Wikimedia-l mailing list
>> Wikimedia-l at lists.wikimedia.org
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
>
>
> --
> @notafish
>
> NB. This gmail address is used for mailing lists. Personal emails will get lost.
> Intercultural musings: Ceci n'est pas une endive - http://blog.notanendive.org
> Photos with simple eyes: notaphoto - http://photo.notafish.org
>
> _______________________________________________
> Wikimedia-l mailing list
> Wikimedia-l at lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l

-- 
Tilman Bayer
Senior Operations Analyst (Movement Communications)
Wikimedia Foundation
IRC (Freenode): HaeB