Hi,
On Fri, Sep 14, 2012 at 6:08 PM, Jan-bart de Vreede <jdevreede@wikimedia.org
wrote:
Hey
So someone sent me the internal-l mail and I do think that the "zero real-world" thing is taken out of context here. But a few points
- Lets not have these discussions on internal-l, there is no reason to
not have those in public 2) See 1) 3) I am sure that the data set allows us to see chapter's individual responses, depending on whether or not we know the country (I would figure we do?)
I believe the biggest issue from a methodological point of view, once you filter out responses from non-chapter countries, is that the question asks for a composite average rating on chapterS performance. There is doubt that the results obtained in this way would be valid or reliable measures of chapters ratings. (On the face of it, if one has to rate together a well-performing known organization and 38 unknown ones, it is unlikely that he would adjust his rating of the known organization upwards to get to an average, it seems more likely that he would do a downwards adjustment and perhaps vice-versa in case of a low-performing local organization; different people would apply different adjustments and it becomes doubtful whether we can ever get an accurate average from a simple question asking for opinion about 39 organizations. Asking individuals to rate one chapter if relevant and than taking the [weighed] average of the ratings from the 39 relevant countries would be more accurate perhaps. And as a control group, people from outside chapter countries could be asked to rate all chapters in one question to see if there is anything to learn there from the answers.) In general, when a concept is complex, surveys tend to ask more questions to get a better picture, just as the WESI-score on Wikipedia satisfaction is a composite from a number of questions on this survey.
- Doesn't every survey contain questions that don't apply to the whole
responding audience?
Probably not - depends on the survey design and goals.
Best regards, Bence
Jan-Bart
On 14 sep. 2012, at 17:05, Florence Devouard anthere9@yahoo.com wrote:
On 9/12/12 4:14 PM, Jan-Bart de Vreede wrote:
Hey
So I might have missed some mails on this thread (perhaps because they
were not posted on this public list) but I highly doubt that Sue perform surveys that do not have a real-world impact on our operations. I know that the results of the previous surveys were used in several discussions (including at a board level) in order to provide more insight….
Indeed. I agree. These surveys do have real-world impact, which is why
we objected to a survey asking people from all over the world how they would rate Wikimedia Chapters activities when
- there is likely no chapter in their country
- they may have no idea that a "wikimedia chapter" is for example
"Wikimedia Washington DC" or "Wikimedia Israel"
- all chapters are collectively considered regardless of individual
differences
And since you comment on that specific sentence
"I'll point out also that there are zero real-world implications for the
survey results."
I'd like to clarify that these exact words come from Sue herself in an
email sent on the 10th of September on internal-l.
I am glad to read that you disagree with that statement and recognize
that there is real-world impact.
(did not want to comment any further on that problematic survey, but
wanted to attribute statement properly)
Flo
On the other hand, using these surveys to gain more insight is not the
same as using them to "hold each other accountable" which is sometimes easy to do. Every survey (and questions) has a lots of interpretation magic which can easily lead you astray, but I don't have to tell you (the community) this :)
Jan-Bart
On 10 Sep 2012, at 23:01, Delphine Ménard <
notafishz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Contrarily to Sue, I do think that these surveys (should) have a real-world impact and (should) keep us all on our toes, fine tuned to the critisicism, needs and wishes of the editors of the WIkimedia projects. As such I expect us to make sure that we do get as precise a picture as possible of what those are.
Wikimedia-l mailing list Wikimedia-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
Wikimedia-l mailing list Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l