[WikiEN-l] Worst. Survey. Ever.

Sun Nov 2 19:48:28 UTC 2008

Unfortunate, but I agree

I stopped at 50%. Too long. Not friendly enough.

Immediately after it asked me how many hours per week I spent on the 
project. Follow a looooooonnnnnnng list of figures, from 0 to 168. One 
by one. Honestly... that's a bit ridiculous. Why not ranges ?

How many wikipedians do I meet every week ? 1, 2, 3, 4 ... 150.

I also did a mistake in entering my number of years of study. Usually, 
it means "superior studies". So, I put 5.
Later learned that it meant total years of studies.
Should have been ... around 20 then. Provided that life long learning is 
not studying.

Anyway, I wanted to go back, to fix my number of years of studies. Well, 
no such luck, there are some back buttons missing.

Gnannnnaaaa.

Okay, suggestion for next time Erik. Much shorter survey. And separate 
"contribution to" and "use of" surveys.

Ant

Steve Summit wrote:
> Anybody know where on-wiki the current survey is being discussed?
> I've got a thing or two to say.  (Message I just sent to
> info at wikipediastudy.org appended.)
> 
> 			*	*	*
> 
> From: Steve Summit <scs at eskimo.com>
> Date: Sat, 01 Nov 2008 10:16:41 -0400
> To: info at wikipediastudy.org
> Subject: your survey has problems
> 
> I just completed the survey at http://survey47.wikipediastudy.org/
> survey.php.  I'm sorry to be harsh and blunt.  It's terrible.
> You can't use my results accurately -- they're wrong.
> I doubt you can use anyone's results accurately.
> 
> This survey could only be completed accurately by someone:
> * with nothing to do / too much time on their hands
> * who never makes mistakes
> * who can anticipate future questions before they're asked
> * who can be bothered to search for his country and language
>   (several times) in strictly-alphabetical lists of every single
>   country and language in the world
> * who knows the 2-character ISO code for the languages he knows,
>   even when they're not obvious (e.g. DE for German)
> * who knows the 3-character ISO code for the currency he uses
> 
> The survey told me I couldn't use my browser's Back and Forward
> buttons, but had to use its own.  That's rude.
> 
> The survey then failed to provide Back buttons on all pages.
> That's incompetent.
> 
> The survey then asked me questions like "How many hours do
> you spend contributing to Wikipedia, per week?", followed by
> "How many hours to you spend administering Wikipedia?", followed by
> "How many hours do you spend supporting Wikipedia in technical ways?"
> And that ended up being profoundly insulting.  Here's why.
> 
> The administrative and technical work I do on Wikipedia feels
> like "contributions" to me, so (not knowing the next questions
> were coming up) I included those hours in my first answer.
> And the technical work I do feels like "administration", so
> (not knowing the next question was coming up) I included that
> in my second answer.  Therefore, if (as I suspect) you're
> assuming those three categories are disjoint, and since my major
> contributions lately have all been technical, I've inadvertently
> overstated my overall contributions in this survey by a factor
> of three.
> 
> And those particular survey pages were among those without
> Back buttons, so I couldn't fix my mistake.  Do you know how
> incredibly frustrating that is, to have wanted to spend time
> contributing to a survey, to know I've contributed false
> information, and to not be able to fix it?
> 
> Also, the survey took *way* too long.  And there was no
> information given up-front about how long it might take.
> The progress bar in the upper right-hand corner was a clue
> and a nice touch, but it came too late.
> 
> The survey also took too long in relationship to the impression
> of the data likely to be gleaned from it.  Short, tightly-focused
> surveys give the surveyee the impression that some well-thought-out,
> concise questions are being addressed by the surveyer.  Long,
> scattershot surveys give the impression that the surveyers aren't
> quite sure what they're looking for, are trying to ask everything
> they can think of, and are imagining that they'll mine the data
> later for interesting results later.  But, with poorly-defined
> surveys, that task often ends up being difficult or impossible.
> So I'm left begrudging the time I spent filling out the survey,
> because it feels like the ratio of time investment (by me) to
> useful information which can be gleaned (by you) is not good.
> 
> The survey asked me to specify things like "approximate number of
> articles edited" and "percentage of time spent translating" using
> drop-down selection boxes -- and with an increment of 1 between
> the available choices!  That's just silly.  (I dreaded how long I
> was going to have to scroll down to find my article edit count --
> 1196 -- and was both relieved and annoyed to discover that, after
> 500 entries, the drop-down list ended with "more than 500".)
> 
> The survey's categories were too-bluntly taken from existing
> lists.  For example, the list I had to choose my employment from
> was apparently taken from one of those dreadful Department of
> Commerce categorizations, that I have just as much trouble
> finding my job in when I fill out my tax forms.
> 
> At the very end, the survey asked if I wanted to submit my
> results, or fix any mistakes.  But the provided way to fix
> mistakes was to use the Back button -- perhaps several dozen
> times -- which I wouldn't have felt like doing even if the chain
> of Back buttons were complete.
> 
> The survey was clearly designed by someone who was thinking about
> the data they wanted to collect, and in a scattershot way.  The
> survey was clearly not designed with the person completing it in
> mind.  The survey was clearly not designed or vetted by anyone
> who knew anything about designing good surveys.
> 
> I probably had more complaints to list, but I shouldn't waste as
> much time on this letter as I already wasted taking the survey,
> so I'll stop here.
> 
> Bottom line: Please use the results of this survey with extreme
> care, if at all.  The results are going to be heavily, heavily
> biased by the inadvertent selection criteria involved in the
> survey's hostility towards its participants.  If you conduct a
> survey like this again, please find someone to assist in the
> process who knows something about real-world survey work.
> 
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at lists.wikimedia.org
> To unsubscribe from this mailing list, visit:
> https://lists.wikimedia.org/mailman/listinfo/wikien-l
> 