All,
the Expert Barrier survey is ready to run. We took into account most of the suggestions we received during the pilot phase and we are planning to release the survey by Wednesday morning London time, in time for the Imperial College Recruitment Drive [1]. Here is a link to a live preview:
http://nitens.org/ls/?sid=21693
(some blocks of questions are hidden depending on how participants answer in the first screen when asked whether they ever contributed to WP)
If you have a moment to give it a try, comments are welcome by Tuesday night PST.
Best, Dario, Giota and Daniel
I appreciate that phrasing questions neutrally can be somewhat difficult. But I think there are too many leading or negatively phrased questions to produce useful information. Starting the section with "Editing Wikipedia may damage one's scientific reputation" rather sets an anti wikipedia tone.
Also I'd switch the sequence between individual and collective perception. Putting the section about the individual before the collective section would start with something that respondents should more easily be able to answer - some people simply won't feel that they can answer questions on behalf of people in their field if they are unaware of those other people's views.
WereSpielChequers
On 7 February 2011 12:16, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
All,
the Expert Barrier survey is ready to run. We took into account most of the suggestions we received during the pilot phase and we are planning to release the survey by Wednesday morning London time, in time for the Imperial College Recruitment Drive [1]. Here is a link to a live preview:
http://nitens.org/ls/?sid=21693
(some blocks of questions are hidden depending on how participants answer in the first screen when asked whether they ever contributed to WP)
If you have a moment to give it a try, comments are welcome by Tuesday night PST.
Best, Dario, Giota and Daniel
[1] http://en.wikipedia.org/wiki/Wikipedia:CONTRIB/Imperial _______________________________________________ RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
Hi WSC,
thanks for the detailed feedback. Here's some background on what guided the design of the survey:
Negatively phrased questions All the questions were compiled based on actual input we collected during the pilot. As outrageous as it may sound, editing Wikipedia as a potential threat to one's career is an actual reason some participants reported and we want to assess its importance as a potential barrier. If you google "tenure blogging" you will get an idea of how hot the debate has been around the question whether blogging is a risk or not for one's career (especially for non-tenured faculty). Something similar may happen (or may have happened in the past) for Wikipedia and we want to capture this i the survey.
Starting the section with... The order of the questions in each block is randomised for each participant, so it was just bad luck :)
Individual motivations vs shared perception This is the most delicate issue of the survey, for which we considered several alternate designs. We want to contrast the participants' perception of Wikipedia participation in general with one's individual motivation to contribute or not to contribute. The rationale for this is that we noticed that participants often can dissociate their judgment on statements they would endorse as members of a professional category ("editing Wikipedia does not count towards improving one's CV") from judgments on what represents an individual barrier to participation ("even if editing Wikipedia does not improve my CV, this is not a reason for me not to contribute"). Phrasing questions as regarding one's peers is a way to have a participant focus on shared perception as opposed to individual motivation. The reason why we put general questions upfront and individual-motivation questions at the end is that we believe the noise added by framing questions "from general to individual" is less important (for what we want to study) than if we ordered them "from individual to general". To address this properly we should randomise the order of the two blocks, not just the order of questions within each block, that's something I can look into.
Dario
On 7 Feb 2011, at 19:00, WereSpielChequers wrote:
I appreciate that phrasing questions neutrally can be somewhat difficult. But I think there are too many leading or negatively phrased questions to produce useful information. Starting the section with "Editing Wikipedia may damage one's scientific reputation" rather sets an anti wikipedia tone.
Also I'd switch the sequence between individual and collective perception. Putting the section about the individual before the collective section would start with something that respondents should more easily be able to answer - some people simply won't feel that they can answer questions on behalf of people in their field if they are unaware of those other people's views.
WereSpielChequers
On 7 February 2011 12:16, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
All,
the Expert Barrier survey is ready to run. We took into account most of the suggestions we received during the pilot phase and we are planning to release the survey by Wednesday morning London time, in time for the Imperial College Recruitment Drive [1]. Here is a link to a live preview:
http://nitens.org/ls/?sid=21693
(some blocks of questions are hidden depending on how participants answer in the first screen when asked whether they ever contributed to WP)
If you have a moment to give it a try, comments are welcome by Tuesday night PST.
Best, Dario, Giota and Daniel
[1] http://en.wikipedia.org/wiki/Wikipedia:CONTRIB/Imperial _______________________________________________ RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
Hello,
Some remarks:
"People in my field are not allowed to write about their original research in Wikipedia" - I stumbled over this; one is not supposed to write about OR in Wikipedia. This could be misunderstood.
"Wikipedia robots arbitrarily revert legitimate contributions" - do bots really do that? And are people afraid of that? See also "Wikipedia robots may arbitrarily revert my legitimate contributions"
"I don't see the purpose of contributing to Wikipedia" - this seems to be too general to me, I would drop it
Kind regards Ziko
2011/2/7 Dario Taraborelli dtaraborelli@wikimedia.org:
Hi WSC, thanks for the detailed feedback. Here's some background on what guided the design of the survey: Negatively phrased questions All the questions were compiled based on actual input we collected during the pilot. As outrageous as it may sound, editing Wikipedia as a potential threat to one's career is an actual reason some participants reported and we want to assess its importance as a potential barrier. If you google "tenure blogging" you will get an idea of how hot the debate has been around the question whether blogging is a risk or not for one's career (especially for non-tenured faculty). Something similar may happen (or may have happened in the past) for Wikipedia and we want to capture this i the survey. Starting the section with... The order of the questions in each block is randomised for each participant, so it was just bad luck :) Individual motivations vs shared perception This is the most delicate issue of the survey, for which we considered several alternate designs. We want to contrast the participants' perception of Wikipedia participation in general with one's individual motivation to contribute or not to contribute. The rationale for this is that we noticed that participants often can dissociate their judgment on statements they would endorse as members of a professional category ("editing Wikipedia does not count towards improving one's CV") from judgments on what represents an individual barrier to participation ("even if editing Wikipedia does not improve my CV, this is not a reason for me not to contribute"). Phrasing questions as regarding one's peers is a way to have a participant focus on shared perception as opposed to individual motivation. The reason why we put general questions upfront and individual-motivation questions at the end is that we believe the noise added by framing questions "from general to individual" is less important (for what we want to study) than if we ordered them "from individual to general". To address this properly we should randomise the order of the two blocks, not just the order of questions within each block, that's something I can look into. Dario On 7 Feb 2011, at 19:00, WereSpielChequers wrote:
I appreciate that phrasing questions neutrally can be somewhat difficult. But I think there are too many leading or negatively phrased questions to produce useful information. Starting the section with "Editing Wikipedia may damage one's scientific reputation" rather sets an anti wikipedia tone.
Also I'd switch the sequence between individual and collective perception. Putting the section about the individual before the collective section would start with something that respondents should more easily be able to answer - some people simply won't feel that they can answer questions on behalf of people in their field if they are unaware of those other people's views.
WereSpielChequers
On 7 February 2011 12:16, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
All,
the Expert Barrier survey is ready to run. We took into account most of the suggestions we received during the pilot phase and we are planning to release the survey by Wednesday morning London time, in time for the Imperial College Recruitment Drive [1]. Here is a link to a live preview:
http://nitens.org/ls/?sid=21693
(some blocks of questions are hidden depending on how participants answer in the first screen when asked whether they ever contributed to WP)
If you have a moment to give it a try, comments are welcome by Tuesday night PST.
Best,
Dario, Giota and Daniel
[1] http://en.wikipedia.org/wiki/Wikipedia:CONTRIB/Imperial
RCom-l mailing list
RCom-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
Hi Ziko,
Yes there are bots that revert edits, they aren't perfect http://fr.wikipedia.org/w/index.php?title=Keith_Barnes&diff=next&old... but they usually get it right.
In my experience people who have good edits reverted are far more likely to have been reverted by human editors.
WereSpielChequers
On 7 February 2011 22:36, Ziko van Dijk zvandijk@googlemail.com wrote:
Hello,
Some remarks:
"People in my field are not allowed to write about their original research in Wikipedia" - I stumbled over this; one is not supposed to write about OR in Wikipedia. This could be misunderstood.
"Wikipedia robots arbitrarily revert legitimate contributions" - do bots really do that? And are people afraid of that? See also "Wikipedia robots may arbitrarily revert my legitimate contributions"
"I don't see the purpose of contributing to Wikipedia" - this seems to be too general to me, I would drop it
Kind regards Ziko
2011/2/7 Dario Taraborelli dtaraborelli@wikimedia.org:
Hi WSC, thanks for the detailed feedback. Here's some background on what guided the design of the survey: Negatively phrased questions All the questions were compiled based on actual input we collected during the pilot. As outrageous as it may sound, editing Wikipedia as a potential threat to one's career is an actual reason some participants reported and we want to assess its importance as a potential barrier. If you google "tenure blogging" you will get an idea of how hot the debate has been around the question whether blogging is a risk or not for one's career (especially for non-tenured faculty). Something similar may happen (or may have happened in the past) for Wikipedia and we want to capture this i the survey. Starting the section with... The order of the questions in each block is randomised for each participant, so it was just bad luck :) Individual motivations vs shared perception This is the most delicate issue of the survey, for which we considered several alternate designs. We want to contrast the participants' perception of Wikipedia participation in general with one's individual motivation to contribute or not to contribute. The rationale for this is that we noticed that participants often can dissociate their judgment on statements they would endorse as members of a professional category ("editing Wikipedia does not count towards improving one's CV") from judgments on what represents an individual barrier to participation ("even if editing Wikipedia does not improve my CV, this is not a reason for me not to contribute"). Phrasing questions as regarding one's peers is a way to have a participant focus on shared perception as opposed to individual motivation. The reason why we put general questions upfront and individual-motivation questions at the end is that we believe the noise added by framing questions "from general to individual" is less important (for what we want to study) than if we ordered them "from individual to general". To address this properly we should randomise the order of the two blocks, not just the order of questions within each block, that's something I can look into. Dario On 7 Feb 2011, at 19:00, WereSpielChequers wrote:
I appreciate that phrasing questions neutrally can be somewhat difficult. But I think there are too many leading or negatively phrased questions to produce useful information. Starting the section with "Editing Wikipedia may damage one's scientific reputation" rather sets an anti wikipedia tone.
Also I'd switch the sequence between individual and collective perception. Putting the section about the individual before the collective section would start with something that respondents should more easily be able to answer - some people simply won't feel that they can answer questions on behalf of people in their field if they are unaware of those other people's views.
WereSpielChequers
On 7 February 2011 12:16, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
All,
the Expert Barrier survey is ready to run. We took into account most of the suggestions we received during the pilot phase and we are planning to release the survey by Wednesday morning London time, in time for the Imperial College Recruitment Drive [1]. Here is a link to a live preview:
http://nitens.org/ls/?sid=21693
(some blocks of questions are hidden depending on how participants answer in the first screen when asked whether they ever contributed to WP)
If you have a moment to give it a try, comments are welcome by Tuesday night PST.
Best,
Dario, Giota and Daniel
[1] http://en.wikipedia.org/wiki/Wikipedia:CONTRIB/Imperial
RCom-l mailing list
RCom-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
-- Ziko van Dijk The Netherlands http://zikoblog.wordpress.com/
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
Hi Ziko,
I stumbled over this; one is not supposed to write about OR in Wikipedia.
precisely, and that's what makes it an important barrier to researchers. Just to clear any confusion, the goal of the survey is not to suggest we should change WP to accommodate more experts but to help us understand what may or may not work as an effective recruitment strategy. Does that make sense?
Best, Dario
Dario, "People in my field are not allowed to write about their original research in Wikipedia" I see what you mean. "People in my field" sounded to me as if they are not allowed because of their colleagues. Maybe: "People in my field dislike it that they cannot write in Wikipedia about their original research".
WereSpielChecquers, Interesting, I hardly ever saw such reverts in de.wp. It seems to be a more technical problem; again, it is the question if experts really anticipate such "barriers" to contributing.
Kind regards Ziko
2011/2/8 Dario Taraborelli dtaraborelli@wikimedia.org:
Hi Ziko,
I stumbled over this; one is not supposed to write about OR in Wikipedia.
precisely, and that's what makes it an important barrier to researchers. Just to clear any confusion, the goal of the survey is not to suggest we should change WP to accommodate more experts but to help us understand what may or may not work as an effective recruitment strategy. Does that make sense?
Best, Dario _______________________________________________ RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
Hi Ziko,
DE wiki has instituted a system called flagged revisions, which means that every newby edit there gets checked, but only gets checked once. This has various advantages over the EN system where some new edits are checked many times and others slip through unnoticed. One theory is that good edits are less likely to be reverted if they are only checked once, as there is only one chance of a mistake...... This is one of the cross wiki research projects where it would be really helpful to have an independent researcher do a study.
WereSpielChequers
On 8 February 2011 11:53, Ziko van Dijk zvandijk@googlemail.com wrote:
Dario, "People in my field are not allowed to write about their original research in Wikipedia" I see what you mean. "People in my field" sounded to me as if they are not allowed because of their colleagues. Maybe: "People in my field dislike it that they cannot write in Wikipedia about their original research".
WereSpielChecquers, Interesting, I hardly ever saw such reverts in de.wp. It seems to be a more technical problem; again, it is the question if experts really anticipate such "barriers" to contributing.
Kind regards Ziko
2011/2/8 Dario Taraborelli dtaraborelli@wikimedia.org:
Hi Ziko,
I stumbled over this; one is not supposed to write about OR in Wikipedia.
precisely, and that's what makes it an important barrier to researchers. Just to clear any confusion, the goal of the survey is not to suggest we should change WP to accommodate more experts but to help us understand what may or may not work as an effective recruitment strategy. Does that make sense?
Best, Dario _______________________________________________ RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
-- Ziko van Dijk The Netherlands http://zikoblog.wordpress.com/
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l