Hi Jonathan,
Can you please give a concrete example of what, for example, the http://ide.mit.edu/sites/default/files/publications/SSRN-id3039505.pdf researchers would have had to do differently under this new policy?
Best regards, Jim
Date: Tue, 2 Jan 2018 15:29:03 -0800 From: Jonathan Morgan jmorgan@wikimedia.org To: Wiki Research-l Wiki-research-l@lists.wikimedia.org Subject: [Wiki-research-l] New policy about performing research on English Wikipedia
Hi there wiki-research folks,
This is just a heads-up that English Wikipedia has adopted a new policy[1] about research on that project. The policy codifies some new requirements for community notification and disclosure that potentially apply to all research projects (regardless of the affiliation of the researcher).
You can read more about the policy on WP:NOT[1], but I've included the major points below for your convenience:
- any research project that involves directly changing article content,
surveying a large number of editors, or asking editors sensitive questions about their real-life identities needs to be discussed on Wikipedia's Village Pump[2] before it is begun[3]
- researchers should disclose who they are on their user pages,
including their institutional affiliation, sources of research funding (if applicable), and the intentions behind their research[4]
Many aspects of this policy boil down to either common sense, existing ethical standards for human subjects research, or both. However, this policy also leaves certain definitions and thresholds undefined. What is a "large number" of surveyed users? What is a "sensitive question"? There are no concrete answer to these questions yet, and that's probably a good thing. The best way to keep this policy from becoming overly restrictive[5] is for researchers to follow its guidance in good faith, and ask questions when they're uncertain.
Projects that are deemed to be in violation of these guidelines may lose editing privileges. If the violations are deemed particularly frequent or severe, the EnWiki community may decide to make even more rules, which could have a chilling effect on wikiresearch in general. Nobody wants that.
If you have general questions about this policy or its application, the best place to ask is the WP:NOT talkpage.[6]
If you have questions related to a specific planned research project, the best thing to do is to err on the side of caution and open up a discussion on the Village Pump before you begin.
You are also welcome to post your project plan to this list, where we, your friendly peers, will hopefully offer constructive feedback and links to relevant resources.
Wikimedia Foundation research staff are not in charge of these guidelines, but are happy to offer advice "from the trenches" so to speak if asked. We are on this list too.
As always, if you are currently researching Wikipedia, or plan to do so, please create a Research Project page on MetaWiki[7] (example[8], tips[9]), keep it up to date, and link to it from your userpage[10]. That way interested parties can follow your research and ask questions, and you won't need to constantly re-explain what you're doing every time someone asks.
Happy researching,
Jonathan
https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#Wikipedia_is_n... 2. https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals) 3. https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#cite_note-7 4. https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#cite_note-8 5. https://meta.wikimedia.org/wiki/Instruction_creep 6. https://en.wikipedia.org/wiki/Wikipedia_talk:What_Wikipedia_is_not 7. https://meta.wikimedia.org/wiki/Research:Projects 8. https://meta.wikimedia.org/wiki/Research:Supporting_Commons_contribution_by_... 9. https://meta.wikimedia.org/wiki/Research:Project_documentation_best_practice... 10. https://meta.wikimedia.org/wiki/User:LZia_(WMF)
-- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)
Interesting question. I'll bite :-)
From reading what I hope the pertinent sections of the research paper (I didn't read it all), the research required new Wikipedia articles to be written on topics in chemistry and econometrics. These Wikipedia articles were not written by the researchers themselves but rather they recruited PhD students in those disciplines to write them. The topics were chosen by looking at lists of topics in the discipline based on text books and university syllabuses and then looking to see which topics did not yet have a Wikipedia article. The set of articles were then split into one group which were uploaded to Wikipedia and one group that was not (control group). What happened after that was more-or-less business-as-usual on Wikipedia (although some of the PhD students remained engaged in order to get the article to satisfy the reviewers to get the article accepted - I assume they were uploaded via Articles for Creation). The research itself was to see the impact of these new articles (compared to the topics that were not uploaded); this did not interfere with the articles, merely observing.
In terms of the new policy of "not a laboratory", I don't see any reason to regard these uploaded articles as "disruptive" or "negatively impacting articles". The method by which the topics were chosen and the selection of PhD students in that discipline to write the articles appears similar to that used in most edit-a-thons. The topics chosen seem likely to be notable and the authors were presumably competent in that discipline so presumably the quality should be at least equal to most new articles. It is unclear if the PhD students were paid to write the articles but, even if so, there seems to no conflict-of-interest as the researchers had no "agenda" other than to write a typical Wikipedia article on that topic. I guess the only argument for disruption might be that uploading a large number of articles in the same discipline presumably around the same time which might have generated a higher than normal workload for those competent to review them, but then an edit-a-thon may have had the same impact.
So my take is that it may have been courteous (under the new policy) to discuss the project at the Village Pump, but I think the researchers would have been operating within the policies even if they did not. Since many of the articles survived in some form (some were merged) would suggest there was benefit to Wikipedia from the research.
But a similar project that chose the topics more carelessly or used people with inferior discipline knowledge to write the articles or remained actively engaged with the article (e.g. gatekeeping/ownership) could well have been disruptive. So there probably would be benefit in having a conversation on the Village Pump (or wherever, I'm not convinced Village Pump is the right place) to establish exactly how certain aspects of the project should be conducted to avoid disruption and negative effects on articles. Simply, if the researchers are not active Wikipedians (by which I mean more than "I think I have mastered the syntax") , I am not convinced they are capable of judging what might be disruptive or negative.
Personally I think this list might be a better place than the Village Pump as we understand both research and Wikipedia while I am not convinced that the Village Pump understands research. But the reality is that many researchers will not know of the "not a laboratory policy" so the first we may know about research is either the resultant publications or the screaming and yelling that arises from discovering the research being executed (possibly because of the disruption being created). How do we communicate this policy to researchers?!
Kerry
-----Original Message----- From: Wiki-research-l [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of James Salsman Sent: Wednesday, 3 January 2018 10:30 PM To: wiki-research-l@lists.wikimedia.org Subject: Re: [Wiki-research-l] New policy about performing research on English WikipediaWiki-research-l Digest, Vol 149, Issue 1
Hi Jonathan,
Can you please give a concrete example of what, for example, the http://ide.mit.edu/sites/default/files/publications/SSRN-id3039505.pdf researchers would have had to do differently under this new policy?
Best regards, Jim
Date: Tue, 2 Jan 2018 15:29:03 -0800 From: Jonathan Morgan jmorgan@wikimedia.org To: Wiki Research-l Wiki-research-l@lists.wikimedia.org Subject: [Wiki-research-l] New policy about performing research on English Wikipedia
Hi there wiki-research folks,
This is just a heads-up that English Wikipedia has adopted a new policy[1] about research on that project. The policy codifies some new requirements for community notification and disclosure that potentially apply to all research projects (regardless of the affiliation of the researcher).
You can read more about the policy on WP:NOT[1], but I've included the major points below for your convenience:
- any research project that involves directly changing article content,
surveying a large number of editors, or asking editors sensitive questions about their real-life identities needs to be discussed on Wikipedia's Village Pump[2] before it is begun[3]
- researchers should disclose who they are on their user pages,
including their institutional affiliation, sources of research funding (if applicable), and the intentions behind their research[4]
Many aspects of this policy boil down to either common sense, existing ethical standards for human subjects research, or both. However, this policy also leaves certain definitions and thresholds undefined. What is a "large number" of surveyed users? What is a "sensitive question"? There are no concrete answer to these questions yet, and that's probably a good thing. The best way to keep this policy from becoming overly restrictive[5] is for researchers to follow its guidance in good faith, and ask questions when they're uncertain.
Projects that are deemed to be in violation of these guidelines may lose editing privileges. If the violations are deemed particularly frequent or severe, the EnWiki community may decide to make even more rules, which could have a chilling effect on wikiresearch in general. Nobody wants that.
If you have general questions about this policy or its application, the best place to ask is the WP:NOT talkpage.[6]
If you have questions related to a specific planned research project, the best thing to do is to err on the side of caution and open up a discussion on the Village Pump before you begin.
You are also welcome to post your project plan to this list, where we, your friendly peers, will hopefully offer constructive feedback and links to relevant resources.
Wikimedia Foundation research staff are not in charge of these guidelines, but are happy to offer advice "from the trenches" so to speak if asked. We are on this list too.
As always, if you are currently researching Wikipedia, or plan to do so, please create a Research Project page on MetaWiki[7] (example[8], tips[9]), keep it up to date, and link to it from your userpage[10]. That way interested parties can follow your research and ask questions, and you won't need to constantly re-explain what you're doing every time someone asks.
Happy researching,
Jonathan
https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#Wikipedia_is_n... 2. https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals) 3. https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#cite_note-7 4. https://en.wikipedia.org/wiki/Wikipedia:What_Wikipedia_is_not#cite_note-8 5. https://meta.wikimedia.org/wiki/Instruction_creep 6. https://en.wikipedia.org/wiki/Wikipedia_talk:What_Wikipedia_is_not 7. https://meta.wikimedia.org/wiki/Research:Projects 8. https://meta.wikimedia.org/wiki/Research:Supporting_Commons_contribution_by_... 9. https://meta.wikimedia.org/wiki/Research:Project_documentation_best_practice... 10. https://meta.wikimedia.org/wiki/User:LZia_(WMF)
-- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)
_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
wiki-research-l@lists.wikimedia.org