Yes, the *IRB* might have chosen to waive the requirement of consent, but it appears the
IRB was never given the opportunity to review the proposed research. The student
researcher and/or their advisor is not in a position to make that decision on the IRB’s
behalf by not submitting an application.
And as per (3), it would appear that recruiting Wikipedia readers and editors to review
the generated articles could have been a simple alternative to placing them in Wikipedia
main space and avoided the problem being discussed. Perhaps if the application had been
properly presented to the IRB, that might have been the outcome.
As a Dean of Research in an IT faculty (prior to my retirement), I am well aware that
research students (and sometimes their advisors/supervisors) often take the view that “I’m
doing research on software, not people, I don’t need to worry about ethics”. I agree there
is no ethics issue in relation to the automatic construction of Wikipedia-like articles,
but there is an issue about putting them into Wikipedia mainspace and/or its processes to
observe the reaction to them (which did not seem to occur in the prior research
mentioned).
Kerry
From: Wiki-research-l [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of
Giovanni Luca Ciampaglia
Sent: Saturday, 13 August 2016 12:57 AM
To: Research into Wikimedia content and communities
<wiki-research-l(a)lists.wikimedia.org>
Subject: Re: [Wiki-research-l] Research on automatically created articles
Kerry,
I haven't had the time to read the paper itself, but regarding your comments on the
need for informed consent, I would like to point out that, at least from what I have
gleaned in this thread so far, it seems to me that consent could have probably been
waived. Let me quote the relevant regulations here (cf.
<http://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/#46.116> CFR
46.116(d)):
(d) An IRB may approve a consent procedure which does not include, or which alters, some
or all of the elements of informed consent set forth in this section, or waive the
requirements to obtain informed consent provided the IRB finds and documents that:
(1) The research involves no more than minimal risk to the subjects;
(2) The waiver or alteration will not adversely affect the rights and welfare of the
subjects;
(3) The research could not practicably be carried out without the waiver or alteration;
and
(4) Whenever appropriate, the subjects will be provided with additional pertinent
information after participation.
I think that given the few article titles seen so far, the research did involve no more
than minimal risk to users and editors. Requiring them to ask informed consent from
*every* person that came across to those pages without assistance from the developers
seems unfeasible; and waiving consent in this case does not seem to adversely affect the
rights and welfare of subjects either. Ditto for follow-up information.
Should they have applied to get exempt status from their IRB (i.e. essentially to get a
stamp of approval on what I just said)? Yes, they should have. This doesn't change the
nature of their research though, which is what matters to the present discussion.
Could they have put the articles in another namespace instead of the main one? Yes, but
that is a question of being considerate to other users/editors, not about whether the
research is legitimate/ethical or not.
Best
Giovanni
<http://glciampaglia.com> Giovanni Luca Ciampaglia ∙ Assistant Research Scientist,
Indiana University
On Fri, Aug 12, 2016 at 4:22 AM, Kerry Raymond <kerry.raymond(a)gmail.com
<mailto:kerry.raymond@gmail.com> > wrote:
And to its policies
http://guru.psu.edu/policies/RP03.html
With particular reference to
"Intervention includes both physical procedures by which data are gathered (for
example, venipuncture) and manipulations of the participant or the participant’s
environment that are performed for research purposes."
Putting that articles into Wikipedia manipulated the environment of Wikipedia readers and
editors.
Now I am not saying that huge harm was done, you would have to ask those who subsequently
edited the articles (a known group) and those who read the articles (an unknown group) to
find out if they are unhappy about what took place.
What I am saying is that if consideration had been given to the question who is impacted
by this research plan, the maybe the research plan would have been redesigned to prevent
the problem, and we would not have to have this conversation.
Kerry
Sent from my iPad
On 12 Aug 2016, at 6:08 PM, Kerry Raymond <kerry.raymond(a)gmail.com
<mailto:kerry.raymond@gmail.com> > wrote:
I draw attention to Penn State's IRB website
https://www.research.psu.edu/irb/submit
Sent from my iPad
On 12 Aug 2016, at 6:03 PM, Kerry Raymond <kerry.raymond(a)gmail.com
<mailto:kerry.raymond@gmail.com> > wrote:
I am asking you to share the documentation of the ethical clearance or exemption your
institution would have required, not what people did or didn't say to you as part of
conference reviewing or at conferences. Ethical clearance is a process that should have
been undertaken before your research commenced, not when you are writing the paper or
attending a conference. Are you saying you undertook the research without any
consideration of the ethics? Does your university have no guidelines about this?
The Wikipedia guidelines about content analysis are not particularly relevant here. You
were not analysing existing Wikipedia articles but injecting new articles of dubious
quality into Wikipedia.
Nor is the data about individuals my point. If you wasted people's time reacting to
the articles created, you did them harm. If people derived incorrect information from
reading your articles, you did them harm. None of those people were aware they were part
of your research experiment; that means they did not have informed consent in relation to
choosing to participate in your experiment. You could have generated the articles and
sought the opinions of readers and editors of Wikipedia on those articles without placing
them into Wikipedia itself. That way would have enabled informed consent; others not
wishing to take part would not be mislead into doing so.
Sent from my iPad
On 12 Aug 2016, at 3:24 PM, siddhartha banerjee <sidd2006(a)gmail.com
<mailto:sidd2006@gmail.com> > wrote:
I thought I should add this too as I missed it in the previous email.
This link:
https://en.wikipedia.org/wiki/Wikipedia:Ethically_researching_Wikipedia
talks about the Content Analysis (seeing number of references removed, or content
removed)-- which we did (with the few articles) and that is what we followed as it says
"generally considered exempt from such requirements and does not require an IRB
approval.".
My advisor should be able to add more thoughts on it (I have requested him to reply on
this thread).
Thanks,
Sidd
On Thu, Aug 11, 2016 at 9:36 PM, siddhartha banerjee <sidd2006(a)gmail.com
<mailto:sidd2006@gmail.com> > wrote:
As I have mentioned earlier, this is not the first work on article generation. This is one
of the first work we know:
https://people.csail.mit.edu/csauper/pubs/sauper-sm-thesis.pdf
https://people.csail.mit.edu/regina/my_papers/wiki.pdf
All these did not mention anything about human subjects as finally no personal information
is used (about the person, who is deleting, etc). Nor did any reviewers/attendees in the
conferences in this area question on this aspect.
Also,
https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2015-01-28/Recen… is
relevant here as it talks about our previous work.
if "record of someone doing something" is relevant from human subjects point of
view, any data on Wikipedia can be used to find the editors (if not the real person). For
example:
https://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/viewFile/3505/3968
https://alchemy.cs.washington.edu/papers/wu08/wu08.pdf
I have met several researchers who work using data (revisions from Wikipedia) and nothin
on IRB ever came up.
Nevertheless, as I said, if there are concrete rules, I think it would help the research
community as a whole to know what can or cannot be done and also ask for permissions.
I appreciate the suggestions that Stuart mentioned in a previous email abut experimenting
on would be deleted or articles lacking sources. But, as of now we are not planning
anything and if we do, we would for sure get in touch with Denny (who had a video chat
with me before starting this thread) and would try to know the best ways of doing it.
I have asked my PhD advisor (other author on the paper) to check this thread and he will
be able to give more inputs as I am not very qualified to comment on these aspects.
Thanks,
Sidd
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l