Yes, the *IRB* might have chosen to waive the requirement of consent, but it appears the IRB was never given the opportunity to review the proposed research. The student researcher and/or their advisor is not in a position to make that decision on the IRB’s behalf by not submitting an application.
And as per (3), it would appear that recruiting Wikipedia readers and editors to review the generated articles could have been a simple alternative to placing them in Wikipedia main space and avoided the problem being discussed. Perhaps if the application had been properly presented to the IRB, that might have been the outcome.
As a Dean of Research in an IT faculty (prior to my retirement), I am well aware that research students (and sometimes their advisors/supervisors) often take the view that “I’m doing research on software, not people, I don’t need to worry about ethics”. I agree there is no ethics issue in relation to the automatic construction of Wikipedia-like articles, but there is an issue about putting them into Wikipedia mainspace and/or its processes to observe the reaction to them (which did not seem to occur in the prior research mentioned).
Kerry
From: Wiki-research-l [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Giovanni Luca Ciampaglia
Sent: Saturday, 13 August 2016 12:57 AM
To: Research into Wikimedia content and communities <wiki-research-l@lists.wikimedia.org>
Subject: Re: [Wiki-research-l] Research on automatically created articles
I haven't had the time to read the paper itself, but regarding your comments on the need for informed consent, I would like to point out that, at least from what I have gleaned in this thread so far, it seems to me that consent could have probably been waived. Let me quote the relevant regulations here (cf. CFR 46.116(d)):
(d) An IRB may approve a consent procedure which does not include, or which alters, some or all of the elements of informed consent set forth in this section, or waive the requirements to obtain informed consent provided the IRB finds and documents that:
(1) The research involves no more than minimal risk to the subjects;
(2) The waiver or alteration will not adversely affect the rights and welfare of the subjects;
(3) The research could not practicably be carried out without the waiver or alteration; and
(4) Whenever appropriate, the subjects will be provided with additional pertinent information after participation.
I think that given the few article titles seen so far, the research did involve no more than minimal risk to users and editors. Requiring them to ask informed consent from *every* person that came across to those pages without assistance from the developers seems unfeasible; and waiving consent in this case does not seem to adversely affect the rights and welfare of subjects either. Ditto for follow-up information.
Should they have applied to get exempt status from their IRB (i.e. essentially to get a stamp of approval on what I just said)? Yes, they should have. This doesn't change the nature of their research though, which is what matters to the present discussion.
Could they have put the articles in another namespace instead of the main one? Yes, but that is a question of being considerate to other users/editors, not about whether the research is legitimate/ethical or not.
On Fri, Aug 12, 2016 at 4:22 AM, Kerry Raymond <kerry.raymond@gmail.com> wrote:
With particular reference to
"Intervention includes both physical procedures by which data are gathered (for example, venipuncture) and manipulations of the participant or the participant’s environment that are performed for research purposes."
Putting that articles into Wikipedia manipulated the environment of Wikipedia readers and editors.
Now I am not saying that huge harm was done, you would have to ask those who subsequently edited the articles (a known group) and those who read the articles (an unknown group) to find out if they are unhappy about what took place.
What I am saying is that if consideration had been given to the question who is impacted by this research plan, the maybe the research plan would have been redesigned to prevent the problem, and we would not have to have this conversation.
I draw attention to Penn State's IRB website
I am asking you to share the documentation of the ethical clearance or exemption your institution would have required, not what people did or didn't say to you as part of conference reviewing or at conferences. Ethical clearance is a process that should have been undertaken before your research commenced, not when you are writing the paper or attending a conference. Are you saying you undertook the research without any consideration of the ethics? Does your university have no guidelines about this?
The Wikipedia guidelines about content analysis are not particularly relevant here. You were not analysing existing Wikipedia articles but injecting new articles of dubious quality into Wikipedia.
Nor is the data about individuals my point. If you wasted people's time reacting to the articles created, you did them harm. If people derived incorrect information from reading your articles, you did them harm. None of those people were aware they were part of your research experiment; that means they did not have informed consent in relation to choosing to participate in your experiment. You could have generated the articles and sought the opinions of readers and editors of Wikipedia on those articles without placing them into Wikipedia itself. That way would have enabled informed consent; others not wishing to take part would not be mislead into doing so.
I thought I should add this too as I missed it in the previous email.
talks about the Content Analysis (seeing number of references removed, or content removed)-- which we did (with the few articles) and that is what we followed as it says "generally considered exempt from such requirements and does not require an IRB approval.".
My advisor should be able to add more thoughts on it (I have requested him to reply on this thread).
On Thu, Aug 11, 2016 at 9:36 PM, siddhartha banerjee <sidd2006@gmail.com> wrote:
As I have mentioned earlier, this is not the first work on article generation. This is one of the first work we know: https://people.csail.mit.edu/csauper/pubs/sauper-sm-thesis.pdf
All these did not mention anything about human subjects as finally no personal information is used (about the person, who is deleting, etc). Nor did any reviewers/attendees in the conferences in this area question on this aspect.
if "record of someone doing something" is relevant from human subjects point of view, any data on Wikipedia can be used to find the editors (if not the real person). For example:
I have met several researchers who work using data (revisions from Wikipedia) and nothin on IRB ever came up.
Nevertheless, as I said, if there are concrete rules, I think it would help the research community as a whole to know what can or cannot be done and also ask for permissions.
I appreciate the suggestions that Stuart mentioned in a previous email abut experimenting on would be deleted or articles lacking sources. But, as of now we are not planning anything and if we do, we would for sure get in touch with Denny (who had a video chat with me before starting this thread) and would try to know the best ways of doing it.
I have asked my PhD advisor (other author on the paper) to check this thread and he will be able to give more inputs as I am not very qualified to comment on these aspects.
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l