Ziko,
Thanks for your detailed email. Agree on all the comments.
Some earlier comments might have been harsh, but I understand that there is a valid reason behind it and also the dedication of so many people involved to help reach Wikipedia where it is today.
We should have been more diligent in finding out policies and rules (including IRB) before entering content on Wikipedia. We promise not to repeat anything of this sort in the future and also I am trying to summarize all that has been discussed here to prevent such unpleasant experiences from other researchers in this area.
-- Sidd
Thanks Sidd for responding actively in this thread.
The biggest problem here: the algorithm used in this research were bad. They produced nonsense that wasn't remotely grammatical. You should have caught most of these problems. (The early version of the bot (for just plays) had a poor success rate as well, but it seemed plausible that a template for tiny play articles could be effectively filled out with automation.)
Two interesting results IMO: + A nonsensical article with a decent first sentence & sections, and refs (however random), can serve as encouragement to write a real article. Possibly more of an encouragement than just the first sentence alone. I believe there's some related research into how people respond to cold emails that include mistakes & nonsense. (Surely there's a more effective \ non-offensive way to produce similar results) + We could use even a naive measure of the coverage & consistency of new article review. (If it drops below a certain threshhold, we could do something like change the background color & search-engine metadata for pages that haven't been properly reviewed yet)
For future researchers: If we encourage people to spend more time making tools work – rather than doing something simple (even counterproductive) and writing a paper about it – everyone will benefit. The main namespace is full of bots, both fully automatic and requiring a human to run them. Anyone considering or implementing wiki automation should look at them and talk to the community of bot maintainers.
Sam
On Mon, Aug 15, 2016 at 1:28 PM, siddhartha banerjee sidd2006@gmail.com wrote:
Ziko,
Thanks for your detailed email. Agree on all the comments.
Some earlier comments might have been harsh, but I understand that there is a valid reason behind it and also the dedication of so many people involved to help reach Wikipedia where it is today.
We should have been more diligent in finding out policies and rules (including IRB) before entering content on Wikipedia. We promise not to repeat anything of this sort in the future and also I am trying to summarize all that has been discussed here to prevent such unpleasant experiences from other researchers in this area.
-- Sidd
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
For the sake of completeness, the archival URL for the thread at ANI is
https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incide...
cheers stuart
-- ...let us be heard from red core to black sky
On Tue, Aug 16, 2016 at 7:04 AM, Samuel Klein meta.sj@gmail.com wrote:
Thanks Sidd for responding actively in this thread.
The biggest problem here: the algorithm used in this research were bad. They produced nonsense that wasn't remotely grammatical. You should have caught most of these problems. (The early version of the bot (for just plays) had a poor success rate as well, but it seemed plausible that a template for tiny play articles could be effectively filled out with automation.)
Two interesting results IMO:
- A nonsensical article with a decent first sentence & sections, and refs
(however random), can serve as encouragement to write a real article. Possibly more of an encouragement than just the first sentence alone. I believe there's some related research into how people respond to cold emails that include mistakes & nonsense. (Surely there's a more effective \ non-offensive way to produce similar results)
- We could use even a naive measure of the coverage & consistency of new
article review. (If it drops below a certain threshhold, we could do something like change the background color & search-engine metadata for pages that haven't been properly reviewed yet)
For future researchers: If we encourage people to spend more time making tools work – rather than doing something simple (even counterproductive) and writing a paper about it – everyone will benefit. The main namespace is full of bots, both fully automatic and requiring a human to run them. Anyone considering or implementing wiki automation should look at them and talk to the community of bot maintainers.
Sam
On Mon, Aug 15, 2016 at 1:28 PM, siddhartha banerjee sidd2006@gmail.com wrote:
Ziko,
Thanks for your detailed email. Agree on all the comments.
Some earlier comments might have been harsh, but I understand that there is a valid reason behind it and also the dedication of so many people involved to help reach Wikipedia where it is today.
We should have been more diligent in finding out policies and rules (including IRB) before entering content on Wikipedia. We promise not to repeat anything of this sort in the future and also I am trying to summarize all that has been discussed here to prevent such unpleasant experiences from other researchers in this area.
-- Sidd
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Samuel Klein @metasj w:user:sj +1 617 529 4266
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
The new issue of the Wikimedia Research Newsletter contains a review of the paper by Denny, also mentioning the debate on this list and on ANI: https://blog.wikimedia.org/2016/09/12/research-newsletter-august-2016/ (there is some additional discussion in the comments there and on the talk page of the Signpost version)
On Tue, Aug 23, 2016 at 8:20 PM, Stuart A. Yeates syeates@gmail.com wrote:
For the sake of completeness, the archival URL for the thread at ANI is
https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Incide...
cheers stuart
-- ...let us be heard from red core to black sky
On Tue, Aug 16, 2016 at 7:04 AM, Samuel Klein meta.sj@gmail.com wrote:
Thanks Sidd for responding actively in this thread.
The biggest problem here: the algorithm used in this research were bad. They produced nonsense that wasn't remotely grammatical. You should have caught most of these problems. (The early version of the bot (for just plays) had a poor success rate as well, but it seemed plausible that a template for tiny play articles could be effectively filled out with automation.)
Two interesting results IMO:
- A nonsensical article with a decent first sentence & sections, and refs
(however random), can serve as encouragement to write a real article. Possibly more of an encouragement than just the first sentence alone. I believe there's some related research into how people respond to cold emails that include mistakes & nonsense. (Surely there's a more effective \ non-offensive way to produce similar results)
- We could use even a naive measure of the coverage & consistency of new
article review. (If it drops below a certain threshhold, we could do something like change the background color & search-engine metadata for pages that haven't been properly reviewed yet)
For future researchers: If we encourage people to spend more time making tools work – rather than doing something simple (even counterproductive) and writing a paper about it – everyone will benefit. The main namespace is full of bots, both fully automatic and requiring a human to run them. Anyone considering or implementing wiki automation should look at them and talk to the community of bot maintainers.
Sam
On Mon, Aug 15, 2016 at 1:28 PM, siddhartha banerjee sidd2006@gmail.com wrote:
Ziko,
Thanks for your detailed email. Agree on all the comments.
Some earlier comments might have been harsh, but I understand that there is a valid reason behind it and also the dedication of so many people involved to help reach Wikipedia where it is today.
We should have been more diligent in finding out policies and rules (including IRB) before entering content on Wikipedia. We promise not to repeat anything of this sort in the future and also I am trying to summarize all that has been discussed here to prevent such unpleasant experiences from other researchers in this area.
-- Sidd
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Samuel Klein @metasj w:user:sj +1 617 529 4266
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
wiki-research-l@lists.wikimedia.org