[WikiEN-l] Link removal experiment; Re: "How the Professor Who Fooled Wikipedia Got Caught by Reddit", _The Atlantic_

[Thomas Morton]

On 31 May 2012 16:59, WereSpielChequers <werespielchequers at gmail.com> wrote:

> There were a number of flaws in this experiment that IMHO reduce its value.
>
> Firstly rather than measure vandalism it created vandalism, and vandalism
> that didn't look like typical vandalism. Aside from the ethical issue
> involved, this will have skewed the result. In particular the edit
> summaries were very atypical for vandalism, if I'd seen that edit summary
> on my watchlist I would probably have just sighed  and taken it as another
> example of deletionism in action. Of the more than 13,000 pages on my
> watchlist I doubt there are 13 where I would look at such an edit, and
> that's if it was one of the changes on my watchlist that I was even aware
> of - it is far too big to fully check every day. Most IP vandals don't use
> jargon in edit summaries, and I know I'm not the only editor who is more
> suspicious of IP edits with blank edit summaries.
>

This, I think, is a major issue which make the results useless

* The edit summary implies policy knowledge, I'd only check an edit like
that on my watchlist on occasion. Not every edit needs checking, so we use
our common sense over what likely need checking

* I believe that edit summary probably met a number of heuristics used by
the anti-vandal tools to filter out "good" edits. Which means it
immediately removes them from the "front line" of scrutiny.

Tom