[WikiEN-l] Link removal experiment; Re: "How the Professor Who Fooled Wikipedia Got Caught by Reddit", _The Atlantic_

[WereSpielChequers]

There were a number of flaws in this experiment that IMHO reduce its value.

Firstly rather than measure vandalism it created vandalism, and vandalism
that didn't look like typical vandalism. Aside from the ethical issue
involved, this will have skewed the result. In particular the edit
summaries were very atypical for vandalism, if I'd seen that edit summary
on my watchlist I would probably have just sighed  and taken it as another
example of deletionism in action. Of the more than 13,000 pages on my
watchlist I doubt there are 13 where I would look at such an edit, and
that's if it was one of the changes on my watchlist that I was even aware
of - it is far too big to fully check every day. Most IP vandals don't use
jargon in edit summaries, and I know I'm not the only editor who is more
suspicious of IP edits with blank edit summaries.

You only ran the experiment for one month. I often revert older vandalism
than that, I may be unusual there in that I've got some tools for finding
vandalism that has got past the hugglers, but I'm not unusual in sometimes
taking articles back to the "last clean version". I'm also aware that we
have a large number of editors who we don't see every month - probably the
fastest growing part of the community. For example we have 543 admins who
have edited in the last three months but have fewer than thirty edits in
the last two months,
http://en.wikipedia.org/wiki/Wikipedia:List_of_administrators/Semi-activeMany
of these will be among the editors who visit occasionally to keep an
eye on some articles that they care about.

If someone wants to work out how good we are at clearing up vandalism then
I would suggest a more accurate way would be to:

   1. Take a random set of edits from 12 months ago.
   2. Check all of them, including the deleted ones, and classify them as
   vandalism or not
   3. Measure how long they persisted in the article
   4. Revert any vandalism still extant.

Of course that won't pick up those pre-empted by the edit filter, but if
the sample was sufficiently large and random it would give us a measure of
the effectiveness of our anti-vandalism work.

Regards

WSC


On 30 May 2012 23:02, Charles Matthews <charles.r.matthews at ntlworld.com>wrote:

> On 30 May 2012 20:41, Gwern Branwen <gwern0 at gmail.com> wrote:
>
> >> My view is that if such experiments are to be carried
> >> out, it would be better if they were designed and conducted by those
> >> able to restrain themselves from such snark.
>
> > Better how?
>
> I'll add this to my list of "If you have to ask, you may never know"
> topics.
>
> Charles
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at lists.wikimedia.org
> To unsubscribe from this mailing list, visit:
> https://lists.wikimedia.org/mailman/listinfo/wikien-l
>