[WikiEN-l] "How the Professor Who Fooled Wikipedia Got Caught by Reddit", _The Atlantic_

[Anthony]

On Mon, May 21, 2012 at 11:39 AM, Gwern Branwen <gwern0 at gmail.com> wrote:
> On Mon, May 21, 2012 at 8:15 AM, Anthony <wikimail at inbox.org> wrote:
>> So, you are not removing random links at all.
>
>>.< I should just link XKCD here, but I'll forebear. I am reminded of an anecdote describing a court case involving the draft back in Vietnam, where the plaintiff's lawyer argued that the little cage and balls method was not random and was unfair because the balls on top were much more likely to be selected. The judge asked, "Unfair to *whom*?" Indeed.
---
>From the beginning you seem to be under the mistaken impression that I
am trying to defend Wikipedia or defend the current Wikipedia
processes or something.  I am not.  I find your experiment
interesting.  I think it would be more interesting if your selection
of links were truly random, though.

I don't think you should describe your experiment as "removal of 100
random external links by an IP", because your selection was not at all
random.  I don't say this because I am trying to prove something about
the results.  I say it because it is a flaw in your methodology.

> And I'd note that my methodology, while being quite as random as most
> methods, carries the usual advantages of determinism: anyone will be
> able to check whether I did in fact remove only last links which are
> not official or template-generated in External Link sections, and that
> I did not simply cherrypick the links that I thought were worst and so
> least likely to be restored.

How could we do that?  You could have just cherrypicked the worst
links that were last links which are not official or
template-generated in External Link sections.  I'm not saying I think
you did that.  But you certainly could have.

Anyway, the main thing I'd like to say about all of this is simply
that your selection is not random.  Your sample is biased.  Biased in
which direction, I don't know.  Biased intentionally, I doubt.  But
your sample is biased.