[WikiEN-l] Peer Review (PC PRO #154)

[michael west]

The latest PC Pro has a 5 page spread on Wikipedia, the analysis is very
fair.  They did WPs ability to tackle deliberate or unintentional
misinformation by posting ten small rewordings from an IP address (a single
address!), all were corrected within an hour (suprised NOT). Later that week
they tried the same thing from different addresses - 10 were correct within
an hour and two went unnoticed for a week and they say they were self
corrected.  The article isn't online yet and I only managed to read it in
the bookstore.

It's mentioned in peer review
http://en.wikipedia.org/wiki/Wikipedia:External_peer_review

In http://en.wikipedia.org/wiki/Reliability_of_Wikipedia it covers it as
such

A further informal assessment by the popular IT magazine "PC Pro", for its
2007 article *Wikipedia
Uncovered*[8]<http://en.wikipedia.org/wiki/Reliability_of_Wikipedia#_note-uncovered>tested
Wikipedia by a similart device to those described above, by
introducing 10 errors that "varied between bleeding obvious and deftly
subtle" into articles (the researchers later put right the articles it had
edited). Labelling the results "impressive" it noted that all but one was
noted and fixed within the hour, and that "the Wikipedians tools and
know-how were just too much for our team". A second series of another 10
tests, using "far more subtle errors" and additional techniques to conceal
their nature, met similar results: "despite our stealth attempts the vast
majority ... were discovered remarkably quickly... the ridiculously minor
Jesse James error was corrected within a minute and a very slight change to
Queen Ann's entry was put right within two minutes." Two of the latter
series were not detected. The article concluded that "Wikipedia corrects the
vast majority of errors within minutes, but if they're not spotted within
the first day the chances ...dwindle as you're then relying on someone to
spot the errors while reading the article rather than reviewing the edits."

I don't condone vandalism of this kind, but I think it proves we have a
finger on the pulse.

Mike