[WikiEN-l] UIC Journal: Evaluating quality control of Wikipedia's feature[d] articles

[Nathan]

On Fri, Apr 16, 2010 at 2:50 PM, David Lindsey <dvdlndsy at gmail.com> wrote:
>  5) Alzheimer's Disease: The reviewer found that "there are a number of
> areas where there is little discussion about important aspects of the
> disease" while other areas were (relatively) over-discussed and noted a
> variety of omissions, primarily related to understanding of the causes of
> the disease.  He also criticized the article for including "lots of 'facts'
> that are so new that I don't know if they will hold up to further research"
> and for throwing together strings of facts without making their meaning
> apparent.


>  7) The Swimming Hole: The reviewer criticized the article for presenting
> speculation as fact, particularly with regard to the painter's sexuality and
> criticized the sourcing, drawing attention to the fact that it relied
> primarily on online material and that one reference was to a journal of
> undergraduate research.
>

I pulled these two out because they point up a problem with the idea
of "expert review." It's been said multiple times, but it bears
repeating - expert review is not a panacea for article quality. The
Alzheimer's article, for instance, was written, edited and reviewed by
several expert editors, including a psychologist, a psychiatrist and
at least one or two other physicians. The Swimming Hole was, according
to it's author (Raul), reviewed by a very prominent recognized expert
on the subject. Raul didn't go into detail with the results of that
review, but he did say it was positive and at odds with the Lindsey
reviewer's low score. Another of Raul's articles received a low score,
in the field of his own Ph.D.

I wonder if there might be a subtle bias playing into these reviews.
Perhaps if reviewers begin with the assumption that the article was
written by amateur hobbyists, that influences the outcome. If Lindsey
went back to them and let them know that the articles had been written
or comprehensively reviewed by recognized experts, would that alter
the results?

-Nathan