[WikiEN-l] UIC Journal: Evaluating quality control of Wikipedia's feature[d] articles

[David Lindsey]

I hope that the following will help to provide a littler more clarity.  I
have listed those articles that clearly failed and those that were
borderline along with a brief summation of some of the most significant
points raised by the reviewers.
Clear failure:
 1) Max Weber: The reviewer wrote that the article was "full of errors" and
also found that the sources used were not of high quality.  Unfortunately
from the perspective of improvement, he did not provide a detailed list of
the errors.
 2) Toru Takemitsu: The reviewer found one significant omission and several
errors and criticized the sources used - his primary issue here was that
only English-language sources were used while there is a much more
sophisticated Japanese-language literature.
 3) California Gold Rush: The reviewer criticized the quality of writing,
comparing it to that of a high school junior.  He also noted several
omissions, but mentioned that if the intended audience for the article was
high school students (this of course is not the case) most of these could be
forgiven.  He wrote that due to poor-quality sourcing and many omissions the
article would not be worthwhile for serious readers.
 4) Belarus: The reviewer cited a very great number of errors giving five
examples and writing that he saw "many more" but did not have the time to
note all of them.  Several omissions were also noted.  He also criticized
the quality of the writing, but his focus was on the factual errors and
omissions.
 5) Alzheimer's Disease: The reviewer found that "there are a number of
areas where there is little discussion about important aspects of the
disease" while other areas were (relatively) over-discussed and noted a
variety of omissions, primarily related to understanding of the causes of
the disease.  He also criticized the article for including "lots of 'facts'
that are so new that I don't know if they will hold up to further research"
and for throwing together strings of facts without making their meaning
apparent.
 6) Ten Commandments in Roman Catholicism: The reviewer primarily criticized
the article for a large number of omissions and wrote that the author(s) of
the author appeared to have a "genuine misunderstanding" of some of the
finer points of Catholic doctrine.
 7) The Swimming Hole: The reviewer criticized the article for presenting
speculation as fact, particularly with regard to the painter's sexuality and
criticized the sourcing, drawing attention to the fact that it relied
primarily on online material and that one reference was to a journal of
undergraduate research.

*I should note at this point that the article on relativity was held up as
passing the criteria, though it scored lower than the borderline cases all 3
of which received an 8*

"Borderline" cases:
1) Maiden Castle, Dorset: The reviewer criticized the article for an
error/omission in that "there is a tendency to assume nothing on the hill if
it was not found in excavations but large areas of the hilltop have not been
excavated and we should be a bit more cautious about saying the hilltop was
abandoned in the later Bronze Age or after the Roman temple went out of
use".  He also wrote that a few sentences were misleading though not
strictly inaccurate.  Finally, he criticized the writing as "clunky"
2) Funerary Monument to Sir John Hawkwood: The reviewer primarily took issue
with sourcing and for omitting an adequate discussion of earlier critical
reaction to the piece.
3) Poliomyelitis: The reviewer complimented the article for being "quite
thorough on the medical and scientific aspects of the disease." but
criticized it for omitting "any discussion of the experience of having polio
or living with its consequences".  He also described the further reading
section as "woefully inadequate" and wrote that it included weaker works
while omitting stronger ones, such as the Pulitzer Prize winning "Polio: An
American Story".

I'd also like to take this opportunity to comment on a few other points,
particularly those that are being raised about age.  Again, I have to stress
that the sample size is not large enough to make anything like solid
conclusions about age vs. quality.  Because people are really seizing onto
this point, though, I will also note that the data tends to support a
"rising standards" rather than a "deterioration" hypothesis.  The problems
noted by the experts were present in the version of each article that passed
FAC (for those that I have checked) and were not introduced by later edits.
Again, we are talking about a sample of 22 articles.  Of these, only 7
"clearly failed" the criteria, so it's quite likely than any correlations
are simply random noise.

To respond to another point, Steve Smith is perfectly correct that the
earlier studies evaluating articles on a scale of 1 to 10 were nothing like
random trials and that comparing the assessments I requested to those
assessments is not the best procedure.  The point of that discussion is to
note that there are *certainly* some non-featured articles that are better
than some or most featured articles.

David Lindsey

On Fri, Apr 16, 2010 at 6:40 PM, Andrew Gray <andrew.gray at dunelm.org.uk>wrote:

> On 16 April 2010 16:38, Amory Meltzer <amorymeltzer at gmail.com> wrote:
> > Three were "on the fence" so while the article may report a 55%
> > success rate, it also is stating a 32% failure rate.
>
> It's hard to tell from their scoring system which the three borderline
> ones were, though.
>
> Interestingly, the seven "clear failures" exhibit a strong correlation
> between quality and time - the points get lower as they get older. For
> the other articles, there's little or no correlation between the time
> since they passed FAC (or FAR) and their quality.
>
>
> http://www.generalist.org.uk/blog/2010/quality-versus-age-of-wikipedias-featured-articles/
>
> I suspect this points up a problem with maintenance more than initial
> quality, but we shall see.
>
> --
> - Andrew Gray
>  andrew.gray at dunelm.org.uk
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at lists.wikimedia.org
> To unsubscribe from this mailing list, visit:
> https://lists.wikimedia.org/mailman/listinfo/wikien-l
>