[WikiEN-l] Re: Wikipedia in the news

[Ray Saintonge]

Sheldon Rampton wrote:

> But that wasn't the question. The article asked if Wikipedia had 
> tested its reliability by taking a number of RANDOMLY-SELECTED 
> articles and submitting them to scholars for fact-checking, to which 
> Jimbo admitted that they hadn't.
>
> Just out of curiosity, I clicked the "Random page" link a few times. 
> Out of ten articles, I found five stubs, two of which had frequent 
> grammatical errors. The remaining five included one article that 
> seemed strongly opinionated about the [[Nintendo Seal of Quality]], 
> and four articles of varying length that appear from what I can tell 
> to be accurate and appropriate for their topics. (Of course, I'm not 
> really qualified to judge the accuracy of some of the articles which 
> discuss topics outside my areas of interest.)
>
> My little experiment isn't sufficient to serve as the basis for any 
> conclusions, but if two out of ten articles are grammatically-flawed 
> stubs, it would seem to support the article's contention that 
> Wikipedia doesn't yet meet the quality-control standards of a 
> commercial encyclopedia like Encyclopedia Britannica.

This kind of sampling technique is certainly valid as a measure of 
quality, though I think that the sample size may be too small.  If we 
take a sample of 25 articles and rate them on a zero to four scale (for 
a maximum score of 100) it will give us some idea of quality levels.  
The first time that it is done it will give us base-line data.  Doing it 
again after three months or a year will let us know if we have made any 
progress.  To improve objectivity the testing should be done by several 
people, and their scores averaged.

One possible general scale:
    0.  A stub or little more than a stub.
    1.  A short incomplete article; this may indicate that it should be 
merged with something else.  A stable POV article that everyone has ignored.
    2.  An average article.  It may be POV, but it has a lot of 
activity, or even in the middle of an edit war.  The current POV phase 
is not relevant since that is likely to change within the next five 
minutes.  The writing style may be awkward.
    3.  A good quality article may still have some uncertain material, 
or faulty grammar but these can probably be fixed with ease.
    4.  A superior article that provides a comprehensive and neutral 
approach to the subject.  The reading of the text flows with ease, and 
there are very few minor spelling errors or typos.

Of course similar scales could be established that focus solely on such 
things as factual accuracy, grammar, etc.

Another technique might be to sent randomly selected articles to 
sympathetic professors who do not for themselves write for Wikipedia, 
and ask them to rate the articles as they would term papers.  It does 
not matter if the sample includes articles outside of that professor's 
field since an encyclopedia might be a place that they would look if 
they were trying to inform themselves about it.

Ec