Sheldon Rampton wrote:
But that wasn't the question. The article asked if Wikipedia had tested its reliability by taking a number of RANDOMLY-SELECTED articles and submitting them to scholars for fact-checking, to which Jimbo admitted that they hadn't.
Just out of curiosity, I clicked the "Random page" link a few times. Out of ten articles, I found five stubs, two of which had frequent grammatical errors. The remaining five included one article that seemed strongly opinionated about the [[Nintendo Seal of Quality]], and four articles of varying length that appear from what I can tell to be accurate and appropriate for their topics. (Of course, I'm not really qualified to judge the accuracy of some of the articles which discuss topics outside my areas of interest.)
My little experiment isn't sufficient to serve as the basis for any conclusions, but if two out of ten articles are grammatically-flawed stubs, it would seem to support the article's contention that Wikipedia doesn't yet meet the quality-control standards of a commercial encyclopedia like Encyclopedia Britannica.
This kind of sampling technique is certainly valid as a measure of quality, though I think that the sample size may be too small. If we take a sample of 25 articles and rate them on a zero to four scale (for a maximum score of 100) it will give us some idea of quality levels. The first time that it is done it will give us base-line data. Doing it again after three months or a year will let us know if we have made any progress. To improve objectivity the testing should be done by several people, and their scores averaged.
One possible general scale: 0. A stub or little more than a stub. 1. A short incomplete article; this may indicate that it should be merged with something else. A stable POV article that everyone has ignored. 2. An average article. It may be POV, but it has a lot of activity, or even in the middle of an edit war. The current POV phase is not relevant since that is likely to change within the next five minutes. The writing style may be awkward. 3. A good quality article may still have some uncertain material, or faulty grammar but these can probably be fixed with ease. 4. A superior article that provides a comprehensive and neutral approach to the subject. The reading of the text flows with ease, and there are very few minor spelling errors or typos.
Of course similar scales could be established that focus solely on such things as factual accuracy, grammar, etc.
Another technique might be to sent randomly selected articles to sympathetic professors who do not for themselves write for Wikipedia, and ask them to rate the articles as they would term papers. It does not matter if the sample includes articles outside of that professor's field since an encyclopedia might be a place that they would look if they were trying to inform themselves about it.
Ec