[Wikipedia-l] some thoughts on peer-review and obscure subjects

[Delirium]

Marco Krohn wrote:

>On Thursday 26 August 2004 12:16, Tomasz Wegrzanowski wrote:
>  
>
>>On Thu, Aug 26, 2004 at 11:46:32AM +0200, Peter Gervai wrote:
>>    
>>
>
>  
>
>>>Why, this article is pretty fair. It states only the truth:
>>>* Wikipedia is not reviewed by the academics
>>>* Wikipedia does not guarantee validity or accuracy
>>>      
>>>
>>Neither are paper encyclopedias, but somehow he didn't have any problems
>>with them. This article is pure biased bullshit.
>>    
>>
>
>I haven't read the article, but AFAIK at least some parts of paper 
>encyclopedias are reviewed. At the Brockhaus they claim that every article 
>was at least reviewed three times. O.k. I can's validate this information 
>neither can I proof that it was always reviewed by academics. Nevertheless 
>Brockhaus lists >1000 external authors (many of have a PhD or higher degree) 
>which helped with the new edition. The latter is at least some indication 
>that their claims are not completely without base.
>
>In contrast to that we can not even say if a part of an article was read more 
>than once. 
>  
>
Although the wikievangelists will disagree with me, I do think this is 
somewhat of a problem.  Given a random article from an encyclopedia, 
there is a higher probability that there are errors, even major ones, in 
a Wikipedia article.  Of course, there are more Wikipedia articles, but 
this doesn't help in terms of "chance any given fact I look up is 
accurate."  It makes it difficult, without some additional heuristics, 
to use Wikipedia articles as anything but a pointer to further reading.

Now there are, of course, additional heuristics.  Articles which have 
many contributors, especially many well-respected contributors, have a 
much higher chance of being accurate, of having controversial material 
labeled as such (with all sides represented), and so on.  Articles 
written by only one or two people, on the other hand, can have almost 
anything: urban legends someone heard from a friend or off google and 
repeated; biased takes on an issue from a particular point of view with 
no mention of controversy over those views; and so on.  Those of us who 
are active Wikipedians have an easier time telling the two apart than a 
random person reading Wikipedia, who is not necessarily going to browse 
through the article's history and the contribution histories of the 
various contributors to the article.

So, as others have proposed, I do think we eventually need some sort of 
tagging system, so people know whether a particular article is a 
well-worked-over draft, or an initial draft that has undergone little to 
no review.

This is particularly important because one of Wikipedia's strengths is 
that we cover a lot of subjects that other encyclopedias do not cover 
_at all_.  Unfortunately, it is these obscure subjects where the 
probability that there are erroneous or slanted articles is the highest, 
because there are not very many Wikipedians knowledgeable enough to 
review such articles.  _But_, some of these articles do happen to be of 
very high quality, and these ought to somehow be marked.  Otherwise, the 
heuristic a lot of people will end up using (based on some discussions 
with non-Wikipedians who use it) is "on famous stuff, like [[Israel]] or 
[[George Washington]], Wikipedia is pretty good, but on obscure stuff 
you can't trust it."  We need to somehow indicate to people which of our 
obscure stuff they can put a higher degree of confidence in, since it's 
one of our strengths that we do have a lot of accurate, well-researched 
obscure stuff.

-Mark