On 8/29/07, Desilets, Alain Alain.Desilets@nrc-cnrc.gc.ca wrote:
Alain Désilets wrote:
I need a good solid reference to substantiate the following claim: "Besides leading to high quality content, wikis have been shown to be good tools
for fostering the emergence of active communities"
Does anyone know of a good research paper that looks specifically at this kind of
impact of wikis?
Brian replied:
I think the first part of your claim needs to be substantiated first! Almost all of the
content on the English Wikipedia, for example, is of low quality.
http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index
I already have a solid reference to substantiate the fact that Wikipedia has quality content, i.e. the Nature Magazne study of 2005:
http://www.nature.com/news/2005/051212/full/438900a.html
This study found that the quality of Wikipedia content was comparable to that of Britannica (on a sample of 42 pages in various scientific domains).
The page you quote above does not constitute a scientific assessment of the quality of WikiPedia pages. For one thing, it's not a random sample. These are pages that people on the editorial board selected for review, and it could be that they naturally focused on pages that were a-priori more likely to present quality issues. Second, it does not compare the quality of wikipedia pages to the quality of corresponding pages on more traditional resources.
The Nature study on the other hand was scientifically sound on both of those points. It did a random sample of pages on a broad range of scientific topics. And it compared the quality of those pages to comparable pages on Brittanica.
The Nature study is also consistent with my personal experience (and that of most people I talk to) as a user of Wikipedia. In other words, whenever I go to a Wikipedia page on a topic that I know well, I find it to be good quality and accurate.
So... Until someone shows me a scientifically sound demonstration that Wikipedia pages are of significantly lower quality than other more traditional resources, I will continue claiming that wikis like Wikipedia can lead to quality content.
They lead to high-quality content. They also lead to low-quality content. They lead to all sorts of content.
The vast majority of Wikipedia articles are low-to-mid quality, with most being also quite short; but this is simply because there are so very many of them. Despite the wiki technology, most of these articles only have one or two primary authors, as well. Most of this "long tail" of short articles are not comparable to Britannica or other encyclopedias, simply because Britannica does not include topics such as every episode of popular TV shows as "articles". I don't know of any published studies "proving" this distribution of quality per se -- in part because it's really hard to measure -- but it's pretty well known by contributors. You also, of course, have to come up with a good definition of "quality" -- obviously, Wikipedia meets some definitions, but completely fails others (like written by experts in the field).
In part because of this, and because of other factors, the English Wikipedia is not, perhaps, the best wiki to look at when trying to make a statement about wiki technology in general. EN:WP has a highly developed ruleset, culture, practices and visibility that can't necessary be carried over to talking about other wikis, such as those with a smaller or restricted community. For communities, are you talking about wikis that foster communities that already exist (such as within a company) or wikis that *create* community by existing (like Wikipedia?) The latter seems true enough by example; I'm not sure about the former.
By the way, the 1.0 project that Brian pointed to was rating by topic (trying to find good articles, not poor ones) and trying to find core articles that could be included in a CD version of WP. They worked with the WikiProjects and on their own to come up with lists within subjects. They've also been working on a "core topics" list: http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Core_topic... (and a variation) http://en.wikipedia.org/wiki/Wikipedia:Core_topics_-_1%2C000
Considering that the Nature study actually relied on a very similar methodology -- asking experts to subjectively rate articles -- it seems like a pretty valid comparison. The reviewers working on 1.0 are academics and experienced Wikipedians; the people working in the WikiProjects presumably have some subject knowledge.
(I think Brian and I are proving the point here that if you want detailed criticism of Wikipedia, ask some long-time contributors :) )
best, phoebe