On 8/29/07, Desilets, Alain <Alain.Desilets(a)nrc-cnrc.gc.ca> wrote:
Alain Désilets wrote:
I need a good solid reference to
substantiate the following claim:
"Besides leading to high quality content, wikis have been shown to be
good tools
for fostering the emergence of active communities"
Does anyone know of a good research paper that looks specifically at this
kind of
impact of wikis?
Brian replied:
I think the first part of your claim needs
to be substantiated first! Almost all of the
content on the English Wikipedia, for example, is of low quality.
http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index
I already have a solid reference to substantiate the fact that Wikipedia has quality
content, i.e. the Nature Magazne study of 2005:
http://www.nature.com/news/2005/051212/full/438900a.html
This study found that the quality of Wikipedia content was comparable to that of
Britannica (on a sample of 42 pages in various scientific domains).
The page you quote above does not constitute a scientific assessment of the quality of
WikiPedia pages. For one thing, it's not a random sample. These are pages that people
on the editorial board selected for review, and it could be that they naturally focused on
pages that were a-priori more likely to present quality issues. Second, it does not
compare the quality of wikipedia pages to the quality of corresponding pages on more
traditional resources.
The Nature study on the other hand was scientifically sound on both of those points. It
did a random sample of pages on a broad range of scientific topics. And it compared the
quality of those pages to comparable pages on Brittanica.
The Nature study is also consistent with my personal experience (and that of most people
I talk to) as a user of Wikipedia. In other words, whenever I go to a Wikipedia page on a
topic that I know well, I find it to be good quality and accurate.
So... Until someone shows me a scientifically sound demonstration that Wikipedia pages
are of significantly lower quality than other more traditional resources, I will continue
claiming that wikis like Wikipedia can lead to quality content.
They lead to high-quality content. They also lead to low-quality
content. They lead to all sorts of content.
The vast majority of Wikipedia articles are low-to-mid quality, with
most being also quite short; but this is simply because there are so
very many of them. Despite the wiki technology, most of these articles
only have one or two primary authors, as well. Most of this "long
tail" of short articles are not comparable to Britannica or other
encyclopedias, simply because Britannica does not include topics such
as every episode of popular TV shows as "articles". I don't know of
any published studies "proving" this distribution of quality per se --
in part because it's really hard to measure -- but it's pretty well
known by contributors. You also, of course, have to come up with a
good definition of "quality" -- obviously, Wikipedia meets some
definitions, but completely fails others (like written by experts in
the field).
In part because of this, and because of other factors, the English
Wikipedia is not, perhaps, the best wiki to look at when trying to
make a statement about wiki technology in general. EN:WP has a highly
developed ruleset, culture, practices and visibility that can't
necessary be carried over to talking about other wikis, such as those
with a smaller or restricted community. For communities, are you
talking about wikis that foster communities that already exist (such
as within a company) or wikis that *create* community by existing
(like Wikipedia?) The latter seems true enough by example; I'm not
sure about the former.
By the way, the 1.0 project that Brian pointed to was rating by topic
(trying to find good articles, not poor ones) and trying to find core
articles that could be included in a CD version of WP. They worked
with the WikiProjects and on their own to come up with lists within
subjects. They've also been working on a "core topics" list:
http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Core_topi…
(and a variation)
http://en.wikipedia.org/wiki/Wikipedia:Core_topics_-_1%2C000
Considering that the Nature study actually relied on a very similar
methodology -- asking experts to subjectively rate articles -- it
seems like a pretty valid comparison. The reviewers working on 1.0 are
academics and experienced Wikipedians; the people working in the
WikiProjects presumably have some subject knowledge.
(I think Brian and I are proving the point here that if you want
detailed criticism of Wikipedia, ask some long-time contributors :) )
best,
phoebe