I think the first part of your claim needs to be substantiated first! Almost all of the content on the English Wikipedia, for example, is of low quality. http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Index On 8/29/07, Desilets, Alain <Alain.Desilets(a)nrc-cnrc.gc.ca> wrote:

I don't have a specific reference for you, just some comments. :) I think perhaps your original claim is too strong--on both counts. Including the word "can" as you did in your reply is important. Wikis CAN lead to high-quality content. And CAN lead to active communities. (Clearly there are examples of active communities and examples of wikis that never get used much.) I am not convinced that the technology itself leads to high-quality articles in Wikipedia, and I don't think that the Nature article supports that claim in any way. One could hypothesize about characteristics of wiki that help support the goal of producing high-quality articles and try to empirically verify the claim, but to my knowledge no one has done so in any systematic fashion. On 8/29/07, Desilets, Alain <Alain.Desilets(a)nrc-cnrc.gc.ca> wrote:

Desilets, Alain wrote: Hi Alain, The following work by Dan Cosley et al. argues that wikis and traditional review-before-publication result in the same quality, but wiki gets there faster: Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2006). Using Intelligent Task Routing and Contribution Review to Help Communities Build Artifacts of Lasting Value. Proc. CHI 2006. http://grouplens.org/papers/pdf/itr-chi2006.pdf (Full disclosure: this work is a product of my own research group.) HTH, Reid

On 8/29/07, Desilets, Alain <Alain.Desilets(a)nrc-cnrc.gc.ca> wrote: They lead to high-quality content. They also lead to low-quality content. They lead to all sorts of content. The vast majority of Wikipedia articles are low-to-mid quality, with most being also quite short; but this is simply because there are so very many of them. Despite the wiki technology, most of these articles only have one or two primary authors, as well. Most of this "long tail" of short articles are not comparable to Britannica or other encyclopedias, simply because Britannica does not include topics such as every episode of popular TV shows as "articles". I don't know of any published studies "proving" this distribution of quality per se -- in part because it's really hard to measure -- but it's pretty well known by contributors. You also, of course, have to come up with a good definition of "quality" -- obviously, Wikipedia meets some definitions, but completely fails others (like written by experts in the field). In part because of this, and because of other factors, the English Wikipedia is not, perhaps, the best wiki to look at when trying to make a statement about wiki technology in general. EN:WP has a highly developed ruleset, culture, practices and visibility that can't necessary be carried over to talking about other wikis, such as those with a smaller or restricted community. For communities, are you talking about wikis that foster communities that already exist (such as within a company) or wikis that *create* community by existing (like Wikipedia?) The latter seems true enough by example; I'm not sure about the former. By the way, the 1.0 project that Brian pointed to was rating by topic (trying to find good articles, not poor ones) and trying to find core articles that could be included in a CD version of WP. They worked with the WikiProjects and on their own to come up with lists within subjects. They've also been working on a "core topics" list: http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team/Core_topi… (and a variation) http://en.wikipedia.org/wiki/Wikipedia:Core_topics_-_1%2C000 Considering that the Nature study actually relied on a very similar methodology -- asking experts to subjectively rate articles -- it seems like a pretty valid comparison. The reviewers working on 1.0 are academics and experienced Wikipedians; the people working in the WikiProjects presumably have some subject knowledge. (I think Brian and I are proving the point here that if you want detailed criticism of Wikipedia, ask some long-time contributors :) ) best, phoebe

wiki editing is a skill, it is a technical skill but more so a social skill. different wikis will have different kinds of material and community based on - the initial goal of the project, - how clearly that is defined and whether - the participants understand and agree with the purpose and therefore the criteria for editing..this would be the case with a wikipedia modelled project. - if the project uses different criteria for editing, eg has authorised people or some other means of managing choices the outcomes and participation will vary accordingly. - the size diversity and commitment/interest of the audience - the skills of the participants to express themselves effectively in a space where negotiating included content on the basis of fit for purpose, or whatever model is used for choosing and editing material.

Sorry, what does WP:ACST? BTW: I love that quote. Is it yours? I'll be using it a lot in the future! ---- Alain Désilets, National Research Council of Canada Chair, WikiSym 2007 2007 International Symposium on Wikis Wikis at Work in the World: Open, Organic, Participatory Media for the 21st Century http://www.wikisym.org/ws2007/

I agree. That's why when we talk about the quality of WP versus traditional sources, it's important to focus on content that is covered by both types of resources. The fact that Wikipedia covers content that is not covered by Brittanica should not be held against the approach. It should be considered a strength, not a weakness! "Written by experts in the field" is a process-oriented definition of quality. Such definitions are always highly suspicious. They're often promoted by tool and process vendors who want to equate quality with their particular brand of tool or process. It's much better to define quality of a product in terms of attributes of the end product than in terms of the process by which it was achieved. Rereading the Nature article, I noticed that they talk about two dimensions of quality: - Accuracy of the information - Clarity of presentation While WP seems to do very well on accuracy, it seems many of the expert reviewers in the study complained about Clarity. So maybe the claim should be that: "Experience shows that wikis *can* lead to *comprehensive and accurate* content" I'm making the claim in the context of this article: http://wikimania2007.wikimedia.org/wiki/Proceedings:AD1 Which talks about creating a world-wide community of translators. OK, thx for the precisions. One way that this differs from the Nature study is that it does not compare WP's quality on those articles to the quality level on Brittanica. What is your impression on that? Would the low quality articles on that list fail the test of being comparable to Brittanica in terms of accuracy? Such self-criticism is a sure sign of a healthy community! ---- Alain Désilets, National Research Council of Canada Chair, WikiSym 2007 2007 International Symposium on Wikis Wikis at Work in the World: Open, Organic, Participatory Media for the 21st Century http://www.wikisym.org/ws2007/

Thx.

Desilets, Alain wrote: It's one of WP:SHORT's :) Which mean, shortcuts (http://en.wikipedia.org/wiki/Wikipedia:Shortcut); this one leads to http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_in_academic_studies, a project I adopted and updated this summer: in theory, it's supposed to hold a list of all academic works mentioning Wikipedia - in practice, we all need to keep updating it, just as any wiki page :) According to http://en.wikiquote.org/wiki/Wikipedia it is attributed (but not sourced!) to Stephen Colbert. I may not be a fan of the man, but the quote *is* good (I use it as a motto of my Wikipedia MT and article based on it, recently submitted for review). -- Piotr Konieczny "The problem about Wikipedia is, that it just works in reality, not in theory."

That's one interpretation of the confusion matrices on page 4, here: http://upload.wikimedia.org/wikipedia/wikimania2007/d/d3/RassbachPincockMin… 44% of FAs were classified correctly, but 17% were classified as A and 20% as good. 37% of As were classified correctly, but 15% were classified as FA and 14% as G 41% of Gs were classified correctly, but 20% were classified as FA, 14% as A and 10% as B Collapsing over these greatly improved classifier performance. One interpretation, which is partially true, is that not all of the features necessary to predict article quality were included. But another, which is also partially true, is that the quality classes used are artificial, and that not all articles are classified correctly. Nonetheless, I provided featured articles as an example because documentation of their loss in quality is documented, whereas in other classes it is not. And my own opinion is that the featured article process does promote quality. They are fairly easy for the classifier to pick out in most cases. Whether or not there is a large swath of articles which deserve FA status which do not have it is only up to speculation. We will have to wait for those articles to go through the process. It's certainly true that there are some hidden nuggets. On 8/30/07, Kat Walsh <kat(a)mindspillage.org> wrote: