As I've made clear already, I'm all for trying to find a way vouchsafe the reliability of those of our articles that actually are reliable. We've just got to make sure we don't shoot ourselves in the foot doing it!
I'd like to ask this: what is the purpose of this present certification proposal?
*If* you're trying to improve the credibility of Wikipedia among people who *really need* accurate information about the reliability of articles--people like students, teachers (at all levels), librarians, and anyone doing serious research--then I don't see how this proposal can work. Why should any of these sorts of people believe that an article *actually is* reliable, just because a dozen participants have pressed a button saying it's good--if it's possible that not a single actual expert has looked at it? We've all seen instances of articles that looked OK to nonexperts but that turned out to be, in the opinion of an expert that happens by, decidedly inadequate. It doesn't take an epistemologist to see that accuracy cannot be vouchsafed by a vote--10, or 100, or 1000 approving Wikipedians certainly *can* be wrong!
But that isn't the most important issue here. The most important issue is this: I suspect that, if I understand it correctly, implementing the proposal would actually *undermine* Wikipedia's credibility. Here is what all sorts of sober thinkers, not on the project, will think about it when they learn about how articles are certified: "The same people who *write* these articles, of uneven quality and obviously questionable credibility, are the ones who presume to certify that they're accurate? Well, if the people in charge of the project think *that's* how to guarantee the reliability of an article, that's reason to think the articles *aren't* reliable, and that the project shouldn't be taken seriously."
By contrast, the proposal I made not long ago (I know, I haven't followed up--I suddenly got very busy), of having a completely separate website, managed by actual subject area experts, containing a subset of Wikipedia articles, achieves the purpose (improving the credibility of some Wikipedia articles among librarians et al.) much more handily.
By no means am I saying that credentialled people, or "experts," are (because they're experts) *necessarily* reliable. Plenty of people with lots of credentials can't give us a trustworthy opinion as to the reliability of an article. But we can't do better, and it's the opinion of these people--the duly anointed "knowers" (the tongue's in cheek here)--that students, teachers, and others look up to for their benchmark as to what is currently known. That's a fact. It's not a WikiWiki fact or attitude, but it *is* a fact, and we can't change it. The more "elitist" proposal might not be "WikiWiki," but there IS no WikiWiki way to satisfy teachers, librarians, and serious researchers about the reliability of a WikiWiki.
Now, if satisfying the librarians et al. is not your aim with the proposal, I think that's too bad; but then I really do have to wonder what the point of this certification proposal is. Is it a way for Wikipedians to win kudos from each other? If so, can we please not do that? Wikipedia is *not*, I think, about building a community and winning kudos and stroking egos. It's first and foremost about building an *encyclopedia*.
Larry Adding a new sig! "We have now sunk to a depth at which the re-statement of the obvious is the first duty of intelligent men." --George Orwell
At 12:41 PM -0800 10/31/02, Larry Sanger wrote:
...It doesn't take an epistemologist to see that accuracy cannot be vouchsafed by a vote--10, or 100, or 1000 approving Wikipedians certainly *can* be wrong!
Sure its not fireproof, but it gives a measure of the degree of consensus behind the article, which has some interpretable value:
- Firstly it indicates that the article hasn't been vandalized and is not contentious - at least as far as the generalist editorial midwives are concerned.
- Secondly it shows that it has passed at least first levels of evolution - perhaps only of structure and linking. Interrelationships to other info is itself valuable even if the body of the article isn't top notch.
To suggest that a votes flag would highlight collective wikipedian ignorance on a subject implys the fearful belief that members are obliged to know everything. Isn't this belief opposite to the whole idea of a Wikipedia?
It sounds like the project is overshadowed by the old social expectation of the high standards expected of an encyclopedia publisher. The project is bound to lose credibiity if it fails to unload this expectation from immature pages. Clear labelling would dispell this and reflect that pages are a living, evolving work, as well as invite improved contributions.
All that would be necessary to gain credibility with students and librarians and experts is to accurately label the status of an article. Votes is only part of it. Another label could helpfully be added indicating whether the article has been reviewed by subject area experts or not.
An endorsement list would be the ultimate way to go, showing names/URLs of individuals or bodies who have accepted the page as OK/useful, along with a rating value. Amazon.com book ratings could be a first model - open to all with a star rating. Note that such a list could also reflect variation in evaluation and people could then follow links to those endorsers who diverged in their rating to find out why.
Another benefit if this is it could divert the energies of those with strongly diverging POVs from "vandalizing" the page. They could instead channel their energies into expressing their difference via the rating and creating a linked counter-page...always room for one more page.
Just 2 cents from an interested lurker
Peter.
Hi Larry, hi list!
As I've made clear already, I'm all for trying to find a way vouchsafe
the
reliability of those of our articles that actually are reliable.
We've
just got to make sure we don't shoot ourselves in the foot doing it!
Yes! As you and many other have already said, it's better do this in a separate project.
There's another reason that hasn't been mentioned (I think): Every sort of certification procedure makes Wikipedia more complicated. Even if it's (another) option in the preferences for logged in users, the results of the procedure would be shown to the passing by surfer (with a "flag" or whatever). Otherwise it would be useless. And it's not just a word, it's a whole new concept that you introduce this way.
But Wikipedia is already far too complicated! Please don't compare Wikipedia to Photoshop, Emacs, or your favorite complicated computer application, but to the good old UseMod software. I love all the new features we gained through the new software, but as a compromise between the editors and the only-readers (but could become editors) we're at the point where for every new feature on the screen an old one must go away. IMO, of course.
Or better two. How about making the special pages drop down box a user option?
Kurt
Larry et al. (I'm responding to several concerns in a single mail here),
I understand your point, and I agree with it to some extent: People who write articles aren't typically the best judges of their own work. But I think you're making a mistake by assuming that someone who would take part in a certification project that is a subset of Wikipedia would necessarily also be a) a contributor to the specific article/subject he certified, or even b) a contributor to Wikipedia at all. The experts you want could well use the system, see below.
Elian suggested a scheme where we would simply get aggregated ratings from everyone. I am familiar with such schemes (Kuro5hin uses it), and I have considered that option and decided to submit a different proposal, the team system. The idea here is that teams can make their own rules, and by selecting a team to trust, I select a whole ruleset according to which I want to view articles.
With aggregated ratings, on the other hand, I either have to trust the aggregation (which includes trolls and people I may not find trustworthy) or select individuals to trust, which is cumbersome (Slashdot uses such a system, it's very very much work to build a useful list of friends/foes). With teams, I just have to trust the team maintainers to keep a consistent policy and kick out people who violate the rules. [As an aside, automatic account creation is a serious problem even in communities with a more complex account creation process. Once you can get greater power by creating more accounts, people will do so. Our system should not allow such abuses.]
For example, I could decide to trust Team Nupedia. Team Nupedia might have a policy, defined on its team page in the [[Team:]] namespace, that members of the team must a) be experts in the subjects they certify, b) not be direct contributors to the articles they certify. They could also set a high certification thresholds, e.g. 5 members of the team must rate an article for it to be certified. This is just an example, you could make up the rules you want. Then if Team Nupedia gets enough members, it might provide many useful high quality evaluations of selected articles.
I think the modus operandi here would not necessarily be different from what you envision for a separate project, but have the advantage to be directly and visibly integrated into WP, thereby attracting more people (even if you want an expert-centric team, you will probably get more experts by addressing a larger sample of users).
Other teams might adopt more liberal approaches, trying to separate obvious low quality articles from possibly high quality articles (i.e. detecting egregious NPOV violations, spam, bad writing etc.). These teams might produce more output and be valuable to do basic filtering (which might go both ways -- I think it might be valuable to have negative certification, too, to detect bad articles; I believe it was Ed who suggested something similar). This is useful, but a separate goal from creating a truly trustworthy encyclopaedia -- in the team system, it can be accomplished within the same framework.
I think that such competing but yet cooperating teams with different rules are better than either forking the whole project or forking selected articles. I agree that one aspect of my proposal, namely that teams be approved by the "cabal", was not very well thought out, but I do think some selection process needs to take place, because if we tolerate troll teams, those might create bad articles just for the purpose of certifying them, regardless of them being replaced later; after all, it would always be possible to view the certified version.
So much for now. To Ed: Feel free to create the page you mentioned, but perhaps it is better to keep this discussion on the list until concerns are addressed. Also, let's not move too fast. Wikipedia has survived well without such a scheme for almost two years, and we should be careful with any changes that could alienate contributors or make Wikipedia in any way less valuable. Believe me, though, that I'm just as much in favor of openness and massive collaboration as everyone else here -- if I had the slightest belief that certification would undermine this in any way, I wouldn't suggest it.
Peace,
Erik
wikipedia-l@lists.wikimedia.org