One key problem with a wiki encyclopaedia is that there's no quality control whatsoever. An article may have been vandalized 5 seconds ago, or be grossly non-NPOV etc. As we get more and more articles, this problem becomes more urgent.
Fortunately, the solution is rather simple. Articles can be certified by contributors to be high quality. But who is allowed to certify articles? The system works by allowing groups of people to form certification teams. Anyone can submit a new team to be created, and anyone can apply to join an existing team and certify articles in its name. Users can then decide to view only article revisions certified by members of selected teams.
So I could decide in my user preferences: Certification: Approved Teams Team Nupedia Team Wiki-Fiction Team Wiki-Maths
Then there would have to be a way to display certified article revisions. This could be accomplished by having a "Certified Mode", showing *only* articles that have received certs, with the most recently certified revision shown. Somewhat weaker, where an article has been certified, a link "There is a version of this article certified by Team X" could be placed above the article, showing the certified revision when clicked (or a text "This article has been certified by .." if the current revision is the certified one). This could be the default view, making users aware of the cert system.
Each team could have its own quality standards, policies, and subject preferences. I suggest that the creation of new teams would have to be approved by the Wikipedia cabal to avoid "Team Trolls". New team members would either be voted on or approved by team members that have a certain status flag ("can_approve_newcomers"). Teams could get their own namespace as well.
A decision would have to be made as to which teams to include in the default view, i.e. the one that anonymous and newly registered users get. In the short term such decisions may be made by the cabal, in the long term I would prefer voting.
Implementation:
There needs to be a teams table that at least has a TID and a TDesc, and a team-member table that links TIDs and UIDs and grants individual users certain permissions within the team, as well as a "pending" status flag for newly applied members. A TCert table would link UIDs, TIDs certs and article IDs (must include timestamp).
I am currently assuming that certifications would be simple "Good article" binary flags. My reasoning is that a rating of "good, but not good enough" may not be very helpful. Still, practice may prove that wrong, and the table should therefore be designed to accomodate possibly more flexible ratings.
Giving each individual member the power to rule an article certified in the name of the whole team may be undesirable. Thus, teams should be configurable to set their own min_number_of_certs, where certifications would only be valid if so many team numbers agree that the article is complete and high quality.
Finally, the PHP scripts of course have to be updated to reflect this functionality. There needs to be some way to apply for team membership, to approve team membership, and to submit a new team for cabal approval.
A module listing all articles approved by a certain team would be nice and could substitute the "Brilliant Prose" page.
Results: -------- If this works as intended, it should solve the quality problem and allow users to browse Wikipedia as a high quality content only encyclopaedia. The more teams you would admit to your personal filter, the more content you would see, but quality standards of individual teams might not be up to par. By distributing the job of quality approval on several team leaders, we can get competition of quality standards and social methods, which is probably a good thing and reduces social problems.
Potential problems: ------------------- If too many people use highly customized views, caching will get harder. I don't see this as too big a problem as a) most people typically don't customize views, b) article retrieval is already very fast with or without caching.
Too many teams may have undesired effects, such as teams deliberately inserting POV articles to certify them. This is not a problem with the team principle per se but with the way teams are approved and moderated. Generally, teams should have a clear NPOV commitment and respect Wikipedia policy, otherwise they should be deleted.
Comments on this would be appreciated. This is something I probably won't have time to implement fully, but I will gladly help with any/all efforts. I consider it very necessary for Wikipedia in the long term.
Regards,
Erik Moeller
[Note: The post that I'm replying to didn't appear on <wikipedia-l>. Thus I copy it all, except for the technical aspects.]
Erik Moeller wrote:
One key problem with a wiki encyclopaedia is that there's no quality control whatsoever. An article may have been vandalized 5 seconds ago, or be grossly non-NPOV etc. As we get more and more articles, this problem becomes more urgent.
Fortunately, the solution is rather simple. Articles can be certified by contributors to be high quality. But who is allowed to certify articles? The system works by allowing groups of people to form certification teams. Anyone can submit a new team to be created, and anyone can apply to join an existing team and certify articles in its name. Users can then decide to view only article revisions certified by members of selected teams.
So I could decide in my user preferences: Certification: Approved Teams Team Nupedia Team Wiki-Fiction Team Wiki-Maths
Then there would have to be a way to display certified article revisions. This could be accomplished by having a "Certified Mode", showing *only* articles that have received certs, with the most recently certified revision shown. Somewhat weaker, where an article has been certified, a link "There is a version of this article certified by Team X" could be placed above the article, showing the certified revision when clicked (or a text "This article has been certified by .." if the current revision is the certified one). This could be the default view, making users aware of the cert system.
Each team could have its own quality standards, policies, and subject preferences. I suggest that the creation of new teams would have to be approved by the Wikipedia cabal to avoid "Team Trolls". New team members would either be voted on or approved by team members that have a certain status flag ("can_approve_newcomers"). Teams could get their own namespace as well.
You seem to be using the word "cabal" here in a sense that is neither derogatory nor ironic. I find that highly disturbing.
A decision would have to be made as to which teams to include in the default view, i.e. the one that anonymous and newly registered users get. In the short term such decisions may be made by the cabal, in the long term I would prefer voting.
If newcomers see only what is approved by a list of certification teams, then Wikipedia will no longer be a wiki. There will be a wiki underneath, which you can get to by registering and then setting your preferences, but that wiki would be dead without an influx of newcomers.
[technical aspects cut]
Results:
If this works as intended, it should solve the quality problem and allow users to browse Wikipedia as a high quality content only encyclopaedia. The more teams you would admit to your personal filter, the more content you would see, but quality standards of individual teams might not be up to par. By distributing the job of quality approval on several team leaders, we can get competition of quality standards and social methods, which is probably a good thing and reduces social problems.
Potential problems:
If too many people use highly customized views, caching will get harder. I don't see this as too big a problem as a) most people typically don't customize views, b) article retrieval is already very fast with or without caching.
Too many teams may have undesired effects, such as teams deliberately inserting POV articles to certify them. This is not a problem with the team principle per se but with the way teams are approved and moderated. Generally, teams should have a clear NPOV commitment and respect Wikipedia policy, otherwise they should be deleted.
Comments on this would be appreciated. This is something I probably won't have time to implement fully, but I will gladly help with any/all efforts. I consider it very necessary for Wikipedia in the long term.
-- Toby
You seem to be using the word "cabal" here in a sense that is neither derogatory nor ironic. I find that highly disturbing.
You may find it disturbing, but the truth is that there are people like Jimbo who can make high level decisions, and there are sysops that have more power than ordinary users. Many of these decisions already happen in places that most people don't know about (e.g. the mailing list). Glossing over the truth doesn't make it go away: there is a Wikipedia cabal.
I have already suggested a voting scheme that would democratize the decision processes by the inevitable administration. Aside from extreme opinions like "voting doesn't work", I see few arguments against that. We need to talk openly about this kind of stuff, or what you find disturbing will turn into a nightmare eventually.
A decision would have to be made as to which teams to include in the default view, i.e. the one that anonymous and newly registered users get. In the short term such decisions may be made by the cabal, in the long term I would prefer voting.
If newcomers see only what is approved by a list of certification teams, then Wikipedia will no longer be a wiki. There will be a wiki underneath, which you can get to by registering and then setting your preferences, but that wiki would be dead without an influx of newcomers.
You misunderstand me. I am absolutely in favor of creating and keeping a site that is immediately accessible to the newcomer, where the newbie quickly notices that WP is editable and joins the process. I love wikis! My idea centers around the facts that - there will always be many people who just read and who will never be contributors, - even for contributors, it is sometimes desirable to quickly find trustworthy information, - some contributors would like there to be some distinction between the work they have invested much time in and the vandalism of a bored Internet hooligan.
So what I am suggesting is an alternative viewing mode that would *never* be the default but optional. It would allow me to browse a Wikipedia where the article about Mozart *can* not just have been replaced by an image from goatse.cx. Instead, I would view the last certified version of that article, which hopefully would be brilliant prose.
Instead of just telling people to use Britannica if they want trustworthy information, we should substitute this part of Britannica -- quality control -- as well. That doesn't mean we have to give up any part of the massively collaborative project that WP is becoming. As a matter of fact, it would hopefully attract all those skeptics that are afraid that something like the above happens to their sacro-sanct articles -- it could still happen, but they could find peace of mind in the fact that the vandalized version would never be certified.
Last but not least, we should not forget that WP is intended to be a useful tool, not just for those who like to write, but for those who like to read, too. Be it interested adults or curious children, rich or poor, we want Wikipedia to be accessible. We may want to distribute it on CD- ROMs and on paper. Then how on Earth are the schoolchildren in India going to wade through megabytes of Middle Earth mythology and stubs, if not if we supply them with at least the option to filter articles according to criteria developed collaboratively by various teams, working together to find the sparkling gems among the ocean we are creating?
"Being a wiki" doesn't mean that we shouldn't extend the original wiki functionality. Had we stayed with that, Wikipedia would have never become that big. We would still be using CamelCase.
Regards,
Erik
erik_moeller@gmx.de wrote:
You seem to be using the word "cabal" here in a sense that is neither derogatory nor ironic. I find that highly disturbing.
You may find it disturbing, but the truth is that there are people like Jimbo who can make high level decisions, and there are sysops that have more power than ordinary users. Many of these decisions already happen in places that most people don't know about (e.g. the mailing list). Glossing over the truth doesn't make it go away: there is a Wikipedia cabal.
Hush! ;-)
I have already suggested a voting scheme that would democratize the decision processes by the inevitable administration. Aside from extreme opinions like "voting doesn't work", I see few arguments against that. We need to talk openly about this kind of stuff, or what you find disturbing will turn into a nightmare eventually.
Which reminds me: For future issues, we could set up a special "voting" page at the wikipedia, where proposals can be submitted (with several options to choose from, including "just leave it as it is"), where every logged-in wikipedian can vote (once!), with a time limit (say, one or two weeks) so everyone get a chance to vote, but it won't hang there forever. Democratic enough?
<snip explanation>
Why not use Larry's proposed read-only 'pedia instead? A new skin and a few restrictions (that would be set to "unrestricted" for wikipedia, of course), a nice little server, done :-)
Magnus
Which reminds me: For future issues, we could set up a special "voting" page at the wikipedia, where proposals can be submitted (with several options to choose from, including "just leave it as it is"), where every logged-in wikipedian can vote (once!), with a time limit (say, one or two weeks) so everyone get a chance to vote, but it won't hang there forever. Democratic enough?
I've been thinking about this for a while, and currently I'm tending towards a scheme where a user would insert a magic link into a page (talk or otherwise)
[[=poll=]]
or something like that. Rendered, this would first show a red link
Create a poll
On that page, the user could then set poll questions and voting type (I would love to allow both preferential voting and normal voting, and more voting systems could be added later). The poll would then be given an ID and the magic link would be changed to
[[=poll:id=]]
where id would be the poll's internal ID number. Rendered, this would then put a nice selection box
option [ ] option [ ]
on the page. For preferential voting, the [ ] would be an entryfield where the user enters a number from 1 to n, n being the number of options. Otherwise the standard radiobutton.
The poll creator and sysops would have permission to modify the poll. On modification, the poll votes would be reset.
There should be a way to show recent polls, and the poll creation page should require a textfield like this:
Please describe the purpose of the poll in terms that are understandable to people not yet involved in the debate who might be interested. _______________________________ _______________________________
I'm currently torn as to whether we want to record and possibly show who voted how. This has benefits in many situations, it makes abuses visible and creates a better picture of the opinion spectrum. The downside is possible peer pressure to vote a certain way. If it's visible, it should probably only be shown once a user has voted. Abuses could become a serious issue as accounts are trivial to create on WP. IP recording (maybe as hash? we record them clearly in many places anyway) is essential to avoid duplicate votes from a single IP.
If we have such a system in place, we can give sysops the permission to create special polls:
[[=spoll=]]
which would be a poll only usable by sysops but visible to anyone. Such polls could show up in another list, Recent admin polls, so that admins could easily see what new issues have come up.
What do you think about such a scheme?
Regards,
Erik
On 30 Oct 2002 erik_moeller@gmx.de wrote:
You seem to be using the word "cabal" here in a sense that is neither derogatory nor ironic. I find that highly disturbing.
You may find it disturbing, but the truth is that there are people like Jimbo who can make high level decisions, and there are sysops that have more power than ordinary users. Many of these decisions already happen in places that most people don't know about (e.g. the mailing list). Glossing over the truth doesn't make it go away: there is a Wikipedia cabal.
TINC.
Cabal implies secrecy, what we have here is a benevolent dictatorship.
Imran
Summary: I read today that only 6% of the people on the planet have ever logged into the Internet. Only about half have ever used a telephone. So Erik's concerns about schoolchildren in India do resonate with me. I just think it's too early to settle on any particular certification scheme.
erik_moeller@gmx.de wrote:
You may find it disturbing, but the truth is that there are people like Jimbo who can make high level decisions, and there are sysops that have more power than ordinary users. Many of these decisions already happen in places that most people don't know about (e.g. the mailing list). Glossing over the truth doesn't make it go away: there is a Wikipedia cabal.
To the extent this is true, we should work to make it less true. Keep in mind that sysop status is open to anyone. Sysop powers should be kept to just what is absolutely minimally necessary. The mailing list should be (and is, I think) promoted to such an extent that anyone with an interest in policy can understand that this is where to come for most policy discussions, etc.
The problem with getting comfortable with a cabal is that it will tend to make us lazy about our commitment to openness.
I have already suggested a voting scheme that would democratize the decision processes by the inevitable administration. Aside from extreme opinions like "voting doesn't work", I see few arguments against that. We need to talk openly about this kind of stuff, or what you find disturbing will turn into a nightmare eventually.
Well, voting _doesn't_ work, and I don't think that's an extreme opinion.
The wiki procedure is to try to get near unanimous consensus. This is done in the writing of articles, and is inherent to the wiki process. Authors work together to meet the objections of others, not to campaign for sufficient numbers to vote down the opposing point of view. We simulate that process, imperfectly but to a large degree, in policy decision making.
So what I am suggesting is an alternative viewing mode that would *never* be the default but optional. It would allow me to browse a Wikipedia where the article about Mozart *can* not just have been replaced by an image from goatse.cx. Instead, I would view the last certified version of that article, which hopefully would be brilliant prose.
I think it is inevitable that we, or someone, will someday create such a thing. One proposal sometimes bandied about is that Nupedia could become a repository for peer-reviewed articles from wikipedia.
As a matter of fact, it would hopefully attract all those skeptics that are afraid that something like the above happens to their sacro-sanct articles -- it could still happen, but they could find peace of mind in the fact that the vandalized version would never be certified.
I think that the tiny percentage of people who treat their own writing as sacrosanct will not be comfortable with any free-license writing project.
Last but not least, we should not forget that WP is intended to be a useful tool, not just for those who like to write, but for those who like to read, too. Be it interested adults or curious children, rich or poor, we want Wikipedia to be accessible. We may want to distribute it on CD- ROMs and on paper. Then how on Earth are the schoolchildren in India going to wade through megabytes of Middle Earth mythology and stubs, if not if we supply them with at least the option to filter articles according to criteria developed collaboratively by various teams, working together to find the sparkling gems among the ocean we are creating?
"Being a wiki" doesn't mean that we shouldn't extend the original wiki functionality. Had we stayed with that, Wikipedia would have never become that big. We would still be using CamelCase.
I totally agree with all of this. But I'm not sure that the time is ripe just yet. Perhaps in another year. I'm open to alternative points of view on this, but I think we're not close enough yet to "well rounded" to start seriously worrying about these issues.
--Jimbo
Am Don, 2002-10-31 um 13.42 schrieb Jimmy Wales:
I totally agree with all of this. But I'm not sure that the time is ripe just yet. Perhaps in another year. I'm open to alternative points of view on this, but I think we're not close enough yet to "well rounded" to start seriously worrying about these issues.
If there's any interest at all in this among other developers, how about working on it on a separate CVS branch? Then we could try it on, say, meta.wp and merge it into HEAD + use it on wikipedia.org when we're happy with the way it works.
Personally, I think that we're reaching the point where we need this because WP is becoming too big for us to oversee, so we can't really guarantee that it is crap-free.
Regards,
Erik
Erik Moeller wrote:
I totally agree with all of this. But I'm not sure that the time is ripe just yet. Perhaps in another year. I'm open to alternative points of view on this, but I think we're not close enough yet to "well rounded" to start seriously worrying about these issues.
If there's any interest at all in this among other developers, how about working on it on a separate CVS branch? Then we could try it on, say, meta.wp and merge it into HEAD + use it on wikipedia.org when we're happy with the way it works.
I take it you mean the "sysops and truster reviews only" thingy? Can-do, sure. But, honestly, I don't think I'd go around checking ~90000 articles, including 35000 one-cow-metropoles ;-) IMHO that experiment would fail not because of missing technology, but of lack of reviewers. I'd prefer "Larrypedia" myself, where people dedicated to reviewing can work their magic.
Personally, I think that we're reaching the point where we need this because WP is becoming too big for us to oversee, so we can't really guarantee that it is crap-free.
We never did! Our warranty always was and is "good luck"! :-)
Magnus
I take it you mean the "sysops and truster reviews only" thingy?
The idea is that anyone can join one of the teams, if the existing team members approve him. Elian suggested to use individual ratings instead, but I think teams are important because they represent aggregated trust. Ratings themselves can be low-quality, so you need a reputation system of some type to get the good ones. Nevertheless, the team thing should be very community-like and open, a bit like SETI@Home, where competition and cooperation work well together.
I don't think the one cow articles would ever be certified, but when we import information from a trusted source like the US Bureau of Census, we could certify all those articles under that name, provided the bot has not made mistakes.
Regards,
Erik
Jimmy Wales wrote:
Summary: I read today that only 6% of the people on the planet have ever logged into the Internet. Only about half have ever used a telephone. So Erik's concerns about schoolchildren in India do resonate with me. I just think it's too early to settle on any particular certification scheme.
erik_moeller@gmx.de wrote:
Last but not least, we should not forget that WP is intended to be a useful tool, not just for those who like to write, but for those who like to read, too. Be it interested adults or curious children, rich or poor, we want Wikipedia to be accessible. We may want to distribute it on CD- ROMs and on paper. Then how on Earth are the schoolchildren in India going to wade through megabytes of Middle Earth mythology and stubs, if not if we supply them with at least the option to filter articles according to criteria developed collaboratively by various teams, working together to find the sparkling gems among the ocean we are creating?
I totally agree with all of this. But I'm not sure that the time is ripe just yet. Perhaps in another year. I'm open to alternative points of view on this, but I think we're not close enough yet to "well rounded" to start seriously worrying about these issues.
The concept is intriguing, but I agree with Jimbo that we still have a long way to go before we can have any positive impact on the third world. Just sending them CDs containing "our" encyclopaedia will likely not accomplish very much. In some of these places the best use for the CDs would be as frisbees unless they have adequate hardware.
If, as some of us believe, a project like Wikipedia is to have any influence in the third world it would likely be practical to concentrate efforts in some selected community or country. We would likely need to find some co-operative arrangement with the target country, notably with its education bureaucracy. Even if there is enough hardware infrastructure simply sending CDs would not be a useful effort, and we would still have some expense in producing all those CDs. Encarta or some other large company that is now producing CD-encyclopaedias could dump its obsolete stock there much more efficiently than we can.
An interesting possibility would arise if we could transfer the "wiki way" along with the encyclopaedias. Middle Earth mythology may very well be alien to these cultures, but the wiki way could be applied in a study of local literature and mythology. I view the critical challenge for education is to communicate critical thinking skills; this has become just as important as the traditional 3 Rs. Third world countries cannot hope cannot hope to advance without that skill being present in very broad cross-sections of their populations. I believe that Wikipedia could eventually help there.
It's just something to think about. Eclecticology
wikitech-l@lists.wikimedia.org