Wikipedians,
We're experimenting with and attempting to develop a new internet based source of reliable crowd sourcing information at http://canonizer.com.
The idea is to add open survey capabilities to a wiki system. "Topics" of controversial issues can have multiple 'camps' where various people that see things similarly can collaboratively develop, defend, and concisely state their POV. It is basically a tool to enable large groups of people to communicate concisely and quantitatively - rather than alone and individually. There are millions of blogs, and long lists of comments to the good ones out there, the question is, how do you know, concisely and quantitatively, what they are all saying about any important controversial issues? And which ones do the people agree are reliable?
One of our first efforts, in a proof of concept way, has been to start an open survey amongst experts on what are currently the best theories of consciousness. The question being: "Is there any kind of scientific consensus at all about what are the best theories of consciousness?" Often times a scientific consensus is claimed, or some experts have a general idea that there is a consensus, but how do you document such rigorously, quantitatively, and in a trusted way for all to accept? An example topic where experts are starting to concicely state and develop the best theories is this one on Theories of Mind and Consciousness:
http://canonizer.com/topic.asp/88
Once you have open survey capabilities like this, all that remains is knowing who are the trusted experts. This can be easily accomplished via a peer ranking process where all the experts rank each other in a top 10 kind of way. An example of this can be found in the "Mind Experts" topic here:
http://canonizer.com/topic.asp/81
And the associated canonization algorithm which uses the peer ranking data documented here:
http://canonizer.com/topic.asp/53/11
The default canonization algorithm is one person one vote. This results in a rigorous and quantitative measure of consensus amongst the general population (or at least of all participators in the open survey). There are various other algorithms that can be used to 'canonize' things as the reader may desire. Instead of filtering things on the way in, canonizer.com allows the browsers of the data to filter (or canonizer, if you will) things any way they want simply by selecting the canonization algorithm on the side bar. When the 'Mind Experts' canoniztion algorithm is selected on the side bar you get a rigorous and quantitative measure of scientific consensus.
The survey results are still far from comprehensive, but already there is a growing number of experts like Steven Lehar, John Smithies, Jonathan Edwards... are participating, basically declaring their beliefs in a dynamic and real time way about what they think are the best theories of consciousness. So far, the more experts that 'canonizer' their beliefs in this open survey the more the 'Consciousness is Representational and real' camp continues to extend it's lead in the amount of scientific consensus it has:
http://canonizer.com/topic.asp/88/6
There has already been several starting attempts to use this definitive information as references in various wikipedia articles on philosophy of mind. One example being the article on qualia. An initial proposal to include some of this data was made in the talk page of this article here:
http://en.wikipedia.org/wiki/Talk:Qualia#Proposed_addition_to_the_.22Scienti...
But this, and other similar entries on other pages were initially shut down by Jw2035 and a wiki war seems to be in process on this issue with possibly different points of view. A topic has been created at canonizer.com to consolidate the various descussions on different article talk pages, and to find out how much consensus there might be on both sides of this issue here (If there is any other real competing POV about the validity of such):
http://canonizer.com/topic.asp/104
A second attempt is now being proposed for the qualia article here:
http://en.wikipedia.org/wiki/Talk:Qualia#Smythies_section_needs_rewriting
in which it is simply being used to definitively document John Smythies (one of the somewhat arbitrary listed 'proponents of qualia') beliefs on this issue.
The ultimate goal would be as things become more developed, to have a quantitative measure of how trusted any particular 'camp' is. Obviously, anyone can create and support a camp, or a camp may not be supported at all. All such should be taken 'with a grain of salt'. But if there is a clear 'scientific consensus' supporting a camp, the degree to which it can be trusted goes up significantly and quantitatively. Perhaps in the future, various scientific publications might stipulate a quantitative value, when using a particular specified canonization algorithm, which a 'camp' must achieve before it can be used as a source in anything published in their peer reviewed scientific publication?
All the people involved in this open source volunteer developed project would love to know what all you wikipedians think about such efforts to 'measure' scientific consensus - and the using of such as trusted sources of information in wikipedia and elsewhere. Sure, no one can claim any of this is 'truth' (except for the fact of who currently believes what is true) - but what better measure of truth might there be than that for which there is a clear scientific consensus? And canonizer.com includes a historical mechanism (see the 'as of' control box on the side bar) so we can watch and rigorously document the various theories or 'camps' as they come and go as ever more scientific data comes in.
What do you all think?
Thanks
Brent Allsop
lennart guldbrandsson wrote:
In other words, the world is moving to the internet and if Wikipedia wants to stay ahead it will need to adapt.