Have a look at Ray King's essay:
http://www.pint2.org/index.php/Contributor_Ranking_System
Although this solution is aimed more generally at sites using Wikimedia software, this is something I believe Ray has been talking with Jimbo about. I happened to be present when Ray presented this the other night, & a known expert suggested an excellent way to grab some of the data needed without causing a strain on the servers.
And Ray is *very* interested in feedback. I hope he concentrates on a simplified version of this proposal, rather than trying to write the finished, with all of the bells-&-whistles version.
Geoff
This looks fantastic, but I'm concerned about the way edit value is calculated. Under the current proposal, as I understand it, copyeditors will be more "valuable" than those who contribute original content. Someone with poor writing skills might contribute a significant body of work, but if it is then rewritten, the original contributor's work won't appear on the page for very long and that person won't be ranked highly. But the copyeditor, whose better writing skills suggest that fewer people will modify the resulting text, will be considered more valuable because the text will remain on the page longer without alteration.
Furthermore, if a user connects all the wikis on which s/he works, those who work on multiple wikipedias would be at a disadvantage. Their article contributions in their non-native language will likely not remain unaltered for long due to stylistic problems, and that will damage their rating on their "home" wikipedia.
I'm not sure how to fix this without getting humans involved. Perhaps it's not a significant problem, since the best editors can both create content and do it in such a way as to make it readable. But it appears to me that average editors will be able to attain the same rating as above-average content creators.
All that said, though, this is a great idea. The ability to view each user's contributions to an article and judge their accuracy based on the author would be key in improving the credibility of wikis in general, and wikipedia specifically.
Nathaniel
On 3/17/06, Geoff Burling llywrch@rdrop.com wrote:
Have a look at Ray King's essay:
http://www.pint2.org/index.php/Contributor_Ranking_System
Although this solution is aimed more generally at sites using Wikimedia software, this is something I believe Ray has been talking with Jimbo about. I happened to be present when Ray presented this the other night, & a known expert suggested an excellent way to grab some of the data needed without causing a strain on the servers.
And Ray is *very* interested in feedback. I hope he concentrates on a simplified version of this proposal, rather than trying to write the finished, with all of the bells-&-whistles version.
Geoff
WikiEN-l mailing list WikiEN-l@Wikipedia.org To unsubscribe from this mailing list, visit: http://mail.wikipedia.org/mailman/listinfo/wikien-l
-- Nathaniel C. Sheetz http://www.personal.psu.edu/ncs124
Hi, Well it's certainly very interesting. I wonder how it can be gamed, and what the consequences that would be for Wikipedia? I also note that deletions don't seem to be covered anywhere. An author who deletes text could be making a valuable contribution...
Steve
On 3/17/06, Geoff Burling llywrch@rdrop.com wrote:
Have a look at Ray King's essay:
http://www.pint2.org/index.php/Contributor_Ranking_System
Although this solution is aimed more generally at sites using Wikimedia software, this is something I believe Ray has been talking with Jimbo about. I happened to be present when Ray presented this the other night, & a known expert suggested an excellent way to grab some of the data needed without causing a strain on the servers.
And Ray is *very* interested in feedback. I hope he concentrates on a simplified version of this proposal, rather than trying to write the finished, with all of the bells-&-whistles version.
Geoff
WikiEN-l mailing list WikiEN-l@Wikipedia.org To unsubscribe from this mailing list, visit: http://mail.wikipedia.org/mailman/listinfo/wikien-l
On Sat, 18 Mar 2006, Steve Bennett wrote:
Hi, Well it's certainly very interesting. I wonder how it can be gamed, and what the consequences that would be for Wikipedia? I also note that deletions don't seem to be covered anywhere. An author who deletes text could be making a valuable contribution...
Steve
On 3/17/06, Geoff Burling llywrch@rdrop.com wrote:
Have a look at Ray King's essay:
http://www.pint2.org/index.php/Contributor_Ranking_System
Although this solution is aimed more generally at sites using Wikimedia software, this is something I believe Ray has been talking with Jimbo about. I happened to be present when Ray presented this the other night, & a known expert suggested an excellent way to grab some of the data needed without causing a strain on the servers.
And Ray is *very* interested in feedback. I hope he concentrates on a simplified version of this proposal, rather than trying to write the finished, with all of the bells-&-whistles version.
I know that the possibility of "gaming" this tool has been raised -- but if we make the assumption that what Ray's Toolbar measures is based on the version currently viewed, then the fact someone contributed "fnord" 100 times (which was quickly deleted) is factored out.
As for negative deletions -- I think that should be handled in version 2.0 (where I expect the cross-Wiki feature will be implimented). What do you think, Ray?
Geoff
On gaming: No matter what we do, the system will be gameable, but if the rewards for doing so aren't too high (which in this case I don't think they are) and we monitor the system, we can likely stay ahead of the gamers. In the end, giving people the ability to quickly and easily see what other contributions an author has made gives the reader the chance to decide how credible the author is.
On deletions: The idea that deleting is also a valuable contribution is a very good point. But that does open a gaming hole in that one can add with one account and delete with the other. I suggest we leave that alone for now.
On getting this going: I completely agree that we should define a narrow and do-able set of requirements for "Phase I". To that end, I would greatly appreciate any help in separating Phase I, II and III features as well as any other thoughts and help in developing specs.
Thanks for the thoughts! - Ray.
On 3/18/06, Geoffrey Burling llywrch@agora.rdrop.com wrote:
On Sat, 18 Mar 2006, Steve Bennett wrote:
Hi, Well it's certainly very interesting. I wonder how it can be gamed, and what the consequences that would be for Wikipedia? I also note that deletions don't seem to be covered anywhere. An author who deletes text could be making a valuable contribution...
Steve
On 3/17/06, Geoff Burling llywrch@rdrop.com wrote:
Have a look at Ray King's essay:
http://www.pint2.org/index.php/Contributor_Ranking_System
Although this solution is aimed more generally at sites using Wikimedia software, this is something I believe Ray has been talking with Jimbo about. I happened to be present when Ray presented this the other
night,
& a known expert suggested an excellent way to grab some of the data needed without causing a strain on the servers.
And Ray is *very* interested in feedback. I hope he concentrates on a simplified version of this proposal, rather than trying to write the finished, with all of the bells-&-whistles version.
I know that the possibility of "gaming" this tool has been raised -- but if we make the assumption that what Ray's Toolbar measures is based on the version currently viewed, then the fact someone contributed "fnord" 100 times (which was quickly deleted) is factored out.
As for negative deletions -- I think that should be handled in version 2.0 (where I expect the cross-Wiki feature will be implimented). What do you think, Ray?
Geoff
-- Raymond King ray@aboutus.com 503.888.8808 (cell)
On 3/18/06, Ray King raymondking@gmail.com wrote:
On gaming: No matter what we do, the system will be gameable, but if the rewards for doing so aren't too high (which in this case I don't think they are) and we monitor the system, we can likely stay ahead of the gamers. In the end, giving people the ability to quickly and easily see what other contributions an author has made gives the reader the chance to decide how credible the author is.
What are the rewards? Equivocating vandalism? Ie: That's probably not vandalism, he has a rating of 80%! Possibly facilitating the road to adminship? Just brainstorming here.
Gaming could take the form of adding a paragraph of text to several different articles on a similar theme, rewording paragraphs meaninglessly, expanding what had been a concise definition to something fluffier.
Well, it doesn't sound too bad, I have to say. Bring it on.
Steve
I think the best idea is to keep this as an outside system and to encourage people to come up with alternative metrics. An API that would allow people to come up with different ranking systems (try to quantify best prose, best copyediting, most interesting, etc..) would be great. The model of the project that you've outlined sounds like a great start.
On 3/18/06, Ray King raymondking@gmail.com wrote:
On gaming: No matter what we do, the system will be gameable, but if the rewards for doing so aren't too high (which in this case I don't think they are) and we monitor the system, we can likely stay ahead of the gamers. In the end, giving people the ability to quickly and easily see what other contributions an author has made gives the reader the chance to decide how credible the author is.
On deletions: The idea that deleting is also a valuable contribution is a very good point. But that does open a gaming hole in that one can add with one account and delete with the other. I suggest we leave that alone for now.
On getting this going: I completely agree that we should define a narrow and do-able set of requirements for "Phase I". To that end, I would greatly appreciate any help in separating Phase I, II and III features as well as any other thoughts and help in developing specs.
Thanks for the thoughts! - Ray.
On 3/18/06, Geoffrey Burling llywrch@agora.rdrop.com wrote:
On Sat, 18 Mar 2006, Steve Bennett wrote:
Hi, Well it's certainly very interesting. I wonder how it can be gamed, and what the consequences that would be for Wikipedia? I also note that deletions don't seem to be covered anywhere. An author who deletes text could be making a valuable contribution...
Steve
On 3/17/06, Geoff Burling llywrch@rdrop.com wrote:
Have a look at Ray King's essay:
http://www.pint2.org/index.php/Contributor_Ranking_System
Although this solution is aimed more generally at sites using
Wikimedia
software, this is something I believe Ray has been talking with Jimbo about. I happened to be present when Ray presented this the other
night,
& a known expert suggested an excellent way to grab some of the data needed without causing a strain on the servers.
And Ray is *very* interested in feedback. I hope he concentrates on a simplified version of this proposal, rather than trying to write the finished, with all of the bells-&-whistles version.
I know that the possibility of "gaming" this tool has been raised -- but if we make the assumption that what Ray's Toolbar measures is based on the version currently viewed, then the fact someone contributed "fnord" 100 times (which was quickly deleted) is factored out.
As for negative deletions -- I think that should be handled in version
2.0
(where I expect the cross-Wiki feature will be implimented). What do you think, Ray?
Geoff
-- Raymond King ray@aboutus.com 503.888.8808 (cell) _______________________________________________ WikiEN-l mailing list WikiEN-l@Wikipedia.org To unsubscribe from this mailing list, visit: http://mail.wikipedia.org/mailman/listinfo/wikien-l