[WikiEN-l] two-tiered ratings system (Was: To boldy delete what no one had deleted before!)

[Neil Harris]

wjhonson at aol.com wrote:
> With the right hand, you rate the raters.  So each of us gets a clue 
> stick and goes around whacking good editors "good rater" rate up a 
> notch by voting for them as raters.
>
> With the left hand, you rate the articles, and when other editors agree 
> with you, they whack you and your "good rater" score goes up.
>
> Now with the giant nose of Zenobia, you multiply the article rating by 
> the raters rating, and average.
>
> Thusly and so, articles get a good rating based on the best raters 
> rating them good, and nasty bad evil raters, ratings fall into the 
> first circle  (i.e. they are weighted as nothing).
>
> Will Johnson
>
>   

You should be able to do the whole thing in one go.

It's a bit like clocks; accurate clocks are defined to be those which 
tend to give similar times to other accurate clocks. Inaccurate clocks 
do not have this property.

In this case, good raters are defined to be those who give ratings which 
tend to correlate well with true ratings, which are in turn extracted by 
ratings given by other good raters. Even though this is necessarily a 
recursive definition, it can still be used to generate a tractable set 
of simultaneous equations.

"True" ratings being a matter of subjectivity, and people being people, 
there may also be more than one mutually-coherent cluster of raters. If 
you're worried about active coordinated attacks by ratings spammers, or 
want to try to average across political viewpoints, you can seed things 
with a core of users known to be likely to be both good and impartial 
raters.

-- Neil