wjhonson(a)aol.com wrote:
With the right hand, you rate the raters. So each of
us gets a clue
stick and goes around whacking good editors "good rater" rate up a
notch by voting for them as raters.
With the left hand, you rate the articles, and when other editors agree
with you, they whack you and your "good rater" score goes up.
Now with the giant nose of Zenobia, you multiply the article rating by
the raters rating, and average.
Thusly and so, articles get a good rating based on the best raters
rating them good, and nasty bad evil raters, ratings fall into the
first circle (i.e. they are weighted as nothing).
Will Johnson
You should be able to do the whole thing in one go.
It's a bit like clocks; accurate clocks are defined to be those which
tend to give similar times to other accurate clocks. Inaccurate clocks
do not have this property.
In this case, good raters are defined to be those who give ratings which
tend to correlate well with true ratings, which are in turn extracted by
ratings given by other good raters. Even though this is necessarily a
recursive definition, it can still be used to generate a tractable set
of simultaneous equations.
"True" ratings being a matter of subjectivity, and people being people,
there may also be more than one mutually-coherent cluster of raters. If
you're worried about active coordinated attacks by ratings spammers, or
want to try to average across political viewpoints, you can seed things
with a core of users known to be likely to be both good and impartial
raters.
-- Neil