[WikiEN-l] two-tiered ratings system (Was: To boldy delete what no one had deleted before!)

[Neil Harris]

Ian Woollard wrote:
> On 14/01/2009, Ray Saintonge <saintonge at telus.net> wrote:
>   
>> Not everybody pays attention to GA/FA. A public rating system where
>> anyone can rate each article on a 0-10 scale might be controversial to
>> implement, but on a cumulative basis would give a good statistically
>> based valuation of the article.
>>     
>
> Possibly not. The experience with these kinds of systems at Amazon for
> example shows that interpreting votes is not simple. A lot of people
> give consistently high, middle or low votes and there are many
> pathologies, averaging them out gives much worse results than you
> could expect.
>
>   
That sounds like an interesting hidden-variable Bayesian estimation 
problem: given n reviewers of various propensities and m reviewed 
objects, and # of reviews >> n+m, make a joint maximum likelihood 
estimate of both the "true" properties of both reviewers and reviewed 
objects. Bogus "outlier" editors could just be modeled as all variance, 
with their mean irrelevant.

I'd be quite surprised if someone hasn't solved this already, and 
written a paper about it.

-- Neil