Ian Woollard wrote:
On 14/01/2009, Ray Saintonge saintonge@telus.net wrote:
Not everybody pays attention to GA/FA. A public rating system where anyone can rate each article on a 0-10 scale might be controversial to implement, but on a cumulative basis would give a good statistically based valuation of the article.
Possibly not. The experience with these kinds of systems at Amazon for example shows that interpreting votes is not simple. A lot of people give consistently high, middle or low votes and there are many pathologies, averaging them out gives much worse results than you could expect.
That sounds like an interesting hidden-variable Bayesian estimation problem: given n reviewers of various propensities and m reviewed objects, and # of reviews >> n+m, make a joint maximum likelihood estimate of both the "true" properties of both reviewers and reviewed objects. Bogus "outlier" editors could just be modeled as all variance, with their mean irrelevant.
I'd be quite surprised if someone hasn't solved this already, and written a paper about it.
-- Neil