David Gerard wrote:
Another idea: make all ratings public information, because they're part of the process of working on the encyclopedia so should be viewable for transparency.
- d.
...which is also the correct answer to the problem of selecting any one particular rating algorithm in advance. By publishing the raw ratings data, with associated timestamp and userid, anyone will be able to analyze the data any way they like.
It would be fairly easy to institute such a rating system, given that we already have an integrated login and edittoken verification system: we just need a small form that generates an appropriate GET query on each page, and an extra table to stash the results into.
Dump that table into a downloadable text file at regular intervals, and you're done. Armies of programmers and statisticians will descend on the data to see what they can do with it. If they can do something useful with it, we could eventually integrate that analysis into the software. If not, the experiment can eventually be abandoned.
The last time we tried something like this, it degenerated into a massive discussion of which ratings parameters and rating methodology should be used[1], and nothing ever happened.
-- N.
[1] my suggestion: two rating parameters: importance and overall quality, each judged from 0 to 5. Matches current article rating system, has an even number of options. Three clicks, and you're done.