Cross-posting from: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#WikiProject_assessments_vs._external_reviewers_vs._ORES

Hey folks, I have been collaborating with some researchers who are publishing a dataset of externally reviewed Wikipedia articles (the sample was taken back in 2006). I'd like to take the opportunity to compare the prediction quality of ORES' article quality model with these external reviewers, but in order get a good picture of the situation, it would also be very helpful to get a set of Wikipedian assessments[1] for the same dataset. So, I have gathered all of the versions of externally reviewed articles in User:EpochFail/ORES_audit[2] and I'm asking for your help to gather assessments. There's 90 old revisions of articles that I need your help assessing. I don't think this will take long, but I need to borrow your judgement here to make sure I'm not biasing things.

To help out, see User:EpochFail/ORES_audit[2].

The more we know about how ORES performs against important baselines, the better use of it we can make it to measure Wikipedia and direct wiki work.

1. https://en.wikipedia.org/wiki/Wikipedia:WikiProject_assessment
2. https://en.wikipedia.org/wiki/User:EpochFail/ORES_audit

Thanks!

-Aaron