On 0, Casey Brown cbrown1023.ml@gmail.com scribbled:
It should also keep in mind what the summary the user is providing... like if someone were to do something very radical per OTRS, or do an urgent blank. These times of things are necessary but may give someone an unnecessary or bad rating.
Certainly that'd be a concern, but I'm reading through the linked paper, and they seem to have structured it to analyze text in two ways; one measure for the actual text itself, and another for the organization of the text.
But things seem to work out and avoid that problem:
"The fact that so few of the short-lived edits performed by high-reputation authors were judged to be of bad quality points to the fact that edits can be undone for reasons unrelated to quality. Many Wikipedia articles deal with current events; edits to those articles are undone regularly, even though they may be of good quality. Our algorithms do not treat in any special way current-events pages. Other Wikipedia edits are administrative in nature, tagging pages that need work or formatting; when these tags are removed, we classify it as text deletion..."
(My take is that the algorithm does penalize a user slightly for those listed actions, but nowhere near enough to really count, and that moving text between articles looks like simultaneous deletion and addition, so the plus and minuses would cancel out.)
-- gwern monarchist SGC 127 NAVELEXSYSSECENGCEN Z7 CACI POCSAG Ti cybercash Infrastructure