Hi Bowen, after reading your project proposal I have a few questions and concerns.
You mention a perceived tension between protecting newcomers and protecting the quality of content. I am wondering whether that is a false dichotomy. In my experience, test edits and blatant vandalism usually look different from mistakes from good faith editors.
There is a feature that allows users to adjust ORES-supported edit scoring in our watchlists and Recdent Changes: https://www.mediawiki.org/wiki/Edit_Review_Improvements/New_filters_for_edit.... Have you tested this feature? How would your research be useful for that feature's future development?
I think that ORES is supposed to aid human judgment, not to substitute for human judgment. How certain are you that "ORES applications will play a role in drawing a line between acceptable freestyle edits and editing policies in standard."? There may well be some human patrollers who adjust their definitions for vandalism based on ORES recommendations, but I think that you would want to know to what extent ORES has that effect.
I would also like to mention that Wikipedia policies and guidelines, like offline human laws and customs, may change over time, may have varying interpretations, and may have varying degrees of adherence among the populace.
Thanks for your interest in studying ORES. I am glad that you are collaborating with Aaron.
On Thu, Dec 13, 2018, 7:08 AM Bowen Yu bowen-yu@umn.edu wrote:
Hello,
ORES has been out and served for the Wikipedia community for a while, for the purpose such as counter-vandalism. Having seen the wide usage and effectiveness of ORES in the community, we'd like to continue working on ORES development. We plan to improve and redesign ORES algorithms by incorporating feedbacks of all the stakeholders involved in the entire ORES ecosystem, such as ORES application developers, ORES application operators, etc. We want to understand their concerns and values, and come up with effective algorithmic designs that can balance trade-offs and mitigate potential conflicts of interests (such as edit quality control v.s. newcomer protection) to further improve ORES performance.
We will work with Aaron Halfaker and his team to make improvements on ORES quality control models, and identify its limitations. Here is the project proposal on Meta-Wiki < https://meta.wikimedia.org/wiki/Research:Applying_Value-Sensitive_Algorithm_...
.
If you are interested or have any thoughts, please feel free to reach out to me. Thanks! _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l