Hi everyone,
The June 2025 Research Showcase will be live-streamed next Wednesday, June 18, at 9:30 AM PT / 16:30 UTC. Find your local time here https://zonestamp.toolforge.org/1750264200. Our theme this month is *Ensuring Content Integrity on Wikipedia*.
*We invite you to watch via the YouTube stream: https://www.youtube.com/live/GgYh6zbrrss https://www.youtube.com/live/GgYh6zbrrss.* As always, you can join the conversation in the YouTube chat as soon as the showcase goes live.
Our presentations this month: The Differential Effects of Page Protection on Wikipedia Article QualityBy *Manoel Horta Ribeiro (Princeton University)*Wikipedia strives to be an open platform where anyone can contribute, but that openness can sometimes lead to conflicts or coordinated attempts to undermine article quality. To address this, administrators use “page protection"—a tool that restricts who can edit certain pages. But does this help the encyclopedia, or does it do more harm than good? In this talk, I’ll present findings from a large-scale, quasi-experimental study using over a decade of English Wikipedia data. We focus on situations where editors requested page protection and compare the outcomes for articles that were protected versus similar ones that weren’t. Our results show that page protection has mixed effects: it tends to benefit high-quality articles by preventing decline, but it can hinder improvement in lower-quality ones. These insights reveal how protection shapes Wikipedia content and help inform when it’s most appropriate to restrict editing, and when it might be better to leave the page open.
Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms By *Joshua Ashkinaze (University of Michigan)*Large language models (LLMs) are trained on broad corpora and then used in communities with specialized norms. Is providing LLMs with community rules enough for models to follow these norms? We evaluate LLMs' capacity to detect (Task 1) and correct (Task 2) biased Wikipedia edits according to Wikipedia's Neutral Point of View (NPOV) policy. LLMs struggled with bias detection, achieving only 64% accuracy on a balanced dataset. Models exhibited contrasting biases (some under- and others over-predicted bias), suggesting distinct priors about neutrality. LLMs performed better at generation, removing 79% of words removed by Wikipedia editors. However, LLMs made additional changes beyond Wikipedia editors' simpler neutralizations, resulting in high-recall but low-precision editing. Interestingly, crowdworkers rated AI rewrites as more neutral (70%) and fluent (61%) than Wikipedia-editor rewrites. Qualitative analysis found LLMs sometimes applied NPOV more comprehensively than Wikipedia editors but often made extraneous non-NPOV-related changes (such as grammar). LLMs may apply rules in ways that resonate with the public but diverge from community experts. While potentially effective for generation, LLMs may reduce editor agency and increase moderation workload (e.g., verifying additions). Even when rules are easy to articulate, having LLMs apply them like community members may still be difficult.
Best, Kinneret
Hello everyone,
This is a friendly reminder that this month's research showcase on *Ensuring Content Integrity on Wikipedia *will be starting in an hour at 9:30 AM PT / 16:30 UTC. *We invite you to watch via the YouTube stream: https://www.youtube.com/live/GgYh6zbrrss https://www.youtube.com/live/GgYh6zbrrss.*
Best, Kinneret
On Thu, Jun 12, 2025 at 8:12 PM Kinneret Gordon kgordon@wikimedia.org wrote:
Hi everyone,
The June 2025 Research Showcase will be live-streamed next Wednesday, June 18, at 9:30 AM PT / 16:30 UTC. Find your local time here https://zonestamp.toolforge.org/1750264200. Our theme this month is *Ensuring Content Integrity on Wikipedia*.
*We invite you to watch via the YouTube stream: https://www.youtube.com/live/GgYh6zbrrss https://www.youtube.com/live/GgYh6zbrrss.* As always, you can join the conversation in the YouTube chat as soon as the showcase goes live.
Our presentations this month: The Differential Effects of Page Protection on Wikipedia Article Quality By *Manoel Horta Ribeiro (Princeton University)*Wikipedia strives to be an open platform where anyone can contribute, but that openness can sometimes lead to conflicts or coordinated attempts to undermine article quality. To address this, administrators use “page protection"—a tool that restricts who can edit certain pages. But does this help the encyclopedia, or does it do more harm than good? In this talk, I’ll present findings from a large-scale, quasi-experimental study using over a decade of English Wikipedia data. We focus on situations where editors requested page protection and compare the outcomes for articles that were protected versus similar ones that weren’t. Our results show that page protection has mixed effects: it tends to benefit high-quality articles by preventing decline, but it can hinder improvement in lower-quality ones. These insights reveal how protection shapes Wikipedia content and help inform when it’s most appropriate to restrict editing, and when it might be better to leave the page open.
Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms By *Joshua Ashkinaze (University of Michigan)*Large language models (LLMs) are trained on broad corpora and then used in communities with specialized norms. Is providing LLMs with community rules enough for models to follow these norms? We evaluate LLMs' capacity to detect (Task 1) and correct (Task 2) biased Wikipedia edits according to Wikipedia's Neutral Point of View (NPOV) policy. LLMs struggled with bias detection, achieving only 64% accuracy on a balanced dataset. Models exhibited contrasting biases (some under- and others over-predicted bias), suggesting distinct priors about neutrality. LLMs performed better at generation, removing 79% of words removed by Wikipedia editors. However, LLMs made additional changes beyond Wikipedia editors' simpler neutralizations, resulting in high-recall but low-precision editing. Interestingly, crowdworkers rated AI rewrites as more neutral (70%) and fluent (61%) than Wikipedia-editor rewrites. Qualitative analysis found LLMs sometimes applied NPOV more comprehensively than Wikipedia editors but often made extraneous non-NPOV-related changes (such as grammar). LLMs may apply rules in ways that resonate with the public but diverge from community experts. While potentially effective for generation, LLMs may reduce editor agency and increase moderation workload (e.g., verifying additions). Even when rules are easy to articulate, having LLMs apply them like community members may still be difficult.
Best, Kinneret
--
Kinneret Gordon
Lead Research Community Officer
Wikimedia Foundation https://wikimediafoundation.org/
*Learn more about Wikimedia Research https://research.wikimedia.org/*
wiki-research-l@lists.wikimedia.org