Hi everyone,
The June 2025 Research Showcase will be live-streamed next Wednesday, June
18, at 9:30 AM PT / 16:30 UTC. Find your local time here
<https://zonestamp.toolforge.org/1750264200>. Our theme this month is *Ensuring
Content Integrity on Wikipedia*.
*We invite you to watch via the YouTube
stream: https://www.youtube.com/live/GgYh6zbrrss
<https://www.youtube.com/live/GgYh6zbrrss>.* As always, you can join the
conversation in the YouTube chat as soon as the showcase goes live.
Our presentations this month:
The Differential Effects of Page Protection on Wikipedia Article QualityBy
*Manoel Horta Ribeiro (Princeton University)*Wikipedia strives to be an
open platform where anyone can contribute, but that openness can sometimes
lead to conflicts or coordinated attempts to undermine article quality. To
address this, administrators use “page protection"—a tool that restricts
who can edit certain pages. But does this help the encyclopedia, or does it
do more harm than good? In this talk, I’ll present findings from a
large-scale, quasi-experimental study using over a decade of English
Wikipedia data. We focus on situations where editors requested page
protection and compare the outcomes for articles that were protected versus
similar ones that weren’t. Our results show that page protection has mixed
effects: it tends to benefit high-quality articles by preventing decline,
but it can hinder improvement in lower-quality ones. These insights reveal
how protection shapes Wikipedia content and help inform when it’s most
appropriate to restrict editing, and when it might be better to leave the
page open.
Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms
By
*Joshua Ashkinaze (University of Michigan)*Large language models (LLMs) are
trained on broad corpora and then used in communities with specialized
norms. Is providing LLMs with community rules enough for models to follow
these norms? We evaluate LLMs' capacity to detect (Task 1) and correct
(Task 2) biased Wikipedia edits according to Wikipedia's Neutral Point of
View (NPOV) policy. LLMs struggled with bias detection, achieving only 64%
accuracy on a balanced dataset. Models exhibited contrasting biases (some
under- and others over-predicted bias), suggesting distinct priors about
neutrality. LLMs performed better at generation, removing 79% of words
removed by Wikipedia editors. However, LLMs made additional changes beyond
Wikipedia editors' simpler neutralizations, resulting in high-recall but
low-precision editing. Interestingly, crowdworkers rated AI rewrites as
more neutral (70%) and fluent (61%) than Wikipedia-editor rewrites.
Qualitative analysis found LLMs sometimes applied NPOV more comprehensively
than Wikipedia editors but often made extraneous non-NPOV-related changes
(such as grammar). LLMs may apply rules in ways that resonate with the
public but diverge from community experts. While potentially effective for
generation, LLMs may reduce editor agency and increase moderation workload
(e.g., verifying additions). Even when rules are easy to articulate, having
LLMs apply them like community members may still be difficult.
Best,
Kinneret
--
Kinneret Gordon
Lead Research Community Officer
Wikimedia Foundation <https://wikimediafoundation.org/>
*Learn more about Wikimedia Research <https://research.wikimedia.org/>*