Reminder - please consider taking our survey and share with others!
Call for Survey Participants - Wikimedia-Engaged Academic Researchers and Scientists
Survey Link: https://iup.co1.qualtrics.com/jfe/form/SV_0wfRHBdZmtbEuW2
Despite the growing interest in open educational resources in higher education, relatively few academics and scientists have significantly committed to sharing their research expertise in open knowledge projects. In fact, they often face opportunity costs when engaging with platforms like Wikipedia, as the time spent contributing can detract from more traditional scholarly outputs, such as peer-reviewed publications, conferences, and pursuing grants. This research project invites Wikimedia-engaged academics, scientists, and researchers to help us better understand how to make Wikimedia contributions “count” for academic researchers.
To that end, we'll be surveying and interviewing academic researchers and scientists who have previously engaged Wikimedia projects in an effort to discover the most suitable metrics and models for our project, as well as to develop a broad network of like-minded individuals interested in discovering how Wikimedia provides a platform for a broad open science infrastructure.
Taking the survey will take approximately 10 minutes of your time. If you agree to participate in an interview as part of the survey, this will be conducted via Zoom and take approximately 30-45 minutes of your time.
Brett Buttliere
University of Warsaw
Warsaw, Poland
b.buttliere(a)uw.edu.pl
Matthew Vetter
Indiana University of Pennsylvania
Indiana, PA, U.S.
mvetter(a)iup.edu
THIS PROJECT HAS BEEN APPROVED BY THE INDIANA UNIVERSITY OF PENNSYLVANIA INSTITUTIONAL REVIEW BOARD FOR THE PROTECTION OF HUMAN SUBJECTS (PHONE 724.357.7730).
Survey Link: https://iup.co1.qualtrics.com/jfe/form/SV_0wfRHBdZmtbEuW2
Matt Vetter, PhD (he/him)
Professor of English
Dept. of Language, Literature, and Writing
Indiana University of Pennsylvania
http://mattvetter.net<http://mattvetter.net/>
Connect with me on Zoom,
https://iupvideo.zoom.us/my/dr.vetterzooms
Managing co-editor, Writing Spaces<http://www.writingspaces.org/>
Co-chair, CCCC Wikipedia Initiative<https://cccc.ncte.org/cccc/wikipedia-initiative/>
Available as open access ebook, Wikipedia and the Representation of Reality<https://www.taylorfrancis.com/books/oa-mono/10.4324/9781003094081/wikipedia…..>
Dear Wikimedia Research Community,
Following requests and conversations during Wikimania, Iolanda Pensa and I
have partnered to update the* Research:People page on Meta *(
https://meta.wikimedia.org/wiki/Research:People) and make it more relevant
for everyone. We’ve refreshed the page to make it easier to navigate.
We hope this update enables the following:
- Maintain an up-to-date list of people who consider themselves part of
the Wikimedia Research Community.
- Gain a better understanding of who is in the community, their research
interests, and how to contact them.
- Provide a central resource where researchers can seek support or
consult with one another.
- Facilitate networking and foster collaborations and joint research
projects.
We warmly invite new community members to add their details to the
directory. If you're already listed, feel free to update your information
to reflect any recent changes.
As a bonus and in case you didn't know, the Research:Index page on Meta (
https://meta.wikimedia.org/wiki/Research:Index) gathers a lot of helpful
resources for our community, many of them maintained by the community
itself. I invite you to explore the Index and the listed resources and
consider updating or contributing to them for the benefit of the entire
research community. If you need any assistance with this, feel free to
reach out!
Thank you for helping us keep this resource up-to-date and valuable for
everyone in the community. And a big thank you to Iolanda for partnering
with me on this :)
Best regards,
Kinneret
--
Kinneret Gordon
Lead Research Community Officer
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi all,
The next Research Showcase will be live-streamed next Wednesday, September
18, at 9:30 AM PST / 16:30 UTC. Find your local time here
<https://zonestamp.toolforge.org/1726677000>. The theme for this showcase is
*Curation of Wikimedia AI Datasets*.
You are welcome to watch via the YouTube stream:
https://youtube.com/live/USzLGJ5LLC8?feature=share. As usual, you can join
the conversation in the YouTube chat as soon as the showcase goes live.
This month's presentations:
Supporting Community-Driven Data Curation for AI Evaluation on Wikipedia
through WikibenchBy *Tzu-Sheng Kuo, Carnegie Mellon University*AI tools are
increasingly deployed in community contexts. However, datasets used to
evaluate AI are typically created by developers and annotators outside a
given community, which can yield misleading conclusions about AI
performance. How might we empower communities to drive the intentional
design and curation of evaluation datasets for AI that impacts them? We
investigate this question on Wikipedia, an online community with multiple
AI-based content moderation tools deployed. We introduce Wikibench, a
system that enables communities to collaboratively curate AI evaluation
datasets, while navigating ambiguities and differences in perspective
through discussion. A field study on Wikipedia shows that datasets curated
using Wikibench can effectively capture community consensus, disagreement,
and uncertainty. Furthermore, study participants used Wikibench to shape
the overall data curation process, including refining label definitions,
determining data inclusion criteria, and authoring data statements. Based
on our findings, we propose future directions for systems that support
community-driven data curation.WikiContradict: A Benchmark for Evaluating
LLMs on Real-World Knowledge Conflicts from WikipediaBy *Yufang Hou, IBM
Research Europe - Ireland*Retrieval-augmented generation (RAG) has emerged
as a promising solution to mitigate the limitations of large language
models (LLMs), such as hallucinations and outdated information. However, it
remains unclear how LLMs handle knowledge conflicts arising from different
augmented retrieved passages, especially when these passages originate from
the same source and have equal trustworthiness. In this work, we conduct a
comprehensive evaluation of LLM-generated answers to questions that have
varying answers based on contradictory passages from Wikipedia, a dataset
widely regarded as a high-quality pre-training resource for most LLMs.
Specifically, we introduce WikiContradict, a benchmark consisting of 253
high-quality, human-annotated instances designed to assess LLM performance
when augmented with retrieved passages containing real-world knowledge
conflicts. We benchmark a diverse range of both closed and open-source LLMs
under different QA scenarios, including RAG with a single passage, and RAG
with 2 contradictory passages. Through rigorous human evaluations on a
subset of WikiContradict instances involving 5 LLMs and over 3,500
judgements, we shed light on the behaviour and limitations of these models.
For instance, when provided with two passages containing contradictory
facts, all models struggle to generate answers that accurately reflect the
conflicting nature of the context, especially for implicit conflicts
requiring reasoning. Since human evaluation is costly, we also introduce an
automated model that estimates LLM performance using a strong open-source
language model, achieving an F-score of 0.8. Using this automated metric,
we evaluate more than 1,500 answers from seven LLMs across all
WikiContradict instances. To facilitate future work, we release
WikiContradict on: this https URL <https://ibm.biz/wikicontradict>.
Best,Kinneret
--
Kinneret Gordon
Lead Research Community Officer
Wikimedia Foundation <https://wikimediafoundation.org/>
Dear All,
----resend because my work email is not in the list sorry for repost if
so---
The WikiSci COST Action
<https://docs.google.com/document/d/19nqb-5WDQYSpvBf1620BuWHQ0746fGdRK48Q2mG…>
application is again going well in our opinion. It is now in a 'first
final' version and ready for feedback in comments etc. Thank you,
sincerely, to everyone who read it so far and provided feedback.
Since the grant is due in about 1 month, it is a sort of final chance to
join with your own working group, so i encourage you to read it and suggest
something if you want. It is a 4 year commitment, but if you have a group
you think can be a track at a conference about wikimedia and science
communication, we will try to support you how we can.
Thus, we meet this friday, *September 20, 2024, at 4pm, Warsaw Time*, at
this Google Meet <https://meet.google.com/yum-qabj-szx>.
Hopefully, see you Friday at this link!
Brett Buttliere, Matt Vetter, Iolanda Pensa, Daniel Mietschen, et al.
University of Warsaw
*** apologies for cross-postings ***
The Max Planck Institute for Demographic Research (MPIDR) is seeking to appoint a full-time post-doctoral researcher to join the ERC-funded Research Group on Migration and Health Inequalities<http://www.demogr.mpg.de/en/research_6120/independent_research_groups_11668…>. The group, led by Silvia Loi<http://www.demogr.mpg.de/en/about_us_6113/staff_directory_1899/silvia_loi_2…>, brings together experts from Demography, Quantitative Sociology, and Social Epidemiology to address the pressing scientific and societal question: Why do immigrants age in poorer health compared to non-immigrants? We are seeking a creative, self-driven, collaborative scholar with a strong quantitative background that can contribute to advancing one or more of these three research areas:
1. quantify the gaps in healthy ageing trajectories between immigrants and non-immigrants by age, gender, socioeconomic status, and their interactions;
2. identify the critical events and circumstances in immigrants' lives that put them on a different healthy ageing trajectory from non-immigrants;
3. study the impact of family composition and family ties in mitigating health inequalities by migration background.
For more information please see https://www.demogr.mpg.de/en/career_6122/jobs_fellowships_1910/post_doctora…
--
This mail has been sent through the MPI for Demographic Research. Should you receive a mail that is apparently from a MPI user without this text displayed, then the address has most likely been faked. If you are uncertain about the validity of this message, please check the mail header or ask your system administrator for assistance.
Dear All,
Small reminder that we will meet today here
<https://meet.google.com/yum-qabj-szx> at 4pm Warsaw/Berlin time to
present/ finalize working groups.
Also if you have any recommendations of people, especially from the Target
Inclusion Countries that we could support (page 2
<https://docs.google.com/document/d/19nqb-5WDQYSpvBf1620BuWHQ0746fGdRK48Q2mG…>),
would be great. I need an email to invite and they have to just write a few
sentences how the action could help them and vice versa.
Thank you,
Brett, Matt, Iolanda, et al.,
On Mon, Aug 26, 2024 at 4:55 PM Brett Buttliere <b.buttliere(a)uw.edu.pl>
wrote:
> Dear All,
>
> The COST Action application
> <https://docs.google.com/document/d/19nqb-5WDQYSpvBf1620BuWHQ0746fGdRK48Q2mG…>
> is moving along well in our opinion, thank you for all of your involvements
> and happy to have met so many of you at Wikimania.
>
> The grant is due October 26th, and given that it is about 2 months away,
> we want to get feedback on the grant as it is, and to talk about how you
> can join and support the Action, also in terms of creating the ECOST
> profile so you can participate formally.
>
> To do this, we will meet this Friday, August 30th, at 4pm, Warsaw Time,
> at this Google Meet <https://meet.google.com/yum-qabj-szx>.
>
> At this stage we are formulating working groups (~p.20 here
> <https://docs.google.com/document/d/19nqb-5WDQYSpvBf1620BuWHQ0746fGdRK48Q2mG…>)
> and trying to identify the working group leaders. If you think you would
> like to be a working group leader or if you have a suggestion for a working
> group (Especially if you are in a target country/ early career/ female)
> please come or at least send an email with some paragraphs of the general
> idea/ how you might like to be involved. There are some examples of working
> group descriptions on page 20-22.
>
> The working groups are defined right now approximately as:
>
> -
>
> Measuring and Demonstrating the Impact of Wikimedia for Open
> Knowledge.
> -
>
> Identifying and sharing best practices for research using the
> wikimedia ecosystem.
> -
>
> Development of materials to make it easier to contribute to Wikimedia.
> -
>
> Training trainers and educators in using the Wikimedia ecosystem in
> the classroom.
> -
>
> Establishing TOPS style guidelines for organizations to implement and
> sign onto.
> -
>
> Integrating the Wikimedia ecosystem with EOSC, presenting solutions
> e.g., WikiData.
> -
>
> Building disciplinary organizations to contribute/ review science
> content in their area.
> -
>
> Arming Universities and GLAMs with tools to make their content
> available i.e., open.
>
>
> The 4 year plan is to year 1, develop materials at Wikimania Paris, Year 2
> go together to an EOSC/ EU conference to present materials and bring them
> on board, Year 3, go in groups to present at disciplinary conferences, and
> Year 4 come back all together with a big conference in one of the
> inclusiveness nations.
>
> *We are looking for *group leaders, participants, as well as people who
> might be interested in the more leadership roles e.g., Grant Awarding
> Coordinator, Science Communication Coordinator, and general members of the
> Action Management Committee. Especially if you know some administration who
> has handled such a project before and could consult with us, it would be
> great.
>
> Hopefully, see you Friday at this link
> <https://meet.google.com/yum-qabj-szx>!
>
> Brett Buttliere, Matt Vetter, Iolanda Pensa, et al.
>
> University of Warsaw
>
>