Hi Jonathan,
I spent some time to carefully read the FAQ about research and the
guidelines linked from there.
As for the progress of the project, I will provide regular updates on Meta.
Best,
Kiril
On Wed, Feb 26, 2020 at 12:56 AM Jonathan Morgan <jmorgan(a)wikimedia.org>
wrote:
Hi Kiril!
Thanks for all the patient and thoughtful clarifications and elaborations
:) I left a couple of comments inline, below.
Good luck with your project! As you can tell, we are a curious and
thoughtful group here on wikiresearch-l. If you have methodological
questions in the future, please don't hesitate to ask them here.
On Tue, Feb 25, 2020 at 11:43 AM Kiril Simeonovski <
kiril.simeonovski(a)gmail.com> wrote:
Hi Jonathan,
You have correctly deduced from the description that there will not be
any
interaction with editors and all the data for the
research will be drawn
from the publicly available edit histories. The most confusing part that
gives impression of intervening is perhaps the "experiment", which
conceptually differs in the social sciences from its more common meaning
in
a laboratory environment. That said, this
research is not going to
consume
editor's time for surveys nor it is going to
convert Wikipedia to a
laboratory or ask people to change their behaviour.
I came here with the announcement after creating the proposal on Meta and
following the guidelines regarding research projects with the goal of
getting some useful input from other researchers subscribed to the
mailing
list and learning how to administratively proceed
with the proposal on
Meta
(What should be done next on Meta? Will there be
an appointed WMF
researcher to contact regarding this research?).
Putting your research proposal on Meta is best practice for all research
projects related to Wikimedia. It is not a required step, but it's useful
for increasing awareness of your project among the broader Wikimedia
communities (researchers and everyone else).
There is nothing else you need to do at this point, although we appreciate
it if you would keep your project page up to date as you perform your
research. When you're done, we always appreciate it if you link to any
preprints, demos, code repos, slide decks, etc from that page as well.
Your project won't automatically be assigned a WMF contact. The Wikimedia
Foundation itself does not officially monitor or screen new research
projects that are published on Meta, or review them for support. However,
if you believe your research furthers the goals of the Wikimedia Movement,
you might consider applying for a grant (example
<
https://meta.wikimedia.org/wiki/Grants:Project/Misinformation_And_Its_Disco…
).
Individual WMF teams (including my team, Research) do occasionally partner
with external researchers
<https://research.wikimedia.org/collaborators.html> and those partnerships
can include access to non-public data (under a Non-Disclosure Agreement).
Some partnerships do involve funding, but this is not common. All
partnerships are at the discretion of the team manager. In the case of my
team, that manager is Leila Zia, Head of Research.
Quick question: when you say "the guidelines regarding research projects"
above, what document are you referring to? There are a lot of these pages
in the Research namespace and they are not always up to date,
unfortunately.
My request for help from research community
regarding this research will
be
mostly technical (e.g. smart random sampling of
editors, existing tools
for
research purposes, etc.)
We can definitely help you with these questions! You can also post
questions related to data access and data infrastructure to the analytics
mailing list <https://lists.wikimedia.org/mailman/listinfo/analytics>.
Best,
Kiril
On Tue 25. Feb 2020 at 17:06, Jonathan Morgan <jmorgan(a)wikimedia.org>
wrote:
Taking a quick step back from all the very
enthusiastic questioning of
the
researcher's motives...
Kiril,
Regarding your methods, Your proposal states
<
> >
> that for this study "The editors will be sampled from the pool of
> contributors to all language editions over Wikipedia's entire history
and
> will be classified into groups based on
their longevity on the
project."
But it
says little more than that.
When I read this description, it does not sound to me like you will
necessarily be contacting editors for this study, or intervening in any
way
> into Wikipedia. Stuart and Pine's questions seem to assume that you
will
be
in some way recruiting editors as participants,
asking them to change
their
behavior, asking them questions, etc.
*Will you be performing any of the above activities?* If not, the
questions
> asked so far may be beside the point. Anyone is free to perform
analysis
on
publicly available and free-licensed data.
If you do plan to intervene in Wikipedia in some way, or work with
editors
as research participants or co-researchers, and
you would like the
members
of this mailing list to provide you with feedback
or other support,
please
describe the support or feedback you would like
to receive in more
detail.
>
> If your study is non-interventionist but you still want feedback, we
can
provide
that too. Perhaps you can be more clear about the kind of
feedback
you want; that will keep the conversation going
in an interesting and
productive direction that everyone on the list can benefit from.
Finally, we the members of this list (whether volunteers or WMF staff)
are
> not peer reviewers, do not speak for the Wikipedia community, and are
not
> empowered to approve or deny research
requests.
>
> Best,
> Jonathan
>
> On Mon, Feb 24, 2020 at 11:23 PM Kiril Simeonovski <
> kiril.simeonovski(a)gmail.com> wrote:
>
> > Hi Stuart,
> >
> > Thank you for your thoughts so far. I really like how the discussion
is
progressing.
The methodology will, of course, yield other results about editor
dynamics
> and growth paths. Paid editing and sock puppetry as systemic risk
factors
> > could be included in the model exogenously but it might be possible
to
>
endogenise them in any future research. At this stage, the most
important
> > thing is to lay the grounds for developing a sensible model that can
be
> > later upgraded with new assumptions.
> >
> > As for the editing experience, I've been around since 2008 (this is
my
edit
log <
https://meta.wikimedia.org/wiki/Special:CentralAuth/Kiril_Simeonovski
> >
> ).
>
> Best,
> Kiril
>
> On Tue, Feb 25, 2020 at 12:37 AM Stuart A. Yeates <syeates(a)gmail.com
> > wrote:
> >
> > > Hi Kiril
> > >
> > > Let's just say that history has taught us to be risk-averse to
> > > drive-by researchers.
> > >
> > > Can you point us to other research output using this methodology?
Do
> >
you (or any of your team) have significant editing experience? Are
you
> > > familiar with the firestorm that is paid editing and sock
puppetry??
> > >
> > > cheers
> > > stuart
> > >
> > > --
> > > ...let us be heard from red core to black sky
> > >
> > > On Tue, 25 Feb 2020 at 10:43, Kiril Simeonovski
> > > <kiril.simeonovski(a)gmail.com> wrote:
> > > >
> > > > Hi Pine,
> > > >
> > > > The findings from the research will be articulated to draw clear
> > > > conclusions about what causes utility and disutility from
> > participation,
> > > > and how this is perceived by different editors. For instance, it
is
> >
natural
> > > to assume that editors come to contribute by adding content that
will
> > > remain visible, while blocks and
reverted edits are risk factors
that
> > > drive
> > > > them away, although different editors have different levels of
risk
> >
> aversion. Similarly to any other research, the benefit for the
> community
> > > and individual editors is going to be indirect but yet not
> insignificant
> > to
> > > be accepted in the future process of decision-making (if the
research
> >
demonstrates the existence of high level of risk aversion towards
> > something, then it automatically signals that doing that thing is
harmful
> > for the environment).
> >
> > I know that it's impossible to predict the extent to which this
research
> > would make impact because the body of literature is very poor on
> > volunteer-driven environments in a dynamic setting but it's
definitely
> > > worth to start off something that might attract the attention of
> > > researchers in this direction. At the end, the research is not
meant
to
> > > carve rules in stone that any single editor should respect but
rather
> to
> > > suggest something that individuals and communities might find
useful
> (the
> > > means of doing this will definitely not turn Wikipedia into a
> laboratory
> > or
> > > put someone's privacy in danger).
> > >
> > > Best,
> > > Kiril
> > >
> > > On Mon, Feb 24, 2020 at 9:43 PM Pine W <wiki.pine(a)gmail.com>
wrote:
> > > >
> > > > > Hi Kiril,
> > > > >
> > > > > Thank you for sharing your proposal.
> > > > >
> > > > > I am concerned about the possibility of Wikipedia being used
as a
> > > laboratory for experiments that consume
volunteers' time and/or
> > > personal data, and don't benefit Wikipedia or its participants.
Does
> > > > your research benefit the community, and if so, how? It sounds
like
> > > > > your research intends to develop a model of decision trees for
> > > > > individual Wikipedians, and at first read I don't understand
how
the
> > > > individual research subjects or the community would benefit.
> > > >
> > > > Sorry if this sounds defensive, but I hope that you understand
why
I'm
> > > asking.
> > >
> > > Pine
> > > (
https://meta.wikimedia.org/wiki/User:Pine )
> > >
> > > On Mon, Feb 24, 2020 at 6:00 PM Kiril Simeonovski
> > > <kiril.simeonovski(a)gmail.com> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I am currently working on a research concerned with modelling
user
> > > > > behaviour on Wikipedia. The idea is to design a field
experiment
> > over a
> > > > > random sample of Wikipedians in order to examine their risk
> > preferences
> > > > and
> > > > > define (dis)utilities that will be used in a
utility-maximisation
> > > model.
> > > > > >
> > > > > > I have already submitted an abstract that got accepted for
the
> > > > biennial Foundations
> > > > of Utility and Risk Conference 2020 <
https://www.furconference.org/>
> > > and my
> > > > future plans include presentation of the concept at other
research
> > > > conferences (including
Wikimania 2020).
> > > >
> > > > You can visit the project page
> > > > <
> > >
>
https://meta.wikimedia.org/wiki/Research:Modelling_Behaviour_in_a_Peer_Prod…
> >
> > >
> > > > > of this research on Meta. Your questions and comments are
welcome
at
> any
> > > > time. Thank you!
> > > >
> > > > Best regards,
> > > > Kiril
> > > > _______________________________________________
> > > > Wiki-research-l mailing list
> > > > Wiki-research-l(a)lists.wikimedia.org
> > > >
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
> > >
> > > _______________________________________________
> > > Wiki-research-l mailing list
> > > Wiki-research-l(a)lists.wikimedia.org
> > >
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
> > >
> > _______________________________________________
> > Wiki-research-l mailing list
> > Wiki-research-l(a)lists.wikimedia.org
> >
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
Jonathan T. Morgan
Senior Design Researcher
Wikimedia Foundation
User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)
(Uses He/Him)
*Please note that I do not expect a response from you on evenings or
weekends*
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
Jonathan T. Morgan
Senior Design Researcher
Wikimedia Foundation
User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
(Uses He/Him)
*Please note that I do not expect a response from you on evenings or
weekends*
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l