Pursuant to prior discussions about the need for a research
policy on Wikipedia, WikiProject Research is drafting a
policy regarding the recruitment of Wikipedia users to
participate in studies.
At this time, we have a proposed policy, and an accompanying
group that would facilitate recruitment of subjects in much
the same way that the Bot Approvals Group approves bots.
The policy proposal can be found at:
http://en.wikipedia.org/wiki/Wikipedia:Research
The Subject Recruitment Approvals Group mentioned in the proposal
is being described at:
http://en.wikipedia.org/wiki/Wikipedia:Subject_Recruitment_Approvals_Group
Before we move forward with seeking approval from the Wikipedia
community, we would like additional input about the proposal,
and would welcome additional help improving it.
Also, please consider participating in WikiProject Research at:
http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Research
--
Bryan Song
GroupLens Research
University of Minnesota
Hi!
I am doing a PhD on online civic participation project
(e-participation). Within my research, I have carried out a user
survey, where I asked how many people ever edited/created a page on a
Wiki. Now I would like to compare the results with the overall rate of
wiki editing/creation on country level.
I've found some country-level statistics on Wikipedia Statistics (e.g.
3,000 editors of Wikipedia articles in Italy) but data for UK and
France are not available since Wikipedia provides statistics by
languages, not by countries. I'm thus looking for statistics on UK and
France (but am also interested in alternative ways of measuring wiki
editing/creation in Sweden and Italy).
I would be grateful for any tips!
Sunny regards, Alina
--
Alina ÖSTLING
PhD Candidate
European University Institute
www.eui.eu
This is more on the experimental side of "research" but I just
finished a prototype realtime visualization of tweets that reference
Wikipedia:
http://wikitweets.herokuapp.com/
wikitweets is a NodeJS [1] application that listens to the Twitter
Streaming API [2] for tweets that contain Wikipedia URLs, and then
looks up the relevant Wikipedia article using the API to ultimately
stream the information to the browser using SocketIO [3]. The most
amazing thing for me is seeing the application run comfortably (so
far) as a single process on Heroku with no attached database needed.
If you are curious the code is on GitHub [4].
The key to wikistream working at all is that Twitter allows you to
search and filter the stream using the original (unshorted) URL. So
for example a Tweet with the text:
Question of the Day: What’s the greatest seafaring movie ever?
Some suggestions: http://bit.ly/IqsE1e (But anything on water'll work)
#QOD [5]
Is discoverable with a search query like:
Question of the Day wikipedia.org [6]
Note "wikipedia.org" doesn't exist in the text of the original tweet
at all, since it has been shortened by bit.ly -- but it is still
searchable because Twitter appear to be unshortening and indexing
URLs. Anyhow, I thought I'd share here since this also relied heavily
on the various language Wikipedia APIs.
//Ed
[1] http://nodejs.org
[2] https://dev.twitter.com/docs/streaming-api/methods
[3] http://socket.io
[4] https://github.com/edsu/wikitweets
[5] https://twitter.com/#!/EWeitzman/status/195520487357558784
[6] https://twitter.com/#!/search/realtime/Question%20of%20the%20Day%20wikipedi…
-------- Message original --------
Sujet: Re: [Wiki-research-l] Experimental study of informal rewards in peer production
Date : Thu, 26 Apr 2012 15:50:44 -0400
De : Michael Restivo <mike.restivo(a)gmail.com>
Pour : Chitu Okoli <Chitu.Okoli(a)concordia.ca>, Research into Wikimedia content and communities <wiki-research-l(a)lists.wikimedia.org>
Hi Chitu,
Yes, your conjecture is spot-on. Here is a more detailed response that I sent to Joseph. I tried sending this to the wiki-research-l but the email keeps bouncing back to me. If you're interested and willing to share it with the list, that would be acceptable to me.
We thought about this question quite extensively and there are a few reasons why we sampled the top 1% (which we didn't get around to discussing in this brief paper). First, because of the high degree of contribution inequality in Wikipedia's editing community, we were primarily interested in how status rewards affect the all-important core of highly-active editors. There is also a lot of turn-over in the long tail of the distribution, and even among the most active editors, there is considerable heterogeneity. Focusing on the most active users ensured us sufficient statistical power. (Post-hoc power analysis suggests that our sample size would need to be several thousand users in the 80-90th percentiles, and several hundred in the 90-99th percentiles, to discern an effect of the same strength.) Also, we considered the question of construct validity: which users are deserving (so to speak) of receiving an editing award or social recognition of their work?
You are right that it should be fairly easy to extend this analysis beyond just the top 1%, but just how wide a net to cast remains a question. The issue of power calculation and sample size becomes increasingly difficult to manage for lower deciles because of the power-law distribution. And I don't think it would be very meaningful to assess the effect of barnstars on the bottom half of the distribution, for example, for the substantive reasons I mentioned above. Still, I'd be curious to hear what you think, and whether there might be some variations on this experiment that could overcome these limitations.
In terms of data dredging, that is always a concern and I completely understand where you are coming from. In fact, as both and author and consumer of scientific knowledge, I'm rarely ever completely satisfied. For example, a related concern that I have is the filing cabinet effect - when research produces null (or opposite) results and hence the authors decide to not attempt to have it published.
In this case, I actually started this project with the hunch that barnstars would lead to a slight decline in editing behavior; my rationale was that rewards would act as social markers that editors' past work was sufficient to earn social recognition and hence receiving such a reward would signal that the editor had "done enough" for the time being. In addition to there being substantial support for this idea in the economics literature, this intuition stemmed from hearing about an (unpublished) observational study of barnstars by Gueorgi Kossinets (formerly at Cornell, now at Google) that suggested editors receive barnstars at the peak of their editing activity. Of course, we chose an experimental design precisely to help us to tease out the causal direction as well as what effect barnstars have for recipients relative to their unrewarded counterparts. We felt like no matter what we found - either a positive, negative, or even no effect - it would have been interesting
enough to publish, so hopefully that alleviates some of your concerns.
Please let me know if you have any other questions, and I'd love to hear your thoughts about potential follow-ups to this research.
Regards,
Michael
On Thu, Apr 26, 2012 at 3:30 PM, Chitu Okoli <Chitu.Okoli(a)concordia.ca <mailto:Chitu.Okoli@concordia.ca>> wrote:
One obvious issue is that it would be unethical to award barnstars to contributors who did not deserve them. However, the 1% most productive contributors, by definition, deserved the barnstars that the experimenter awarded them. Awarding barnstars to undeserving contributors for experimental purposes probably would not have flown so easily by the ethical review board. As the article notes:
----------
This study's research protocol was approved by the Committees on Research Involving Human Subjects (IRB) at the State University of New York at Stony Brook (CORIHS #2011-1394). Because the experiment presented only minimal risks to subjects, the IRB committee determined that obtaining prior informed consent from participants was not required.
----------
This is my conjecture; I'd like to hear the author's comments.
~ Chitu
-------- Message original --------
Sujet: [Wiki-research-l] Experimental study of informal rewards in peer production
De : Joseph Reagle <joseph.2011(a)reagle.org <mailto:joseph.2011@reagle.org>>
Pour : michael.restivo(a)stonybrook.edu <mailto:michael.restivo@stonybrook.edu>
Copie à : Research into Wikimedia content and communities <wiki-research-l(a)lists.wikimedia.org <mailto:wiki-research-l@lists.wikimedia.org>>
Date : 26 Avril 2012 11:42:01
In this [study](http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.003…:
> Abstract: We test the effects of informal rewards in online peer production. Using a randomized, experimental design, we assigned editing awards or “barnstars” to a subset of the 1% most productive Wikipedia contributors. Comparison with the control group shows that receiving a barnstar increases productivity by 60% and makes contributors six times more likely to receive additional barnstars from other community members, revealing that informal rewards significantly impact individual effort.
I wonder why it is limited to the top 1%? I'd love to see the analysis repeated (should be trivial) on each decile. Besides satisfying my curiosity, some rationale and/or discussion of other deciles would also address any methodological concern about data dredging.
--
Michael Restivo
Department of Sociology
Social and Behavioral Sciences S-433
Stony Brook University
Stony Brook, NY 11794
mike.restivo(a)gmail.com <mailto:mike.restivo@gmail.com>
Dan,
There are a couple....
Have you tried our Gadget ProveIt?
Go to my preferences/gadgets and check the box to try it out.
It'll show up when you edit.
-- Amy
On Apr 29, 2012, at 5:14 PM, Dan Bolser wrote:
> On this topic, will Wikipedia ever implement a 'citation tool'? I'm
> thinking something like Mendeley or EndNote for wiki text? It would be
> great to have citations more formally implemented within MW (via an
> extension).
>
>
> Cheers,
> Dan.
>
> On 20 April 2012 18:31, phoebe ayers <phoebe.wiki(a)gmail.com> wrote:
>> Hi all,
>>
>> Has there been any research done into: the number of citations (e.g.
>> to books, journal articles, online sources, everything together) on
>> Wikipedia (any language, or all)? The distribution of citations over
>> different kinds or qualities of articles? # of uses of citation
>> templates? Anything like this?
>>
>> I realize this is hard to count, averages are meaningless in this
>> context, and any number will no doubt be imprecise! But anything would
>> be helpful. I have vague memories of seeing some citation studies like
>> this but don't remember the details.
>>
>> Thanks,
>> -- phoebe
>>
>> --
>> * I use this address for lists; send personal messages to phoebe.ayers
>> <at> gmail.com *
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> Wiki-research-l(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Hi all,
Has there been any research done into: the number of citations (e.g.
to books, journal articles, online sources, everything together) on
Wikipedia (any language, or all)? The distribution of citations over
different kinds or qualities of articles? # of uses of citation
templates? Anything like this?
I realize this is hard to count, averages are meaningless in this
context, and any number will no doubt be imprecise! But anything would
be helpful. I have vague memories of seeing some citation studies like
this but don't remember the details.
Thanks,
-- phoebe
--
* I use this address for lists; send personal messages to phoebe.ayers
<at> gmail.com *
I'd like to call your attention to a paper that I hope may be of interest and use to members of this research community, entitled "Validity issues in the use of social network analysis with digital trace data", by two of my students, James Howison and Andrea Wiggins, and me. The paper appeared in the Journal of the Association for Information Systems, a journal that I expect few of you regularly follow, so I thought an email might help bridge the gap.
In the paper we argue that data obtained from online systems, e.g., mailing lists, wiki and the like, are different in important ways from the typical data of SNA, but that the implications of these differences for the validity of research are not always fully appreciated. For example, in contrast to a cross-sectional survey, data from systems often record events that happened over time that have to be collapsed to create a network. However, doing so may create apparent connections in the network that don't occur in the data. We provide a set of suggestions for using such data in SNA studies (and for reviewing papers that use such data). The abstract is included below.
The official URL for the paper is http://aisel.aisnet.org/jais/vol12/iss12/2/. If you're not a subscriber, I'd be happy to mail you a copy or you can find a preprint at http://crowston.syr.edu/content/validity-issues-use-social-network-analysis…. Comments on the paper are always welcomed.
Validity issues in the use of social network analysis with digital trace data. JAIS 12(12) paper 2.
There is an exciting natural match between social network analysis methods and the growth of data sources produced by social interactions via information technologies, from online communities to corporate information systems. Information Systems researchers have not been slow to embrace this combination of method and data. Such systems increasingly provide "digital trace data" that provide new research opportunities. Yet digital trace data are substantively different from the survey and interview data for which network analysis measures and interpretations were originally developed. This paper examines ten validity issues associated with the combination of data digital trace data and social network analysis methods, with examples from the IS literature, to provide recommendations for improving the validity of research using this combination.
Kevin Crowston
Syracuse University Phone: +1 (315) 443-1676
School of Information Studies Fax: +1 (815) 550-2155
348 Hinds Hall Web: http://crowston.syr.edu/
Syracuse, NY 13244-4100 USA
Kevin Crowston
Syracuse University Phone: +1 (315) 443-1676
School of Information Studies Fax: +1 (815) 550-2155
348 Hinds Hall Web: http://crowston.syr.edu/
Syracuse, NY 13244-4100 USA
Dear Wikipedia contributors,
We did it! For the past six weeks, you have all graciously tolerated my
emails requesting participants for my undergraduate senior thesis
project<http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikipe…>on
users’ motivations to contribute to Wikipedia. On Monday, I reached
out
to this community, begging for just eighteen more participants to reach my
target sample size of 100 respondents. We not only reached that goal, but
exceeded it: 161 Wikipedia contributors responded to my questionnaire!
Your insightful responses are invaluable to my project and I cannot express
my gratitude for this community enough, so thank you, thank you, thank you!
When my final paper is written in June, I will make it available to the
Wikipedia community.
Thank you once again for your insight, thoughtful questions, and feedback.
Best,
Audrey