The Wikimedia Research Committee [1] is currently considering a major overhaul of the research section on Meta-Wiki:
http://meta.wikimedia.org/wiki/Research/2011_overhaul
The practical reason to start this process is to clean up and streamline pages used by the Wikimedia community and by the Foundation to document internal research projects and policies. The ambitious goal it to make Meta:Research the main hub where all research on Wikimedia projects (be it internal or external) is discussed, reviewed and tracked. The objectives we are hoping to achieve in the short term with this project are the following:
make it easy for researchers to find the resources and WMF support they are looking for
bring as much transparency as possible to research involving the Wikimedia community, by reducing attrition between the community and researchers and making sure research is not disruptive of editor activity
design a scheme of incentives to increase researcher participation and to increase the number of projects included in the Wikimedia research directory
design a series of incentives to nudge researchers towards releasing their datasets under an open license and publishing/self-archiving their research results via open access outlets/repositories.
Our long-term vision aims to:
provide support to the publication of research data on Wikimedia projects via a unified open data infrastructure [2]
integrate structured bibliographic data into Meta:Research via whatever solution the community decides to adopt [3]
Many on this list are already actively involved in editing and maintaining Meta research pages. Your feedback and suggestions on this project would be very valuable.
Dario
[1] http://meta.wikimedia.org/wiki/Research_Committee
[2] We are currently reviewing a number of solutions to set up a central repository of open research data: http://bit.ly/OpenDataPlatforms
[3] See the long discussion started on this list with this thread: http://lists.wikimedia.org/pipermail/wiki-research-l/2011-March/001361.html
--
Dario Taraborelli, PhD
Senior Research Analyst
Wikimedia Foundation
http://wikimediafoundation.orghttp://nitens.org/taraborelli
Hey guys!
There is another project looking to perform subject recruitment that needs
our assistance.
It looks like the researchers are looking to contact new
article creators with different message styles to detect a causal
relationship in changes to editor behavior.
http://meta.wikimedia.org/wiki/Research:Effects_of_Feedback_on_Participatio…<http://meta.wikimedia.org/wiki/Research:Effects_of_Feedback_on_Participatio…>
Wikipedia<http://meta.wikimedia.org/wiki/Research:Effects_of_Feedback_on_Participatio…>
I'll be moving my comments and questions to the project talk page, so please
continue any discussion there.
-Aaron
---------- Forwarded message ----------
From: Haiyi Zhu <haiyiz(a)cs.cmu.edu>
Date: Fri, Jul 29, 2011 at 4:47 PM
Subject: Re: Conducting Experiments on Effects of Messages in Wikipedia
To: Aaron Halfaker <aaron.halfaker(a)gmail.com>
Cc: Robert Kraut <robert.kraut(a)cmu.edu>, nkittur <nkittur(a)cs.cmu.edu>, Amy X
Zhang <amyz(a)andrew.cmu.edu>, Jenny He <jipingh(a)andrew.cmu.edu>
Hi Aaron
We have created a project page for the experiment we would like to do.
http://meta.wikimedia.org/wiki/Research:Effects_of_Feedback_on_Participatio…
We are ready for review. Would you please help us get the ball rolling?
Thanks a lot!!
Haiyi
On Wed, Jul 6, 2011 at 8:23 AM, Aaron Halfaker <aaron.halfaker(a)gmail.com>wrote:
> Haiyi,
>
> This study sounds like it would be relevant if submitted to the Wikimedia
> Research Committee(http://meta.wikimedia.org/wiki/Research:Committee) for
> review as a Subject Recruitment request(
> http://meta.wikimedia.org/wiki/Research:Subject_recruitment). I think
> this should be an excellent venue for feedback, and if things go well, a
> go-ahead/endorsement from the committee. You'll have to create a project
> page (see the form on
> http://meta.wikimedia.org/wiki/Research:Subject_recruitment) and then let
> me know when you are ready for review and I can get the ball rolling.
>
> The review is relatively unstructured and consists primarily of questions
> and requests raised on the talk page.
>
> Let me know if you have questions.
>
> -Aaron
>
> On Wed, Jul 6, 2011 at 12:13 AM, Haiyi Zhu <haiyiz(a)cs.cmu.edu> wrote:
>
>> Hi Aaron,
>>
>> This is Haiyi Zhu from Carnegie Mellon University. I am working with Bob
>> Kraut and Niki Kittur. I read your work about the effects of revert in
>> Wikipedia. That is very interesting. Also congratulations on getting the
>> summer intern job in Wikipedia.
>>
>> We have been planning on conducting an experiment to investigate the
>> effects of messages on participation in Wikipedia. We are wondering if there
>> is any appropriate place in Wikipedia website for us to post our plan of
>> experiments and get feedback from Wikipedia foundations and experienced
>> Wikipedians. It would be really helpful if you could point us to the right
>> place.
>>
>> I also attach the details of our experiments.
>>
>> Thanks for your help!
>> Haiyi
>>
>>
>> 1. Purpose of the study
>>
>> Wikipedia is one of the most successful examples of social production
>> online, and the key components to its continued success is maintaining a
>> large number of contributions made by a large and diverse community of
>> editors. The quality of Wikipedia articles is also maintained by its
>> community of editors through shared leadership behaviors, which are
>> leadership behaviors that come from all members at all levels (e.g. all
>> Wikipedia users), not only members in a defined leadership position (e.g.
>> Wikipedia administrators). Shared leadership behaviors in Wikipedia include,
>> but are not limited to, giving positive comments to a contributor for
>> creating a well-written new article (positive task-based message),
>> criticisms or reprimands to a contributor for not complying to Wikipedia
>> guidelines (negative task-based message), giving directions to correct an
>> error (directive task-based message). All of these aforementioned messages
>> could be written in a manner that is sociable and person-focused--friendly,
>> supportive and caring, and seeks to retain relationships with others, or not
>> sociable. Past research has shown that person-focused leadership was
>> effective in motivating users to contribute more, while negative messages
>> lead to decrease in motivation (Effectiveness of Shared Leadership, CSCW,
>> 2012, under review,
>> http://www.cs.cmu.edu/~haiyiz/ttt/SharedLeadership_fifthpdf.pdf).
>> Therefore, we are interested in learning how task-based messages (positive,
>> negative, directive) with a sociable(person-focused) component would effect
>> Wikipedia users' contributions.
>>
>> We will test how receiving the different types of messages (positive
>> task-based, negative task-based, or directive message) with and without a
>> sociable component, as well as the recipient's level of experience will
>> influence a recipient's future contributions to Wikipedia. We will do so by
>> having researchers post messages on the recipients' Wikipedia User Talk
>> page, and then we will track the recipients' level of contribution over
>> time.
>>
>> 2. Research procedures:
>>
>> The research procedure will not require any extra effort and activity by
>> the participants, other than their usual participation on Wikipedia. The
>> time period of this experiment will take place from July 1st to August. The
>> location will take place online on the Wikipedia website.
>>
>> We will be posting experimentally altered messages to the participants'
>> Wikipedia User Talk page, which is public and can be seen by anyone. The
>> experiment researchers will identify a list of newly created articles that
>> are two to ten days old. We will only post a response to a newly-created
>> article that is at least two days old, in accordance to Wikipedia's policy
>> of "don't bite the newcomers" (
>> http://en.wikipedia.org/wiki/Wikipedia:BITE).
>>
>> We will then categorize the new articles as containing errors or not
>> containing errors according to a set of criteria determined by Wikipedia
>> guidelines. The creator of the new article containing errors would randomly
>> be assigned to receive a positive task-based message, negative task-based
>> message, directive message, or non task-based message, and each type of
>> message would also be randomly assigned to be sociable or not sociable. The
>> same assignment procedure will be given to the creator of an article without
>> errors, with the exclusion of negative task-based feedback.
>>
>> Also, all messages sent by the experimenter will be prepared beforehand in
>> templates (see the section 4) , which are designed to mimic messages sent by
>> other Wikipedia users, and therefore the risk associated with these messages
>> are no greater than the risks experienced by a Wikipedia user during
>> everyday interactions on Wikipedia. We will then extract the users' activity
>> data after one to two weeks in order to examine the users level of
>> contribution; we will also extract some user info, such as date of first
>> edit, in order to assess the recipients' level of experience. However, no
>> personal information that could be used to identify the identity of the
>> recipient will be extracted, used, or recorded in this experiment.
>>
>> 3. We need your help!
>> If you are interested in our research project and would like to
>> participate in our experiment and send out messages, please sign here.
>>
>> 4. Appendix: example messages
>>
>> 4.1 Each message has some base, non-task focused/non-social content.
>> I see you created the new article [[“{{{{contrib}}}}]]. New articles are
>> typically reviewed by users on the [[Wikipedia:New_pages_patrol|‘New Article
>> Patrol’]]. If your article has not been reviewed, it should happen soon.
>> Thank you, ~~~~
>> I see that you recently created the new article [[“{{{{contrib}}}}]]. As
>> you know, Wikipedia is always a [work in progress]. So thank you, ~~~~
>> I see that you recently created the new article [[contrib]]. It would
>> probably fit into project [[Project]]. (although a good suggestion, this
>> message is not yet compatible with the various categories,
>>
>> 4.2. The social messages have social content in the salutation before the
>> base message and in the closing after the base message:
>>
>> Salutations:
>>
>> Happy Monday, {{SUBST:BASEPAGENAME}}!
>> Greetings {{SUBST:BASEPAGENAME}}!
>> Hi {{SUBST:BASEPAGENAME}}!
>> Welcome {{SUBST:BASEPAGENAME}}! (new users only)
>>
>>
>> Closing:
>>
>> Happy editing! Hope your day is going well and you are having fun. ~~~
>> It’s nice to see you editing! Hope you’ll stick around! ~~~~
>> Have a fun and productive day! Cheers, ~~~~
>> Let me know if there is any way I can help. Hope you sill stick around.
>> Thanks, Amy ~~~~
>> Let me know if I can be of any help and I’ll do my best! ~~~~
>>
>>
>>
>> 4.3. The positive feedback message includes some general positive comment
>> and an article specific comment
>>
>> General: This article looks very well put together. [ Point to a specific
>> praiseworthy element , based on article content.]
>>
>> Specific:
>>
>> The image greatly improves the article.
>>
>> The citations and references seem to abide to Wikipedia’s referencing
>> guidelines.
>>
>> There is a good number of citations and references.
>> The content seems pretty substantial for a new article.
>> The content seems well-organized.
>> The information is presented clearly and is easy to understand.
>> The layout of the article makes it very clear.
>> The headings and sections seem appropriate.
>>
>> 4.4. The negative feedback message includes some general negative comment
>> and an article specific comment
>>
>> General: However, I noticed the article contains errors. [Point to a
>> specific error, based on article content.]
>>
>> Specific:
>>
>> The article contains spelling/grammatical error’s (specifics)
>> The article currently does not contain any [[WP:REF|references]].
>> The article’s image currently does not contain any captions.
>> The references in the article do not follow Wikipedia guidelines. There is
>> a tutorial on formatting citations at [[Wikipedia:Referencing]]
>> The article does not contain any [[Help:Link|Wikilinks]], and so doesn’t
>> follow [[Wikipedia:Manual of Style (linking)|Wikipedia style guidelines]].
>> The article does not contain in-line citations, and so doesn’t follow
>> [[WP:REF|Wikipedia style guidelines]].
>> The article is confusing. I do not understand what you are trying to
>> say.
>> The article’s footer sections are not in the [[WP:FOOTER|standard order]].
>>
>>
>> 4.5. The directive message asks the editor to do a specific comment on an
>> article related to the one they created.
>>
>> Other sample tasks:
>>
>> It would be great if you could also improve the related article __.
>> It would be great if you could also [[Help:Files|upload a picture]] for
>> the related article__.
>> It would be great if you could also [[WP:REF|add references]] to the
>> related article __.
>> It would be great if you could also create new articles for [the red links
>> that do not lead to another article yet].
>> It would be great if you could also [[Talk:__|give feedback]] for the
>> related article __.
>> It would be great if you could also clean-up the related article __.
>> It would be great if you could also [[WP:MOS|Wikify]] the related article
>> __.
>> It would be great if you could also help merge the related article __.
>> It would be great if you could also participate in the [discussion] about
>> deleting __.
>>
>>
>>
>>
>
Dear all,
I always thought that the role of the Rcom would be not only managing the
incoming requests for research (in a broad sense), but also identifying
what research results we would like to have in order to facilitate the
future development of the WMF projects.
Recently, AFT has been deployed on every article in en.wp, which caused
some controversy. Users are unsure on what the rankings actually mean (for
instance, if the readers actually answer the questions they are asked, or
instead give rankings based on their perception of the subject; whether it
would be a good idea to invite the readers also to provide text input, and
a number of other issues).
To me it looks like a good test case. It would be good for us to have some
research results on the issue, and so far (from what I know) nobody
volunteered to perform such research. Should WMF then indicate that they
want these results? Probably at this stage we are not prepared to order it
(I mean to pay for it), but it could be broadly advertised in certain
places.
Just to make sure, what I asked is not one but two separate questions:
1) Do we (badly) want the research on AFT?
Even if the answer is no (not needed, too early, unrealistic etc), it
makes sense in my opinion to discuss another question:
2) Is is appropriate and/or meaningful for us as Rcom to compile a
priority list of needed research topics and try to advertise the necessity
and urgency of research carried out on these topics?
Note that even if the answer to 1 is yes the answer to 2 can still be no
(as a matter of principle).
Cheers
Yaroslav
(* just posted on wiki-research-l *)
We are glad to announce the inaugural issue of the Wikimedia Research Newsletter [1], a new monthly survey of recent scholarly research about Wikimedia projects. This is a joint project of the Signpost [2] and the Wikimedia Research Committee [3] and follows the publication of two research updates in the Signpost, see also last month's announcement on this list [4].
The first issue (which is simultaneously posted as a section of the Signpost and as a stand-alone article in the Wikimedia Research Index) includes 5 "in depth" reviews of papers published over the last few months and a number of shorter notes for a total of 15 publications, covering both peer-reviewed research and results published in research blogs. It also includes a report from the Wikipedia research workshop at OKCon 2011 and highlights from the Wikimedia Summer of Research program.
The following is the TOC of issue #1:
• 1 Edit wars and conflict metrics
• 2 The anatomy of a Wikipedia talk page
• 3 Wikipedians as "Janitors of Knowledge"
• 4 Use of Wikipedia among law students: a survey
• 5 Miscellaneous
• 6 Wikipedia research at OKCon 2011
• 7 Wikimedia Summer of Research
• 7.1 How New English Wikipedians Ask for Help
• 7.2 Who Edits Trending Articles on the English Wikipedia
• 7.3 The Workload of New Page Patrollers & Vandalfighters
• 8 References
We are planning to make the newsletter easy to syndicate and subscribe to. If you wish your research to be featured, a CFP or event you organized to be highlighted, or just join the team of contributors, head over to this page to find out how: [5] We hope to make this newsletter a favorite reading for our research community and we look forward to your feedback and contributions.
Dario Taraborelli, Tilman Bayer (HaeB)
on behalf of the WRN contributors
[1] http://meta.wikimedia.org/wiki/Research:Newsletter/2011-07-25
[2] http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost
[3] http://meta.wikimedia.org/wiki/Research:Committee
[4] http://lists.wikimedia.org/pipermail/wiki-research-l/2011-June/001552.html
[5] http://meta.wikimedia.org/wiki/Research:Newsletter
--
Dario Taraborelli, PhD
Senior Research Analyst
Wikimedia Foundation
http://wikimediafoundation.orghttp://nitens.org/taraborelli
Hi!
I hope that you are all fine. Here a firts version reporting on the Workshop on "Wikipedia & Research: The innovative character of Wikipedia research and the new challenges (and opportunities) associated with it" at the Open Knowledge Conference: June 30th.
http://meta.wikimedia.org/wiki/Report_Workshop_on_Wikipedia_%26_Research-_O…
Please Daniel Mietchen (also participating in the workshop)if you like change or add anything you consider is missing, particularly in order to have a complete version to be used for the Wikimedia Research Newsletter.
Thanks. Cheers! Mayo
«·´`·.(*·.¸(`·.¸ ¸.·´)¸.·*).·´`·»
«·´¨*·¸¸« Mayo Fuster Morell ».¸.·*¨`·»
«·´`·.(¸.·´(¸.·* *·.¸)`·.¸).·´`·»
Research Digital Commons Governance: http://www.onlinecreation.info
Ph.D European University Institute
Postdoctoral Researcher. Institute of Govern and Public Policies. Autonomous University of Barcelona.
Visiting scholar. Internet Interdisciplinary Institute. Open University of Catalonia (UOC).
Visiting researcher (2008). School of information. University of California, Berkeley.
Member Research Committee. Wikimedia Foundation
http://www.onlinecreation.info
E-mail: mayo.fuster(a)eui.eu
Skype: mayoneti
Phone Spanish State: 0034-648877748
Just in case you missed this, a 24-year-old programmer at MIT is facing 35 years of prison for illegally downloading a large number of papers from JSTOR.
In a statement bordering on ridiculous, JSTOR confirmed that it had "secured the digital content" Mr. Swartz allegedly stole.
Hopefully this will give people a sense of how bad things are in 2011 regarding access to publicly funded research. I am really happy that we are putting open access and open data among our priorities at WMF and doing our bit to help fix this broken system.
Dario
----
Internet Activist Charged in M.I.T. Data Theft
http://bits.blogs.nytimes.com/2011/07/19/reddit-co-founder-charged-with-dat…
2:19 p.m. | Updated Added links to documents and comment from United
States Attorney’s office in Boston.
2:48 p.m. | Updated Changed headline and post to remove reference to
Mr. Swartz being a co-founder of Reddit, as this is disputed. Added
comment from Demand Progress.
4:11 p.m. | Updated Corrected reference to timing of indictment.
Aaron Swartz, a 24-year-old programmer and online political activist,
has been indicted in Boston on charges that he stole more than four
million documents from the Massachusetts Institute of Technology and
JSTOR, an archive of scientific journals and academic papers. (Read
the full indictment below.)
Mr. Swartz was indicted last Thursday by the United States Attorney
for the District of Massachusetts, Carmen M. Ortiz, and the indictment
was unsealed Tuesday. The charges could result in up to 35 years in
prison and a $1 million fine.
In a press release, Ms. Ortiz’s office said that Mr. Swartz broke into
a restricted area of M.I.T. and entered a computer wiring closet. Mr.
Swartz apparently then accessed the M.I.T. computer network and took
millions of documents from JSTOR.
Mr. Swartz, a well-known figure in Internet academic circles, created
a site called Infogami that later merged with the social news site
Reddit. He is also a founder and director of the nonprofit group
Demand Progress, which calls itself a political action group hoping to
change public policy that relates to the Internet.
In 2009 Mr. Swartz downloaded 19 million pages of federal court
documents from a government database system, acting on the belief that
they should be made available free.
Demand Progress said on its site that it appeared Mr. Swartz was
“being charged with allegedly downloading too many scholarly journal
articles from the Web.” It quoted the group’s executive director,
David Segal, as saying, “It’s like trying to put someone in jail for
allegedly checking too many books out of the library.”
The charges filed against Mr. Swartz include wire fraud, computer
fraud, obtaining information from a protected computer and criminal
forfeiture.
“Stealing is stealing whether you use a computer command or a crowbar,
and whether you take documents, data or dollars,” said Ms. Ortiz in
the press release.
The United States Attorney’s office in Boston said Mr. Swartz
“surrendered today, was arraigned in U.S. District Court in Boston
this morning and pled not guilty to all counts. He was released on
$100,000 unsecured bond. His next court date is September 9, 2011 for
a status conference.”
Mr. Swartz could not immediately be reached for comment.
Hi folks,
I apologize I've not been able to pay closer attention to this
project; I didn't realize that it was running on such a tight
timeline. There are a number of issues with the Harvard survey.
For now I've set the banner campaign to "2012" and ask that we do not
run this until we've resolved the issues below. Again, I'm sorry for
the last minute notice, but this entire project has moved forward
fairly rapidly since it was first discussed here, and I'm afraid we
may have to put it on hold for several weeks.
1) This page:
http://meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and…
as well as the current banner suggests that participants will be
directed to a "login page". That login page is in fact a wiki signup
sheet on meta.
If this is, per the emails going around earlier, still intended to be
a survey with 2,000 participants, that's an absolutely hackish way to
go about verifying participant eligibility. Everything I can see seems
to indicate that it is so. Directing thousands of people to edit a
wiki signup sheet is _not_ a good idea (potential for edit conflicts,
vandalism, accidental IP address disclosure, etc., as well as reducing
likelihood that people will go through with the whole process).
Whatever this is trying to do, there are better ways to do it.
We can't go live with this if this is indeed the intent of this page.
It's too much of a hack, sorry.
2) From the people organizing the survey, I'd appreciate a working
link or copy of the actual survey questionnaire. The one shared
previously no longer works.
3) I'm also concerned about the massive banner with the Wikipedia
globe that is used for this survey in the current draft banner. This
banner suggests a very strong endorsement by WMF/Wikipedia for this
survey, when most people in WMF have probably never heard of the
project. Moreover, we don't want to assume responsibility for
technical problems, problems with the survey design, lateness of
evaluation or publication, etc. We have to choose a design and
approach that's appropriate to a survey undertaken by a third party.
A less prominent banner with the logos of the institutions undertaking
the survey would be my preference.
4) Relatedly, as a point of clarification, has this banner been coded
(as the previous editor survey) to only show once to each editor? If
not, again, I'm concerned about the intensity and the risk of
overexposing our editors to this invitation.
5)
There's a major scheduling conflict between the Harvard survey and a
survey that Wikimedia Germany has been preparing. The survey Wikimedia
Germany has been working on is a survey concerning editor health. The
project predates most of the work done in organizing and making
visible Wikimedia research, and unfortunately has no Meta wiki page --
but was planned to be deployed in July.
Again, we have to be careful with the risk of oversurveying and have
to stagger and schedule our surveys.
All in all, my preference would be to postpone the Harvard survey to
August or September, to resolve the aforementioned issues fully.
Erik
--
Erik Möller
Deputy Director, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
Hello from Okcon in Berlin!
Sorry yesterday finally I could not attend the meeting. Couldn't combine with the Okcon activities here and at the same time having an access to a public phone.
I will report on the Wikipedia & Research yesterday workshop here at the Okcon in the coming days once back home. It was short, but the room was packed of people >>> sign of a great interest!!!
Cheers! Mayo
«·´`·.(*·.¸(`·.¸ ¸.·´)¸.·*).·´`·»
«·´¨*·¸¸« Mayo Fuster Morell ».¸.·*¨`·»
«·´`·.(¸.·´(¸.·* *·.¸)`·.¸).·´`·»
Research Digital Commons Governance: http://www.onlinecreation.info
Ph.D European University Institute
Postdoctoral Researcher. Institute of Govern and Public Policies. Autonomous University of Barcelona.
Visiting scholar. Internet Interdisciplinary Institute. Open University of Catalonia (UOC).
Visiting researcher (2008). School of information. University of California, Berkeley.
Member Research Committee. Wikimedia Foundation
http://www.onlinecreation.info
E-mail: mayo.fuster(a)eui.eu
Skype: mayoneti
Phone Spanish State: 0034-648877748