Dear All,
this is just to start the discussion. Below I copy the ideas some of us shared at http://eiximenis.wikimedia.org/ResearchCommitteeBios , and also the original functions compiled by Erik.
Functions:
The committee's core function includes:
* developing policy around researcher permissions for non-public data * supporting the development of subject recruitment processes * reviewing research projects when conflicts-of-interest arise * articulating and channeling requests for data and technical resources * helping to formulate the key strategic research objectives of the Wikimedia movement (see strategy.wikimedia.org) * helping to formulate small tactical experiments related to Wikimedia's strategic goals * developing an open access policy as a requirement for significant support from the Wikimedia Foundation * helping create a "starter kit" for researchers to avoid duplication of effort
Opinions on the scope:
What possible subcommittees/workgroups to form?
* Strategy
* Data
* Ethical review
* Subject recruitment
* Code
* Tests
* Funding
* Publishing
* Interface Research
[John Riedl]: My bias is that we should focus on creating workgroups in areas that are exploring research that is particularly hard to carry out in the current Wikipedia. One way to identify these areas is to look for types of Wikipedia research that seems like it would be very valuable but that is seldom done. Two examples of this type of research are: controlled experiments with hundreds of users, and experiments in new Wikipedia interfaces.
[Daniel Mietchen] I wonder how many subcommittees make sense given the size of the committee. Prioritizing is necessary, though, and I think substructures will naturally flow from there.
{Yaroslav Blanter] I guess we should understand 1) how meny members we are; 2) define the scope. Then we can easily decide on the number of committees and their scopes
[Milos Rancic] From the Wikimedia perspective, I would add that we need at least three general tasks, which could be handled by listed committees or by new one(s): (1) How to keep community healthy? (2) How to improve existing projects? and (3) Which new projects are needed in the corpus of free knowledge?
denny: Milos, you talking about Wiki(m/p)edia in general, or about research projects in particular? I also agree, we can only do so much with the size of the committee. Also, our task is obviously not to do the research ourselve, but provide the interface between Wikimedia and the wider research community.
[Milos Rancic] Denny, I am talking about research goals. It looks too general, but as a Wikimedia bureaucrat, I would like to get scientific answers to as many as possible questions related to those three general questions. Moved at the level of particular researches, research which would state that "edit" button should stay at some position because it would improve participation -- would be a perfect example of research which aim to cover the part of the second question (How to improve existing projects?).
[Yaroslav Blanter] Sorry, I was a bit out of business for the weekend. I in general agree with Milosh and with denny at the same time - we need to understand what we can actually do with 8 committee members. Should we first discuss what RESEARCH means? I guess there are several activities in and around WMF projects, which are covered by this term and are in some or other way in the scope of our committee, in no particular order: (1) original research performed by WMF projects (there are only two where OR is not prohibited - Wikibooks and Wikiversity), shoud there be some guidelines? (2) Scientific research on WMF projects (probably mainly Wikipedia) performed by either WMF-related or independent researchers; (3) Non-WMF-related research performed by research institutions: Here our business is only interaction between WMF projects and researchers at these institutions. Do we want to handle all three? Is there is smth else which I overlooked?
P.S. We started a serious discussion and I am not sure this is the best interface. I believe Erik promised to open a mailing list, may be all this can be moved into the list.
[denny] I agree with Yaroslav on maybe waiting for the list. These wiki-like interfaces are known to be not the best for discussions like these <reference needed>. ;)
[WereSpielChequers] I'm with Milos on the need for research to inform decision making on existing projects, there have been a number of decisions where it would have been really helpful to have a researcher look at the stats and prepare a report to inform the discussion. Also I'm with Daniel on thinking that it is a bit early to be forming subcommittes. I'm hoping tha part of our role will be in advising researchers where there are alternatives that are acceptable to both researchers and the community and part will be explaining to other researchers why some things are not acceptable to the community. For example in the last year we have had one research project where someone was vandalising articles in order to see how effective our anti-vandalism efforts were, and another was adding fake spam links to see how many people would click through to a spam site; The first could either have been done as an audit of existing articles to see how many were vandalised, or by analysing vandalsim reversions to see how long it took to spot the vandalism. I'm not convinced that there is an alternative way to do the spam test, but as Wikipedia is unlikely to be taking advertising in the foreseeable future and as the willingness to click on a link is at least partially dependent on the trust one places in the website where you see that link, then I don't see the value in testing how well spam works in Wikipedia.
Cheers Yaroslav
My suggestion to start bringing in some structure would be to take Erik's initial eight core functions and to add our names to those (up to three) that each of us intends to concentrate on.
I have just modified http://eiximenis.wikimedia.org/ResearchCommitteeBios accordingly.
Whenever four (or so) or more of us have signed up for such an item, it starts to make sense to speak of subcommittees, and the members of each should probably try to agree on a suitable name.
For points that do not reach the target of four (or so), we could think of combining them, or of inviting others who might bring in expertise and motivation to deal with these issues.
Daniel
On Fri, Aug 27, 2010 at 5:23 PM, Yaroslav M. Blanter putevod@mccme.ru wrote:
Dear All,
this is just to start the discussion. Below I copy the ideas some of us shared at http://eiximenis.wikimedia.org/ResearchCommitteeBios , and also the original functions compiled by Erik.
Functions:
The committee's core function includes:
* developing policy around researcher permissions for non-public data * supporting the development of subject recruitment processes * reviewing research projects when conflicts-of-interest arise * articulating and channeling requests for data and technical resources * helping to formulate the key strategic research objectives of the Wikimedia movement (see strategy.wikimedia.org) * helping to formulate small tactical experiments related to Wikimedia's strategic goals * developing an open access policy as a requirement for significant support from the Wikimedia Foundation * helping create a "starter kit" for researchers to avoid duplication of effort
Opinions on the scope:
What possible subcommittees/workgroups to form?
* Strategy
* Data
* Ethical review
* Subject recruitment
* Code
* Tests
* Funding
* Publishing
* Interface Research
[John Riedl]: My bias is that we should focus on creating workgroups in areas that are exploring research that is particularly hard to carry out in the current Wikipedia. One way to identify these areas is to look for types of Wikipedia research that seems like it would be very valuable but that is seldom done. Two examples of this type of research are: controlled experiments with hundreds of users, and experiments in new Wikipedia interfaces.
[Daniel Mietchen] I wonder how many subcommittees make sense given the size of the committee. Prioritizing is necessary, though, and I think substructures will naturally flow from there.
{Yaroslav Blanter] I guess we should understand 1) how meny members we are; 2) define the scope. Then we can easily decide on the number of committees and their scopes
[Milos Rancic] From the Wikimedia perspective, I would add that we need at least three general tasks, which could be handled by listed committees or by new one(s): (1) How to keep community healthy? (2) How to improve existing projects? and (3) Which new projects are needed in the corpus of free knowledge?
denny: Milos, you talking about Wiki(m/p)edia in general, or about research projects in particular? I also agree, we can only do so much with the size of the committee. Also, our task is obviously not to do the research ourselve, but provide the interface between Wikimedia and the wider research community.
[Milos Rancic] Denny, I am talking about research goals. It looks too general, but as a Wikimedia bureaucrat, I would like to get scientific answers to as many as possible questions related to those three general questions. Moved at the level of particular researches, research which would state that "edit" button should stay at some position because it would improve participation -- would be a perfect example of research which aim to cover the part of the second question (How to improve existing projects?).
[Yaroslav Blanter] Sorry, I was a bit out of business for the weekend. I in general agree with Milosh and with denny at the same time - we need to understand what we can actually do with 8 committee members. Should we first discuss what RESEARCH means? I guess there are several activities in and around WMF projects, which are covered by this term and are in some or other way in the scope of our committee, in no particular order: (1) original research performed by WMF projects (there are only two where OR is not prohibited - Wikibooks and Wikiversity), shoud there be some guidelines? (2) Scientific research on WMF projects (probably mainly Wikipedia) performed by either WMF-related or independent researchers; (3) Non-WMF-related research performed by research institutions: Here our business is only interaction between WMF projects and researchers at these institutions. Do we want to handle all three? Is there is smth else which I overlooked?
P.S. We started a serious discussion and I am not sure this is the best interface. I believe Erik promised to open a mailing list, may be all this can be moved into the list.
[denny] I agree with Yaroslav on maybe waiting for the list. These wiki-like interfaces are known to be not the best for discussions like these <reference needed>. ;)
[WereSpielChequers] I'm with Milos on the need for research to inform decision making on existing projects, there have been a number of decisions where it would have been really helpful to have a researcher look at the stats and prepare a report to inform the discussion. Also I'm with Daniel on thinking that it is a bit early to be forming subcommittes. I'm hoping tha part of our role will be in advising researchers where there are alternatives that are acceptable to both researchers and the community and part will be explaining to other researchers why some things are not acceptable to the community. For example in the last year we have had one research project where someone was vandalising articles in order to see how effective our anti-vandalism efforts were, and another was adding fake spam links to see how many people would click through to a spam site; The first could either have been done as an audit of existing articles to see how many were vandalised, or by analysing vandalsim reversions to see how long it took to spot the vandalism. I'm not convinced that there is an alternative way to do the spam test, but as Wikipedia is unlikely to be taking advertising in the foreseeable future and as the willingness to click on a link is at least partially dependent on the trust one places in the website where you see that link, then I don't see the value in testing how well spam works in Wikipedia.
Cheers Yaroslav
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l
While I understand all words, it is not clear to me what does "supporting the development of subject recruitment processes" mean exactly. In one of my interpretations, it is the field in which I am interested.
2010/8/27 Milos Rancic millosh@gmail.com:
While I understand all words, it is not clear to me what does "supporting the development of subject recruitment processes" mean exactly. In one of my interpretations, it is the field in which I am interested.
What I meant with this are situations where a researcher is trying to find Wikipedian (or reader) subjects for a particular study or experiment. Say you want to do a survey of active Wikipedia administrators: Who can grant you permission to contact them, and how do you go about doing it?