[RCom-l] Important: Poll for support of subject recruitment for "Anonymity and conformity [...]" study

Wed Nov 16 23:07:00 UTC 2011

This is discussion is awesome.  I'd like to propose two options for moving
forward from this point:

   1. Allow the study to continue and pick this issue up in parallel.
      - Pro: We don't delay the study of a researcher who has been waiting.
      - Con: We set a precedent that future researchers would (possibly)
      not be able to follow.
   2. Delay this study until we can come to a general conclusion about our
   requirements for this scale of recruitment
      - Pro: We wait until approving anything until we have reached
      a consensus of what projects should be approved.
      - Con: Michael Tsikerdekis has to wait an undetermined amount of time
      before continuing with his work.

I'd like to capture our deliberations in a more public forum.  What do you
folks think about moving this discussion to the Wiki (whether it is under
this project proposal or not)?

-Aaron

On Wed, Nov 16, 2011 at 4:28 PM, Goran Milovanovic <
goran.s.milovanovic at gmail.com> wrote:

> Well, when it comes to
>
> "I would be happy to give my approval conditionally on having (at least)
> the student's supervisor involved. I actually think this would be a
> sensible rule to apply to *all* SR proposals submitted by students."
>
> I have to say I could not agree, and here is the reason: we may stumble
> upon many academic environments where students, or PhD candidates, or young
> researchers in general are in fact more competent in specific areas of
> research then their supervisors. Of course, this is not a common situation,
> but if we talk about Internet related research, where new concepts, new
> ideas, and new methods are introduced every now and then, we are talking
> about a field where the above described asymmetry in competences is likely
> to appear.
>
> Personal experience: between 2002. and 2005. a research team that I have
> led in Belgrade, Serbia, published four books on Internet behavior,
> Internet usage, attitudes towards the Internet and related topics, of which
> none were peer-reviewed in Serbia. Why? They were all supported by standard
> methodological, theoretical and statistical instruments of social sciences.
> Most of the team members already had experience in publishing in
> peer-reviewed journals or presented at recognized conferences. Simply,
> because that were the first attempts to study anything related to Internet
> and society in my country at all, and there were no peer reviewers
> available. I am literary saying that back then we were to first to try to
> crawl for the relevant literature and references at all.
>
> What I want to add is that most researchers who are into Internet related
> research are people who are into new ideas, looking for conceptual
> breakthroughs and new theoretical frameworks. Limiting their efforts by
> constrains related to the already bureaucratized relations in academia
> could end up in actually enlarging the pile of problems on the behalf of
> people who simply want to put their new ideas to test. Wikipedia was
> developed by people who decided to abandon the classical rules and try with
> something previously unimaginable: who would say it work to become of the
> landmarks of the Internet? I believe we should keep that spirit when it
> comes to research as well.
>
> "I hope it's clear that I don't want to shut down research for anybody but
> top researchers with big grants, but we also cannot afford spending time
> and effort and community attention for projects of an unclear scientific
> value or interest."
>
> I recognize the importance of the issue you're raising, but I do not think
> the selection criterion you are proposing will help us sort out the
> projects of clear from the projects of unclear value or interest. I
> believe, in the spirit of good faith, that task needs to remain with us
> completely, as well as with the members of the scientific community who we
> might contact and ask for an advice when we are not sure what to do about a
> particular project proposal.
>
> Best,
> Goran
>
>
>
>
> On Wed, Nov 16, 2011 at 11:03 PM, Dario Taraborelli <
> dtaraborelli at wikimedia.org> wrote:
>
>> ok, here's my dispassionate opinion. I post on the list because the
>> problems I refer to below are not specific to this project but to many
>> other requests.
>>
>> We have been receiving a *lot* of SR requests from students over the last
>> few months and I think we should start deciding more aggressively what
>> research is likely to have an impact and what research won't produce any
>> major tangible results.
>>
>> I really like the topic of the Anonymity and Conformity study but I
>> several concerns with the solidity of the current proposal:
>>
>> • we will be approving for the first time some kind of large-scale
>> recruitment approach via user talk pages for a student project: this is
>> something we've never done before and we should only do it if there's a
>> good reason.
>>
>> • the advisor of the proponent doesn't seem to be involved at all in this
>> project and is not even named in the proposal. Aaron asked the proponent to
>> share the name of his supervisor in September, but he hasn't done so (yet?)
>>
>> • the proponent says that no funding is supporting this research and that
>> this study is "conducted with the author's own efforts"
>>
>> • no one else other than the applicant will be implied in the data
>> collection and analysis and the proponent doesn't seem to have an actual
>> research record
>>
>> • there is no trace in this proposal of an approval by an ethics
>> committee. The proponent says that this is not applicable (and it's true
>> that IRB policy is very different between the US and other countries), but
>> some official record would help us assess the credibility of the proposal.
>>
>> for these reasons, I am hesitant whether we should blindly approve this
>> request. I would be happy to give my approval conditionally on having (at
>> least) the student's supervisor involved. I actually think this would be a
>> sensible rule to apply to *all* SR proposals submitted by students.
>>
>> I hope it's clear that I don't want to shut down research for anybody but
>> top researchers with big grants, but we also cannot afford spending time
>> and effort and community attention for projects of an unclear scientific
>> value or interest.
>>
>> I'd love to hear your thoughts on this
>>
>> Dario
>>
>> On Nov 16, 2011, at 12:30 PM, Aaron Halfaker wrote:
>>
>> I was hoping to close this poll hours ago, but we only have three members
>> of RCom participating (thanks Yaroslav and Steven!).
>>
>> It is absolutely crucial that if we end up technically approving this
>> study methodology that such approval actually reflects the consensus of
>> RCom members.
>>
>> For your benefit, I'll summarize the proposed plan:
>>
>> A request to participate in a *survey* about enforcing conformance with
>> community/group outcomes *needs 200-300 responses* from general
>> Wikipedia editors.  Invitations to take the survey will be posted an
>> editors' User_talk pages.  *A pilot set of 15 requests will be posted
>> immediately* following approval from RCom to test for problems and
>> determine the expected response rate.  Afterwards, *up to 500 User_talk
>> postings* will be made (depending on response rate) to illicit enough
>> responses to give statistical confidence.
>>
>>
>> This is the first proposed project of this scale that we are reviewing
>> for approval so I really want to make sure we are doing it right.
>>
>> -Aaron
>>
>> On Mon, Nov 14, 2011 at 8:49 AM, Aaron Halfaker <aaron.halfaker at gmail.com
>> > wrote:
>>
>>> Hey folks,
>>>
>>> This proposal is an important milestone for our subject recruitment
>>> processes, since it represents the first mass recruitment request (200-300
>>> responses needed).  I'm hoping to either show a high level of support with
>>> this poll or discover what problems still need to be dealt with.
>>>
>>> I'd like to close the poll by *Wednesday @ noon UTC*.  Please make sure
>>> to chime in.
>>>
>>> See poll:
>>> http://meta.wikimedia.org/wiki/Research_talk:Anonymity_and_conformity_over_the_net#Poll:_RCom_support_for_this_project
>>>
>>> -Aaron
>>>
>>
>> _______________________________________________
>> RCom-l mailing list
>> RCom-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/rcom-l
>>
>>
>>
>> _______________________________________________
>> RCom-l mailing list
>> RCom-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/rcom-l
>>
>>
>
>
> --
> --------------------------------------------------------------
> "Truth is much too complicated to allow
> anything but approximations."
>                              :: John von Neumann
> --------------------------------------------------------------
> http://www.milovanovicresearch.com
>
>
> _______________________________________________
> RCom-l mailing list
> RCom-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/rcom-l
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/rcom-l/attachments/20111116/6585707f/attachment.htm 