Hello,

Thanks for signalling me Aaron. I would participate in talks about research standards with other people. Perhaps the research committee can convene at the hackathon in Mexico City.

I feel that if these issues were addressed, then there would be many fewer problems.

All research on human subjects in the Wikimedia community has to be registered by creating a research page on meta. The research page needs to identify the researcher, give their contact information (for reporting complaints), describe the research minimally (2-3 sentences at least), and say which demographic of Wikimedians will be contacted and how many of them will be contacted.
There should be a research review process. To start, this could simply require that researchers have set up a research page. If there is demand for more checks then they can be added after review is set up. One additional demand or check which we might want to mandate is some cap on how many times and how many Wikimedians any given research study is allowed to contact, and in what ways. The page for the study in question says nothing about how the study will contact Wikipedians or how many it will contact, and this issue has caused a problem in this case.
Whenever researchers advertise a study, they must be required to provide a link to their research registration page. This gives the Wikimedia community a place to have discussions about the research and lodge complaints if necessary.

The problem that I would like to see addressed is a lack of respect for the time of Wikimedia community members. Practically all researchers assume that the Wikimedia community is infinitely large, and that it is no burden to consume volunteer time with research when that volunteer time could be spent developing Wikimedia projects, and that they are the only researchers seeking to tap into Wikimedia community volunteer time. The reality is that most researchers want access to small slices of the Wikimedia community, and that those people's time is scarce and that it is costly to provide them volunteer time, and that most researchers want access to the same demographics so any given study usually seeks to contact Wikipedians who are routinely contacted for research.

As more research is done, there will be more tension. I want to encourage more and better research and I know if we promoted better research practices then researchers would comply. They are accustomed to following special guidelines for special populations and I think that presenting them clear demands would make them happier and make Wikimedians happier.

yours,

On Jun 27, 2015 12:59 PM, "Aaron Halfaker" <ahalfaker@wikimedia.org> wrote:

RCom, as far as I know has not been active in the past year or more (last meeting was on Dec. 22, 2011).

RCom is not dead. It changed into something less formal and less hierarchical. You can still email me and Dario to get support for your research plans. We'd still reconvene the committee if it looks like that'll help.

While RCom hasn't met in a long time, the process for subject recruitment hasn't slowed. We don't have a technical requirement that all recruitment studies must follow The Process, but I have been helping researchers document their studies and obtain feedback and sometimes consensus for more than five years now.

Really, RCom has morphed slowly into the Research Team at the WMF + a few interested volunteers that we can manage to pull in to help us with review work (shout out to Daniel Mietchen, Nemo, Yaroslav & BluRasberry). Within the research team, we *do* have structured processed for supporting researchers access to data and engineering support, but subject recruitment has been mostly left in my (volunteer time) hands.

Regretfully, I wasn't involved in the planning of this project or I would have directed it towards best practices for minimizing disruption -- e.g. an RFC. I would have also pushed Leila to find a way to make posts on talk pages work (since they are known to be generally preferable, police-able, etc.), but I can understand why concerns around privacy might be worth discussion. I regret that this discussion only happened after-the-fact as it could have informed the study design for the better. FWIW, SuggestBot posts recommendations on user talk pages and also does not filter for offensive content (to my knowledge).

Finally, I think it is important to consider the source of this research work. Leila is not some random academic or industry researcher who is planning to take advantage of Wikipedians for a study, but not give back. Leila is working with a team at the WMF tasked with building better translation tools. She helped them design an experiment that would explore the effectiveness of these tools so that when something is deployed, it's actually better and we know it scientifically. A lot of the work I do with external researchers is to help make sure that their work has the potential to benefit Wikipedia/Wikipedians/Wikimedia/Open knowledge. In this case, the Leila's team is just helping the product teams engage in best practices around empirical software change practice. After all, every software deployment is an experiment that is inflicted upon you without consent. In this case, Leila's job is making sure that we know the effect before we deploy.

So, what I really mean to say is:
You're right. We should do this better. We have a process and everyone should go through it. It might have caught some of the issues that have been raised.
Leila is WMF staff. She's trying to help the WMF build better software for the purpose of benefiting Wikipedians. Her team deserves some slack. The alternative of not running the study is less desirable.
-Aaron

On Sat, Jun 27, 2015 at 12:56 PM, Michelle Paulson <mpaulson@wikimedia.org> wrote:
Hi All,

Please see in-line below.

-Michelle

On Saturday, June 27, 2015, Leila Zia <leila@wikimedia.org> wrote:

> + Michelle Paulson
>
> On Sat, Jun 27, 2015 at 7:37 AM, Pine W <wiki.pine@gmail.com
> <javascript:_e(%7B%7D,'cvml','wiki.pine@gmail.com');>> wrote:
>
>> This issue is also being discussed on the Research mailing list.
>>
>> I have three questions:
>>
>> 1. Was this outreach method approved by RCom?
>>
> No, and RCom, as far as I know has not been active in the past year or
> more (last meeting was on Dec. 22, 2011). This is a research from the
> Research team in the WMF.
>
>> 2. Email addresses are nonpublic information on-wiki unless they are
>> proactively and publicly disclosed by users. Does the bulk collection of
>> nonpublic email addresses in this manner and the bulk provision of those
>> addresses to researchers for their use in this campaign violate the
>> Wikimedia privacy policy? The policy states regarding email, "We use your
>> email address to let you know about things that are happening with the
>> Foundation, the Wikimedia Sites, or the Wikimedia movement, such as telling
>> you important information about your account, letting you know if something
>> is changing about the Wikimedia Sites or policies, and alerting you when
>> there has been a change to an article that you have decided to follow." The
>> bulk scraping of email addresses from account registrations for research
>> and outreach purposes doesn't appear to be contemplated or authorized under
>> the privacy policy.
>>
> Michelle can help with this one as this is related to Legal. Note that
> it's weekend here and this may have to wait until Monday.
>

The research team did speak to me prior to beginning this project to ensure
that they complied with the WMF privacy policy. It is my view that this
type of use falls within the permissible potential uses for email addresses
under the policy. The examples listed in the policy are meant to be
illustrative, not exclusive -- the absence of this situation as an
enumerated example shouldn't be taken as a prohibition.

That said, it is a new use and therefore, will and should be the subject of
discussion and debate. It is such feedback and testing that will help us
refine email practices to be both effective and reflective of community
values.

> 3. Wouldn't talk pages be a more appropriate outreach method than bulk
>> email?
>>
> The reason we chose email over talk pages (or Echo notifications) is
> explained here
> <https://meta.wikimedia.org/wiki/Research_talk:Increasing_article_coverage#.C2.AB_recommander_par_courriel_des_articles_.C3.A0_cr.C3.A9er.E2.80.A6_.C2.BB>.
>
>
> Hope this helps.
>
> Best,
> Leila
>
>> Thanks,
>>
>> Pine
>>
>> _______________________________________________
>> Wiki-research-l mailing list
>> Wiki-research-l@lists.wikimedia.org
>> <javascript:_e(%7B%7D,'cvml','Wiki-research-l@lists.wikimedia.org');>
>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>>
>>
>

--
==
Michelle Paulson
Senior Legal Counsel
Wikimedia Foundation
149 New Montgomery Street, 6th Floor
San Francisco, CA 94105
mpaulson@wikimedia.org
415.839.6885 ext. 6608 (Office)
415.882.0495 (Fax)

*NOTICE: This message may be confidential or legally privileged. If you
have received it by accident, please delete it and let us know about the
mistake. As an attorney for the Wikimedia Foundation and for legal/ethical
reasons, I cannot give legal advice to, or serve as a lawyer for, community
members, volunteers, or staff members in their personal capacity. For more
on what this means, please see our legal disclaimer
<https://meta.wikimedia.org/wiki/Wikimedia_Legal_Disclaimer>.*

_______________________________________________
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l@lists.wikimedia.org
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l