Has this been considered? It seems to apply to us in many ways.
http://news.yahoo.com/course-reddit-imgur-named-research-institute-derp-1429...
Indeed. Jonathan, Dario and I have been in contact with Tim Hwang and the other DERP organizers for months. We're a big fan of the project and we're on the DERP mailing list. Regretfully, there was some confusion around the time that DERP was going to "go live" that required us to back out (e.g. use of the WMF logo needed to be authorized).
On Wed, Sep 3, 2014 at 9:32 AM, Samuel Klein meta.sj@gmail.com wrote:
Has this been considered? It seems to apply to us in many ways.
http://news.yahoo.com/course-reddit-imgur-named-research-institute-derp-1429...
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
So does that mean that it's on the agenda for whichever group approves use of the WMF logo?
On Wed, Sep 3, 2014 at 2:36 PM, Aaron Halfaker ahalfaker@wikimedia.org wrote:
Indeed. Jonathan, Dario and I have been in contact with Tim Hwang and the other DERP organizers for months. We're a big fan of the project and we're on the DERP mailing list. Regretfully, there was some confusion around the time that DERP was going to "go live" that required us to back out (e.g. use of the WMF logo needed to be authorized).
On Wed, Sep 3, 2014 at 9:32 AM, Samuel Klein meta.sj@gmail.com wrote:
Has this been considered? It seems to apply to us in many ways.
http://news.yahoo.com/course-reddit-imgur-named-research-institute-derp-1429...
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Sep 3, 2014, at 8:36 AM, Aaron Halfaker ahalfaker@wikimedia.org wrote:
Indeed. Jonathan, Dario and I have been in contact with Tim Hwang and the other DERP organizers for months. We're a big fan of the project and we're on the DERP mailing list. Regretfully, there was some confusion around the time that DERP was going to "go live" that required us to back out (e.g. use of the WMF logo needed to be authorized).
Is the DERP mailing list public yet?
//Ed
When I started working with DERP, it was just a mailing list. When DERP became something bigger (press release with logos == the WMF Comm Team's concerns), it needed wider consultation within the WMF. For better or worse, it does not seem like that is a priority right now.
For all intents and purposes, we have our own DERP right here. We have a public mailing list. We support volunteer & external researchers access to data (and we're actively working to make that easier[1]). We hold regular outreach events (like the Research Hackathon @ Wikimania). We're also organizing outreach events with other open ecosystem online communities. For example, we have CSCW'15 workshop proposal submitted, and assuming it is accepted, we'll have participants from Imgur, Reddit and Zooniverse.
1. quarry.wmflabs.org
-Aaron
On Wed, Sep 3, 2014 at 3:05 PM, Ed Summers ehs@pobox.com wrote:
On Sep 3, 2014, at 8:36 AM, Aaron Halfaker ahalfaker@wikimedia.org wrote:
Indeed. Jonathan, Dario and I have been in contact with Tim Hwang and
the other DERP organizers for months. We're a big fan of the project and we're on the DERP mailing list. Regretfully, there was some confusion around the time that DERP was going to "go live" that required us to back out (e.g. use of the WMF logo needed to be authorized).
Is the DERP mailing list public yet?
//Ed _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Sigh. Yes. We were early adopters of DERP. Then we had to pull out. But it might still happen someday. It's complicated :/
On Wed, Sep 3, 2014 at 7:42 AM, Aaron Halfaker ahalfaker@wikimedia.org wrote:
When I started working with DERP, it was just a mailing list. When DERP became something bigger (press release with logos == the WMF Comm Team's concerns), it needed wider consultation within the WMF. For better or worse, it does not seem like that is a priority right now.
For all intents and purposes, we have our own DERP right here. We have a public mailing list. We support volunteer & external researchers access to data (and we're actively working to make that easier[1]). We hold regular outreach events (like the Research Hackathon @ Wikimania). We're also organizing outreach events with other open ecosystem online communities. For example, we have CSCW'15 workshop proposal submitted, and assuming it is accepted, we'll have participants from Imgur, Reddit and Zooniverse.
- quarry.wmflabs.org
-Aaron
On Wed, Sep 3, 2014 at 3:05 PM, Ed Summers ehs@pobox.com wrote:
On Sep 3, 2014, at 8:36 AM, Aaron Halfaker ahalfaker@wikimedia.org wrote:
Indeed. Jonathan, Dario and I have been in contact with Tim Hwang and
the other DERP organizers for months. We're a big fan of the project and we're on the DERP mailing list. Regretfully, there was some confusion around the time that DERP was going to "go live" that required us to back out (e.g. use of the WMF logo needed to be authorized).
Is the DERP mailing list public yet?
//Ed _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Sep 3, 2014, at 11:00 AM, Jonathan Morgan jmorgan@wikimedia.org wrote:
Sigh. Yes. We were early adopters of DERP. Then we had to pull out. But it might still happen someday. It's complicated :/
If you can share why it's complicated I'd love to hear ; I suspect it's political, but oftentimes these politics have research implications.
///Ed
On Wed, Sep 3, 2014 at 8:12 AM, Ed Summers ehs@pobox.com wrote:
On Sep 3, 2014, at 11:00 AM, Jonathan Morgan jmorgan@wikimedia.org wrote:
Sigh. Yes. We were early adopters of DERP. Then we had to pull out. But it might still happen someday. It's complicated :/
If you can share why it's complicated I'd love to hear ; I suspect it's political, but oftentimes these politics have research implications.
Ditto.
Ed, SJ,
on top of what Aaron and Jonathan said (for example, the use of WMF’s logo) there were other concerns that we had to address.
The scope of DERP shifted from an informal communication broker between organizations and academic researchers to a body that – at least according to the original press release – would be responsible for:
• setting guidelines for ethical research (for example, by prohibiting any kind of feature testing/experimentation) • aggregating and sharing privacy sensitive data • reviewing data requests
on behalf of the member organizations.
The Foundation has specific policies on what data can be collected from readers and contributors, how long this data is retained, how it’s released and licensed and under what conditions third parties can obtain access to private data hosted by the Foundation. We felt that we could not join an initiative that would commit the organization to make promises about data access to external parties, without having a conversation with our internal stakeholders first–not just WMF Legal or the WMF analytics team but also our volunteer community, chapters etc.
The press release was later amended to remove language that would suggest specific commitments for member organizations but our concerns remain the same.
Jonathan, Aaron and I are all interested in creating opportunities for cross-platform research on online collaboration and we have plans in the pipeline to make more data publicly available. We are big fans of DERP and are still involved in a personal capacity in the initiative, but we are not in a position to take Wikimedia with us at this time.
I hope that clarifies how the decision was made.
Dario
On Sep 3, 2014, at 1:03 PM, Samuel Klein meta.sj@gmail.com wrote:
On Wed, Sep 3, 2014 at 8:12 AM, Ed Summers ehs@pobox.com wrote:
On Sep 3, 2014, at 11:00 AM, Jonathan Morgan jmorgan@wikimedia.org wrote:
Sigh. Yes. We were early adopters of DERP. Then we had to pull out. But it might still happen someday. It's complicated :/
If you can share why it's complicated I'd love to hear ; I suspect it's political, but oftentimes these politics have research implications.
Ditto.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I think many organisations would be reluctant to sign up to DERP on that basis. It seems somewhat ironic that a group apparently committed to ethical research seems not to have considered that existing data sets may have been collected under strict conditions of use and re-use as part of existing ethical processes.
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about data set availability but retains separate control on access. Presumably it would easier for WMF to sign up for this lower level of engagement.
Sent from my iPad
On 4 Sep 2014, at 6:46 am, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
Ed, SJ,
on top of what Aaron and Jonathan said (for example, the use of WMF’s logo) there were other concerns that we had to address.
The scope of DERP shifted from an informal communication broker between organizations and academic researchers to a body that – at least according to the original press release – would be responsible for:
• setting guidelines for ethical research (for example, by prohibiting any kind of feature testing/experimentation) • aggregating and sharing privacy sensitive data • reviewing data requests
on behalf of the member organizations.
The Foundation has specific policies on what data can be collected from readers and contributors, how long this data is retained, how it’s released and licensed and under what conditions third parties can obtain access to private data hosted by the Foundation. We felt that we could not join an initiative that would commit the organization to make promises about data access to external parties, without having a conversation with our internal stakeholders first–not just WMF Legal or the WMF analytics team but also our volunteer community, chapters etc.
The press release was later amended to remove language that would suggest specific commitments for member organizations but our concerns remain the same.
Jonathan, Aaron and I are all interested in creating opportunities for cross-platform research on online collaboration and we have plans in the pipeline to make more data publicly available. We are big fans of DERP and are still involved in a personal capacity in the initiative, but we are not in a position to take Wikimedia with us at this time.
I hope that clarifies how the decision was made.
Dario
On Sep 3, 2014, at 1:03 PM, Samuel Klein meta.sj@gmail.com wrote:
On Wed, Sep 3, 2014 at 8:12 AM, Ed Summers ehs@pobox.com wrote:
On Sep 3, 2014, at 11:00 AM, Jonathan Morgan jmorgan@wikimedia.org wrote: Sigh. Yes. We were early adopters of DERP. Then we had to pull out. But it might still happen someday. It's complicated :/
If you can share why it's complicated I'd love to hear ; I suspect it's political, but oftentimes these politics have research implications.
Ditto.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Wed, Sep 3, 2014 at 10:14 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about data set availability but retains separate control on access.
<lol> ... To boldly go where GIT has gone before. </lol>
Hi all, thanks for all the info. I'm a DERP fellow, which means I was planning on participating in this as a researcher (I'm doing some work on reddit, too) as well as serving as an advisory board. I apparently haven't been involved in the same threads/calls with the DERP organizers that Aaron, Jonathan, and Dario have been on, and I'm kind of shocked at what I'm hearing. I completely believe you guys, it just runs so opposite to what I've been told that I'm dreading the e-mail I think I'm going to have to write to the DERP folks.
This is the first time I've heard anything about DERP being much more than an informal communication broker between organizations and academic researchers. DERP was pitched to me as a big signaling mechanism to researchers, platforms, and the public that there are spaces outside of Facebook and Twitter to do research. Wikimedia obviously doesn't need DERP as much as some of the smaller platforms do, but I thought it would be great for Wikimedia's presence (yes, the logo) to be there, standing in solidarity with the lesser-researched platforms. As it was explained to me, all that was supposed to be involved in a platform joining DERP is 1) a public declaration that they are open to receiving requests from researchers via DERP and 2) a commitment to review and respond to proposals that were e-mailed from researchers to DERP. In one of the fellows calls, I actually think someone asked whether DERP would be like an Institutional Review Board that would independently approve/reject studies, and we all thought that it would be better for these to be done on a case-by-case basis between the researcher and the platform(s).
Early on, I actually suggested adding some language about ethics. I suggested that as we started these projects, it would be great to develop an ongoing, informal set of best practices for doing computational social science in an academic/industry partnership -- particularly in the wake of the Facebook emotion contagion study. Something like a series of blog posts about the various ethical issues we encountered in the course of doing this kind of research across a bunch of different platforms, and ways that they were resolved. Perhaps that might synthesize into a mini workshop culminating in a whitepaper, but it wouldn't ever be binding. As I was told about it, DERP's direct role ends once the researcher has made successful contact with the platform, aside from very high-level community organizing things like discussions about best practices. Same thing with data standards -- it is a fool's errand to mandate those, but I was told that DERP might one day be a hub where people could talk about how to integrate data from different platforms.
I did see the language that "All research supported by DERP will be released openly and made publicly available," but I interpreted this as something even weaker than Green OA -- that even if you publish in a closed access journal, you have to write something up about the research. Kind of like what Aaron did with our ABS paper. [1] The idea was that you should't be able to do studies in the dark without anybody ever knowing about them. The fellows were told that this wouldn't apply to datasets at all. And given how many qualitative researchers are fellows and planning on doing interviews, the concern that we would have to release full interview transcripts was specifically brought up. Again, the idea was that DERP might later develop some optional, guiding best practices to make things easier, but any conditions of data access were supposed to be negotiated between the researcher and the platform. I remember asking if DERP was supposed to be some kind of central repository for storing data, and that was resoundingly rejected.
So if DERP has shifted beyond this, that would be a pretty serious matter of concern for me.
Best, Stuart
[1] http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/
On Wed, Sep 3, 2014 at 2:32 PM, Joe Corneli holtzermann17@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:14 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about data
set
availability but retains separate control on access.
<lol> ... To boldly go where GIT has gone before. </lol>
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I don't think there's cause for you to be concerned, Stu. FWIW, we've talked to Tim since launch, and after we expressed our concerns he assured us that the model of DERP is still just facilitating connections in a non-exclusive way, rather than playing a role as a reviewing body or a data broker of any kind.
There were other reasons we decided to be a little more cautious about committing to this kind of initiative. As Toby Negrin pointed out recently: There is one major difference between the companies involved in DERP and ourselves -- they all use data collected from their users to make money and we explicitly do not. This is frankly a point of pride for many members of the foundation and certainly the community.
More pragmatically, the last week of organizing for the DERP launch just happened too fast for us (and happened during Wikimania, to boot!). Those of us in research-y roles hadn't had a chance to discuss all the evolving details as a team, and on the eve of the launch we didn't all feel we had a 100% clear idea of what commitments we would be making by joining.
But we're still on the DERP mailing list, and (if the review gods are merciful) we plan to co-organize a CSCW workshop with Tim Hwang and Max Goodman at CSCW 2015.
We like DERP! Don't stop DERPing!
- Jonathan
On Wed, Sep 3, 2014 at 3:31 PM, R.Stuart Geiger sgeiger@gmail.com wrote:
Hi all, thanks for all the info. I'm a DERP fellow, which means I was planning on participating in this as a researcher (I'm doing some work on reddit, too) as well as serving as an advisory board. I apparently haven't been involved in the same threads/calls with the DERP organizers that Aaron, Jonathan, and Dario have been on, and I'm kind of shocked at what I'm hearing. I completely believe you guys, it just runs so opposite to what I've been told that I'm dreading the e-mail I think I'm going to have to write to the DERP folks.
This is the first time I've heard anything about DERP being much more than an informal communication broker between organizations and academic researchers. DERP was pitched to me as a big signaling mechanism to researchers, platforms, and the public that there are spaces outside of Facebook and Twitter to do research. Wikimedia obviously doesn't need DERP as much as some of the smaller platforms do, but I thought it would be great for Wikimedia's presence (yes, the logo) to be there, standing in solidarity with the lesser-researched platforms. As it was explained to me, all that was supposed to be involved in a platform joining DERP is 1) a public declaration that they are open to receiving requests from researchers via DERP and 2) a commitment to review and respond to proposals that were e-mailed from researchers to DERP. In one of the fellows calls, I actually think someone asked whether DERP would be like an Institutional Review Board that would independently approve/reject studies, and we all thought that it would be better for these to be done on a case-by-case basis between the researcher and the platform(s).
Early on, I actually suggested adding some language about ethics. I suggested that as we started these projects, it would be great to develop an ongoing, informal set of best practices for doing computational social science in an academic/industry partnership -- particularly in the wake of the Facebook emotion contagion study. Something like a series of blog posts about the various ethical issues we encountered in the course of doing this kind of research across a bunch of different platforms, and ways that they were resolved. Perhaps that might synthesize into a mini workshop culminating in a whitepaper, but it wouldn't ever be binding. As I was told about it, DERP's direct role ends once the researcher has made successful contact with the platform, aside from very high-level community organizing things like discussions about best practices. Same thing with data standards -- it is a fool's errand to mandate those, but I was told that DERP might one day be a hub where people could talk about how to integrate data from different platforms.
I did see the language that "All research supported by DERP will be released openly and made publicly available," but I interpreted this as something even weaker than Green OA -- that even if you publish in a closed access journal, you have to write something up about the research. Kind of like what Aaron did with our ABS paper. [1] The idea was that you should't be able to do studies in the dark without anybody ever knowing about them. The fellows were told that this wouldn't apply to datasets at all. And given how many qualitative researchers are fellows and planning on doing interviews, the concern that we would have to release full interview transcripts was specifically brought up. Again, the idea was that DERP might later develop some optional, guiding best practices to make things easier, but any conditions of data access were supposed to be negotiated between the researcher and the platform. I remember asking if DERP was supposed to be some kind of central repository for storing data, and that was resoundingly rejected.
So if DERP has shifted beyond this, that would be a pretty serious matter of concern for me.
Best, Stuart
[1] http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/
On Wed, Sep 3, 2014 at 2:32 PM, Joe Corneli holtzermann17@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:14 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about
data set
availability but retains separate control on access.
<lol> ... To boldly go where GIT has gone before. </lol>
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
What I think we really need is better standardisation of description of datasets, so that they can shared in machine-readable ways. Then we can have as many different groups working with different sets of datasets as we like and still search, find and publish globally.
cheers stuart
On Thu, Sep 4, 2014 at 11:29 AM, Jonathan Morgan jmorgan@wikimedia.org wrote:
I don't think there's cause for you to be concerned, Stu. FWIW, we've talked to Tim since launch, and after we expressed our concerns he assured us that the model of DERP is still just facilitating connections in a non-exclusive way, rather than playing a role as a reviewing body or a data broker of any kind.
There were other reasons we decided to be a little more cautious about committing to this kind of initiative. As Toby Negrin pointed out recently: There is one major difference between the companies involved in DERP and ourselves -- they all use data collected from their users to make money and we explicitly do not. This is frankly a point of pride for many members of the foundation and certainly the community.
More pragmatically, the last week of organizing for the DERP launch just happened too fast for us (and happened during Wikimania, to boot!). Those of us in research-y roles hadn't had a chance to discuss all the evolving details as a team, and on the eve of the launch we didn't all feel we had a 100% clear idea of what commitments we would be making by joining.
But we're still on the DERP mailing list, and (if the review gods are merciful) we plan to co-organize a CSCW workshop with Tim Hwang and Max Goodman at CSCW 2015.
We like DERP! Don't stop DERPing!
- Jonathan
On Wed, Sep 3, 2014 at 3:31 PM, R.Stuart Geiger sgeiger@gmail.com wrote:
Hi all, thanks for all the info. I'm a DERP fellow, which means I was planning on participating in this as a researcher (I'm doing some work on reddit, too) as well as serving as an advisory board. I apparently haven't been involved in the same threads/calls with the DERP organizers that Aaron, Jonathan, and Dario have been on, and I'm kind of shocked at what I'm hearing. I completely believe you guys, it just runs so opposite to what I've been told that I'm dreading the e-mail I think I'm going to have to write to the DERP folks.
This is the first time I've heard anything about DERP being much more than an informal communication broker between organizations and academic researchers. DERP was pitched to me as a big signaling mechanism to researchers, platforms, and the public that there are spaces outside of Facebook and Twitter to do research. Wikimedia obviously doesn't need DERP as much as some of the smaller platforms do, but I thought it would be great for Wikimedia's presence (yes, the logo) to be there, standing in solidarity with the lesser-researched platforms. As it was explained to me, all that was supposed to be involved in a platform joining DERP is 1) a public declaration that they are open to receiving requests from researchers via DERP and 2) a commitment to review and respond to proposals that were e-mailed from researchers to DERP. In one of the fellows calls, I actually think someone asked whether DERP would be like an Institutional Review Board that would independently approve/reject studies, and we all thought that it would be better for these to be done on a case-by-case basis between the researcher and the platform(s).
Early on, I actually suggested adding some language about ethics. I suggested that as we started these projects, it would be great to develop an ongoing, informal set of best practices for doing computational social science in an academic/industry partnership -- particularly in the wake of the Facebook emotion contagion study. Something like a series of blog posts about the various ethical issues we encountered in the course of doing this kind of research across a bunch of different platforms, and ways that they were resolved. Perhaps that might synthesize into a mini workshop culminating in a whitepaper, but it wouldn't ever be binding. As I was told about it, DERP's direct role ends once the researcher has made successful contact with the platform, aside from very high-level community organizing things like discussions about best practices. Same thing with data standards -- it is a fool's errand to mandate those, but I was told that DERP might one day be a hub where people could talk about how to integrate data from different platforms.
I did see the language that "All research supported by DERP will be released openly and made publicly available," but I interpreted this as something even weaker than Green OA -- that even if you publish in a closed access journal, you have to write something up about the research. Kind of like what Aaron did with our ABS paper. [1] The idea was that you should't be able to do studies in the dark without anybody ever knowing about them. The fellows were told that this wouldn't apply to datasets at all. And given how many qualitative researchers are fellows and planning on doing interviews, the concern that we would have to release full interview transcripts was specifically brought up. Again, the idea was that DERP might later develop some optional, guiding best practices to make things easier, but any conditions of data access were supposed to be negotiated between the researcher and the platform. I remember asking if DERP was supposed to be some kind of central repository for storing data, and that was resoundingly rejected.
So if DERP has shifted beyond this, that would be a pretty serious matter of concern for me.
Best, Stuart
[1] http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/
On Wed, Sep 3, 2014 at 2:32 PM, Joe Corneli holtzermann17@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:14 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about data set availability but retains separate control on access.
<lol> ... To boldly go where GIT has gone before. </lol>
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Jonathan T. Morgan Learning Strategist Wikimedia Foundation User:Jmorgan (WMF) jmorgan@wikimedia.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Hmm, a meta data standard. there must be a few dozen on the shelf to pick from ... I recall writing a few myself once :-)
Sent from my iPad
On 4 Sep 2014, at 9:53 am, "Stuart A. Yeates" syeates@gmail.com wrote:
What I think we really need is better standardisation of description of datasets, so that they can shared in machine-readable ways. Then we can have as many different groups working with different sets of datasets as we like and still search, find and publish globally.
cheers stuart
On Thu, Sep 4, 2014 at 11:29 AM, Jonathan Morgan jmorgan@wikimedia.org wrote: I don't think there's cause for you to be concerned, Stu. FWIW, we've talked to Tim since launch, and after we expressed our concerns he assured us that the model of DERP is still just facilitating connections in a non-exclusive way, rather than playing a role as a reviewing body or a data broker of any kind.
There were other reasons we decided to be a little more cautious about committing to this kind of initiative. As Toby Negrin pointed out recently: There is one major difference between the companies involved in DERP and ourselves -- they all use data collected from their users to make money and we explicitly do not. This is frankly a point of pride for many members of the foundation and certainly the community.
More pragmatically, the last week of organizing for the DERP launch just happened too fast for us (and happened during Wikimania, to boot!). Those of us in research-y roles hadn't had a chance to discuss all the evolving details as a team, and on the eve of the launch we didn't all feel we had a 100% clear idea of what commitments we would be making by joining.
But we're still on the DERP mailing list, and (if the review gods are merciful) we plan to co-organize a CSCW workshop with Tim Hwang and Max Goodman at CSCW 2015.
We like DERP! Don't stop DERPing!
- Jonathan
On Wed, Sep 3, 2014 at 3:31 PM, R.Stuart Geiger sgeiger@gmail.com wrote:
Hi all, thanks for all the info. I'm a DERP fellow, which means I was planning on participating in this as a researcher (I'm doing some work on reddit, too) as well as serving as an advisory board. I apparently haven't been involved in the same threads/calls with the DERP organizers that Aaron, Jonathan, and Dario have been on, and I'm kind of shocked at what I'm hearing. I completely believe you guys, it just runs so opposite to what I've been told that I'm dreading the e-mail I think I'm going to have to write to the DERP folks.
This is the first time I've heard anything about DERP being much more than an informal communication broker between organizations and academic researchers. DERP was pitched to me as a big signaling mechanism to researchers, platforms, and the public that there are spaces outside of Facebook and Twitter to do research. Wikimedia obviously doesn't need DERP as much as some of the smaller platforms do, but I thought it would be great for Wikimedia's presence (yes, the logo) to be there, standing in solidarity with the lesser-researched platforms. As it was explained to me, all that was supposed to be involved in a platform joining DERP is 1) a public declaration that they are open to receiving requests from researchers via DERP and 2) a commitment to review and respond to proposals that were e-mailed from researchers to DERP. In one of the fellows calls, I actually think someone asked whether DERP would be like an Institutional Review Board that would independently approve/reject studies, and we all thought that it would be better for these to be done on a case-by-case basis between the researcher and the platform(s).
Early on, I actually suggested adding some language about ethics. I suggested that as we started these projects, it would be great to develop an ongoing, informal set of best practices for doing computational social science in an academic/industry partnership -- particularly in the wake of the Facebook emotion contagion study. Something like a series of blog posts about the various ethical issues we encountered in the course of doing this kind of research across a bunch of different platforms, and ways that they were resolved. Perhaps that might synthesize into a mini workshop culminating in a whitepaper, but it wouldn't ever be binding. As I was told about it, DERP's direct role ends once the researcher has made successful contact with the platform, aside from very high-level community organizing things like discussions about best practices. Same thing with data standards -- it is a fool's errand to mandate those, but I was told that DERP might one day be a hub where people could talk about how to integrate data from different platforms.
I did see the language that "All research supported by DERP will be released openly and made publicly available," but I interpreted this as something even weaker than Green OA -- that even if you publish in a closed access journal, you have to write something up about the research. Kind of like what Aaron did with our ABS paper. [1] The idea was that you should't be able to do studies in the dark without anybody ever knowing about them. The fellows were told that this wouldn't apply to datasets at all. And given how many qualitative researchers are fellows and planning on doing interviews, the concern that we would have to release full interview transcripts was specifically brought up. Again, the idea was that DERP might later develop some optional, guiding best practices to make things easier, but any conditions of data access were supposed to be negotiated between the researcher and the platform. I remember asking if DERP was supposed to be some kind of central repository for storing data, and that was resoundingly rejected.
So if DERP has shifted beyond this, that would be a pretty serious matter of concern for me.
Best, Stuart
[1] http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/
On Wed, Sep 3, 2014 at 2:32 PM, Joe Corneli holtzermann17@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:14 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about data set availability but retains separate control on access.
<lol> ... To boldly go where GIT has gone before. </lol>
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Jonathan T. Morgan Learning Strategist Wikimedia Foundation User:Jmorgan (WMF) jmorgan@wikimedia.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Equally important are "standards" (wrong term really) for dataset descriptions so they can be shared in HUMAN readable/comprehensible ways.
This is often the bigger problem, resulting in publishing and sharing efforts that don't really work.
On Sep 4, 2014, at 12:05 AM, Kerry Raymond wrote:
Hmm, a meta data standard. there must be a few dozen on the shelf to pick from ... I recall writing a few myself once :-)
Sent from my iPad
On 4 Sep 2014, at 9:53 am, "Stuart A. Yeates" syeates@gmail.com wrote:
What I think we really need is better standardisation of description of datasets, so that they can shared in machine-readable ways. Then we can have as many different groups working with different sets of datasets as we like and still search, find and publish globally.
cheers stuart
On Thu, Sep 4, 2014 at 11:29 AM, Jonathan Morgan jmorgan@wikimedia.org wrote: I don't think there's cause for you to be concerned, Stu. FWIW, we've talked to Tim since launch, and after we expressed our concerns he assured us that the model of DERP is still just facilitating connections in a non-exclusive way, rather than playing a role as a reviewing body or a data broker of any kind.
There were other reasons we decided to be a little more cautious about committing to this kind of initiative. As Toby Negrin pointed out recently: There is one major difference between the companies involved in DERP and ourselves -- they all use data collected from their users to make money and we explicitly do not. This is frankly a point of pride for many members of the foundation and certainly the community.
More pragmatically, the last week of organizing for the DERP launch just happened too fast for us (and happened during Wikimania, to boot!). Those of us in research-y roles hadn't had a chance to discuss all the evolving details as a team, and on the eve of the launch we didn't all feel we had a 100% clear idea of what commitments we would be making by joining.
But we're still on the DERP mailing list, and (if the review gods are merciful) we plan to co-organize a CSCW workshop with Tim Hwang and Max Goodman at CSCW 2015.
We like DERP! Don't stop DERPing!
- Jonathan
On Wed, Sep 3, 2014 at 3:31 PM, R.Stuart Geiger sgeiger@gmail.com wrote:
Hi all, thanks for all the info. I'm a DERP fellow, which means I was planning on participating in this as a researcher (I'm doing some work on reddit, too) as well as serving as an advisory board. I apparently haven't been involved in the same threads/calls with the DERP organizers that Aaron, Jonathan, and Dario have been on, and I'm kind of shocked at what I'm hearing. I completely believe you guys, it just runs so opposite to what I've been told that I'm dreading the e-mail I think I'm going to have to write to the DERP folks.
This is the first time I've heard anything about DERP being much more than an informal communication broker between organizations and academic researchers. DERP was pitched to me as a big signaling mechanism to researchers, platforms, and the public that there are spaces outside of Facebook and Twitter to do research. Wikimedia obviously doesn't need DERP as much as some of the smaller platforms do, but I thought it would be great for Wikimedia's presence (yes, the logo) to be there, standing in solidarity with the lesser-researched platforms. As it was explained to me, all that was supposed to be involved in a platform joining DERP is 1) a public declaration that they are open to receiving requests from researchers via DERP and 2) a commitment to review and respond to proposals that were e-mailed from researchers to DERP. In one of the fellows calls, I actually think someone asked whether DERP would be like an Institutional Review Board that would independently approve/reject studies, and we all thought that it would be better for these to be done on a case-by-case basis between the researcher and the platform(s).
Early on, I actually suggested adding some language about ethics. I suggested that as we started these projects, it would be great to develop an ongoing, informal set of best practices for doing computational social science in an academic/industry partnership -- particularly in the wake of the Facebook emotion contagion study. Something like a series of blog posts about the various ethical issues we encountered in the course of doing this kind of research across a bunch of different platforms, and ways that they were resolved. Perhaps that might synthesize into a mini workshop culminating in a whitepaper, but it wouldn't ever be binding. As I was told about it, DERP's direct role ends once the researcher has made successful contact with the platform, aside from very high-level community organizing things like discussions about best practices. Same thing with data standards -- it is a fool's errand to mandate those, but I was told that DERP might one day be a hub where people could talk about how to integrate data from different platforms.
I did see the language that "All research supported by DERP will be released openly and made publicly available," but I interpreted this as something even weaker than Green OA -- that even if you publish in a closed access journal, you have to write something up about the research. Kind of like what Aaron did with our ABS paper. [1] The idea was that you should't be able to do studies in the dark without anybody ever knowing about them. The fellows were told that this wouldn't apply to datasets at all. And given how many qualitative researchers are fellows and planning on doing interviews, the concern that we would have to release full interview transcripts was specifically brought up. Again, the idea was that DERP might later develop some optional, guiding best practices to make things easier, but any conditions of data access were supposed to be negotiated between the researcher and the platform. I remember asking if DERP was supposed to be some kind of central repository for storing data, and that was resoundingly rejected.
So if DERP has shifted beyond this, that would be a pretty serious matter of concern for me.
Best, Stuart
[1] http://www-users.cs.umn.edu/~halfak/publications/The_Rise_and_Decline/
On Wed, Sep 3, 2014 at 2:32 PM, Joe Corneli holtzermann17@gmail.com wrote:
On Wed, Sep 3, 2014 at 10:14 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
It would seem that DERP needs to permit at least two levels of participation, one of which allows information to be provided about data set availability but retains separate control on access.
<lol> ... To boldly go where GIT has gone before. </lol>
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Jonathan T. Morgan Learning Strategist Wikimedia Foundation User:Jmorgan (WMF) jmorgan@wikimedia.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On 4 Sep 2014, at 9:53 am, "Stuart A. Yeates" syeates@gmail.com wrote:
What I think we really need is better standardisation of description
On Thu, Sep 4, 2014 at 4:05 PM, Kerry Raymond kerry.raymond@gmail.com wrote:
Hmm, a meta data standard. there must be a few dozen on the shelf to pick from ...
There's more to standardisation than writing standards. Often implementing a checker / validator is much more useful in propagating compliance.
cheers stuart
On Sep 3, 2014, at 7:29 PM, Jonathan Morgan jmorgan@wikimedia.org wrote:
There were other reasons we decided to be a little more cautious about committing to this kind of initiative. As Toby Negrin pointed out recently: There is one major difference between the companies involved in DERP and ourselves -- they all use data collected from their users to make money and we explicitly do not. This is frankly a point of pride for many members of the foundation and certainly the community.
Wikimedia is a nonprofit, but that doesn't mean it can't bring in money based on data collected from its users. I think we all know that this is exactly what it does. As a non-profit WMF is just prevented from making a profit, right?
More pragmatically, the last week of organizing for the DERP launch just happened too fast for us (and happened during Wikimania, to boot!). Those of us in research-y roles hadn't had a chance to discuss all the evolving details as a team, and on the eve of the launch we didn't all feel we had a 100% clear idea of what commitments we would be making by joining.
Ok, this is much more plausible. I'm new to the idea of DERP, but based on what Stuart just wrote it does sound like a useful effort to be a part of.
But we're still on the DERP mailing list, and (if the review gods are merciful) we plan to co-organize a CSCW workshop with Tim Hwang and Max Goodman at CSCW 2015.
Do you know how to get on the DERP mailing list?
//Ed
Do you know how to get on the DERP mailing list?
It looks like inquire@derp.institute is the right place to ask.
On Thu, Sep 4, 2014 at 8:52 AM, Ed Summers ehs@pobox.com wrote:
On Sep 3, 2014, at 7:29 PM, Jonathan Morgan jmorgan@wikimedia.org wrote:
There were other reasons we decided to be a little more cautious about committing to this kind of initiative. As Toby Negrin pointed out recently: There is one major difference between the companies involved in DERP and ourselves -- they all use data collected from their users to make money and we explicitly do not. This is frankly a point of pride for many members of the foundation and certainly the community.
Wikimedia is a nonprofit, but that doesn't mean it can't bring in money based on data collected from its users. I think we all know that this is exactly what it does. As a non-profit WMF is just prevented from making a profit, right?
More pragmatically, the last week of organizing for the DERP launch just happened too fast for us (and happened during Wikimania, to boot!). Those of us in research-y roles hadn't had a chance to discuss all the evolving details as a team, and on the eve of the launch we didn't all feel we had a 100% clear idea of what commitments we would be making by joining.
Ok, this is much more plausible. I'm new to the idea of DERP, but based on what Stuart just wrote it does sound like a useful effort to be a part of.
But we're still on the DERP mailing list, and (if the review gods are merciful) we plan to co-organize a CSCW workshop with Tim Hwang and Max Goodman at CSCW 2015.
Do you know how to get on the DERP mailing list?
//Ed
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
<quote who="Aaron Halfaker" date="Wed, Sep 03, 2014 at 04:42:24PM +0200">
For all intents and purposes, we have our own DERP right here. We have a public mailing list. We support volunteer & external researchers access to data (and we're actively working to make that easier[1]). We hold regular outreach events (like the Research Hackathon @ Wikimania). We're also organizing outreach events with other open ecosystem online communities. For example, we have CSCW'15 workshop proposal submitted, and assuming it is accepted, we'll have participants from Imgur, Reddit and Zooniverse.
I think this is right.
The goal of DERP, as I've understood it, is to provide a more clear way to interface between researchers and social media organizations. The goal, in that sense, is to go a little bit of the way toward helping other organizations become more like WMF in this respect. :)
Because the infrastructure and interfacing that DERP aims to provide is pretty similar to what WMF already does, the marginal benefit to Wikimedia researchers may be much smaller than it will be for people interested in working with all of the other DERP data providers.
Of course, that's no reason that WMF shouldn't participate, support, and lend its good reputation in this space to this effort. I hope we can figure out the logo stuff. It seems like this should be a very small issue and a silly reason to have this fall over.
Regards, Mako
Also some coverage here in the Economist: http://www.economist.com/blogs/babbage/2014/09/science-web
On Wed, Sep 3, 2014 at 3:32 AM, Samuel Klein meta.sj@gmail.com wrote:
Has this been considered? It seems to apply to us in many ways.
http://news.yahoo.com/course-reddit-imgur-named-research-institute-derp-1429...
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I would point out the difference between (concept) conflation vs (technological) convergence.
To my knowledge, the meta data, i.e. the possible data about its user-readers and user-contributors are sometimes gathered systematically for user experience (UX) and interface testing, data analytics and research (both in-house and external). Most of the time such metadata could be framed as behavioral, or be aggregated as social groups.
So the questions come back to us, the subscribers of "Research into Wikimedia content and communities". Have we generated some type of *revenue from mining its user contributed data*?
I would thus suggest using the term meta-data to describe any user data/patterns that can be mined from the basic edit data points. It is better not to conflate the two because the latter one has clearer social contract (submit this and you agree these). The two may be converging because of the technological tools we may have as researchers, or the next Talk page system, or the some metric systems to measure content and/or users. Such converging only demonstrates the increasing complexity of the data eco-system but does not grant us the leisure of conceptual conflation.
What does the action of hitting the edit/submit button entail for data governance? Some how I still think a distinction does exist between the edit data and the other contextual data (time stamps, ids, etc.) and derived meta data.
2014-09-04 23:07 GMT+02:00 Brian Keegan b.keegan@neu.edu:
Also some coverage here in the Economist: http://www.economist.com/blogs/babbage/2014/09/science-web
On Wed, Sep 3, 2014 at 3:32 AM, Samuel Klein meta.sj@gmail.com wrote:
Has this been considered? It seems to apply to us in many ways.
http://news.yahoo.com/course-reddit-imgur-named-research-institute-derp-1429...
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Brian C. Keegan, Ph.D. Post-Doctoral Research Fellow, Lazer Lab College of Social Sciences and Humanities, Northeastern University Fellow, Institute for Quantitative Social Sciences, Harvard University Affiliate, Berkman Center for Internet & Society, Harvard Law School
b.keegan@neu.edu www.brianckeegan.com M: 617.803.6971 O: 617.373.7200 Skype: bckeegan
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
wiki-research-l@lists.wikimedia.org