There has been a lot of talk about how to start a journal. The real issue in starting a journal is not the editorial board, or the way it is published, or whether it will gather the citation impact. The real issue is READERSHIP.
If you can get people to read the journal, then it will have editors wanting to serve the journal, and it will gather citation impact.
The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.
--------------- Ed H. Chi, Staff Research Scientist, Google CHI2012 Technical Program co-chair
On Sun, Nov 4, 2012 at 6:14 PM, Ed H. Chi chi@acm.org wrote:
There has been a lot of talk about how to start a journal. The real issue in starting a journal is not the editorial board, or the way it is published, or whether it will gather the citation impact. The real issue is READERSHIP.
I like this observation. A few natural follow up questions to people here would be:
(1) Where do you currently read about wiki research? (2) Where do you currently publish about wiki research? (3) What's missing?
For me:
(1) I get a surprising amount of leads from conversations that happen on this list, and I don't pay all that much attention to where I end up grabbing the papers from in the end.
(2) I've published at WikiSym, and in (for instance) Proceedings of the International Conference on Computational Science, or other subject-specific conferences/workshops. One of these papers was picked up and republished by Digital Education Review. I also contributed to a paper that was published at Alt.CHI.
(3) For me, what seems most "missing" is a place to talk about the future of research in a *productive* (not necessarily "scholarly") way. For some thoughts gleaned from this list, see http://meta.wikimedia.org/wiki/Wiki_Research_Ideas#Rethinking_the_future_of_... -- but where to continue things like this? Not sure.
Yes, that's a good point. Nevertheless, readership is not an inherent quality. It depends on several factors, most of which are attractiveness (is the journal layout fine to read? does the selection brings up insightful content?), positioning (which scientific disciplines are concerned?) and mediatization (how do we come to guarantee a specific notability of the journal per se?).
To this extent, the journal has some valuable assets : *Us :) There have been already a lot of discussion since september. So far, the issue is well-debated and well-explored. *The Wikimedian communities, that comprise many academics. *The Open Access communities that may well be interested in this kind of experiment. In the specific context of the Academic Spring, the journal may possibly receive some comment in the general press. *Having the advantage of Eprint (wider access) without its drawback (economic model…). *International scale. I have actually given some publicity to the concept on my French well-read blog : http://blogs.rue89.com/les-coulisses-de-wikipedia/2012/10/23/libre-acces-les...
So far, the editorial board may have a good latitude in managing (or not managing) to attract readership…
PCL
Le 4 nov. 12 à 20:00, Joe Corneli a écrit :
On Sun, Nov 4, 2012 at 6:14 PM, Ed H. Chi chi@acm.org wrote:
There has been a lot of talk about how to start a journal. The real issue in starting a journal is not the editorial board, or the way it is published, or whether it will gather the citation impact. The real issue is READERSHIP.
I like this observation. A few natural follow up questions to people here would be:
(1) Where do you currently read about wiki research? (2) Where do you currently publish about wiki research? (3) What's missing?
For me:
(1) I get a surprising amount of leads from conversations that happen on this list, and I don't pay all that much attention to where I end up grabbing the papers from in the end.
(2) I've published at WikiSym, and in (for instance) Proceedings of the International Conference on Computational Science, or other subject-specific conferences/workshops. One of these papers was picked up and republished by Digital Education Review. I also contributed to a paper that was published at Alt.CHI.
(3) For me, what seems most "missing" is a place to talk about the future of research in a *productive* (not necessarily "scholarly") way. For some thoughts gleaned from this list, see http://meta.wikimedia.org/wiki/Wiki_Research_Ideas#Rethinking_the_future_of_... -- but where to continue things like this? Not sure.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
So long as Google scholar is indexing the journal, I see no problem with finding a readership.
How often do you start your quest for relevant literature by sitting down with a set of journal titles? Yes, that's how we used to do it. But now everyone just sits down in front of Google Scholar and types keywords and relevant papers are found regardless of the journal they are published in.
Kerry
Ed and others, based on your observations, I'd like to pose a side question:
The impression that I get from many of these symposia (and journals) is that there is not much space for research concerning Wikipedia and Education, such as teaching methodologies, case studies and such, not on the side of hard-science chunks of data. I know of lots of other professors who are doing the same thing as myself, but I see not many places for exchanging our experiences (conference-wise, not online channels, which, franky, I don't think are working much). Do you feel there is good room to topics such as mine?
Thanks, Juliana.
On Sun, Nov 4, 2012 at 4:14 PM, Ed H. Chi chi@acm.org wrote:
There has been a lot of talk about how to start a journal. The real issue in starting a journal is not the editorial board, or the way it is published, or whether it will gather the citation impact. The real issue is READERSHIP.
If you can get people to read the journal, then it will have editors wanting to serve the journal, and it will gather citation impact.
The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.
Ed H. Chi, Staff Research Scientist, Google CHI2012 Technical Program co-chair
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
This is exactly my problem nowadays. I am a historian and don't have much to say about software and data mining, but would like to read more about the humanities approach with regard to Wikipedia. Readers experiences and expectations, Wikipedia in school and university etc. Kind regards Ziko
2012/11/4 Juliana Bastos Marques domusaurea@gmail.com:
Ed and others, based on your observations, I'd like to pose a side question:
The impression that I get from many of these symposia (and journals) is that there is not much space for research concerning Wikipedia and Education, such as teaching methodologies, case studies and such, not on the side of hard-science chunks of data.
actually, with our community, it is not. What other journals die for, we have sort of provided. This is why a Wiki journal may have a better chance than others, but only if it is prepared with the academic career paths and full proper code of conduct nuances considered (double-blind scholarly peer review, proper editorial board, PDFs with page numbers, etc.).
dj
On Sun, Nov 4, 2012 at 7:14 PM, Ed H. Chi chi@acm.org wrote:
There has been a lot of talk about how to start a journal. The real issue in starting a journal is not the editorial board, or the way it is published, or whether it will gather the citation impact. The real issue is READERSHIP.
If you can get people to read the journal, then it will have editors wanting to serve the journal, and it will gather citation impact.
The reason why WikiSym is changing is for the same reason. People are not going to the conference! I think the attendance has been below 100 for some time now. That's not a sustainable number for the amount of work that goes into organizing a conference.
Ed H. Chi, Staff Research Scientist, Google CHI2012 Technical Program co-chair
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
While wiki speeds collaboration within a community, the research literature favors long-lasting contributions outside the community. Wiki or wiki-like collaboration tools might be better applied to stimulating and conducting high-quality research that will then usefully feed existing publications.
I assume here that Wikipedia and the like have been sufficiently successful to raise more research questions and to enable more research methods than this community has time and/or ability to pursue. (If topics were scarce then they might be usefully hoarded.)
Of course there are many ways to share methods and directions and lots have already been applied in this community. However, I get the impression that results still take months or years to produce.
Aside: I have built a data mining tool and methodology, Exploratory Parsing, that can read all of Wikipedia in 10 seconds for a useful notion of "read". I have also created a Federated Wiki that promotes wiki-like sharing without need for a common vision or agreed social norms. I intend to combine the two to make an experiment manager where setups are easily shared and where methodological improvements and/or research redirections easily build on other's work in progress.
What would our environment have to be like before our collaborative cycle time is reduced to days?
On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote:
On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc.
I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct.
Wondering further, I recently became acquainted with the Sage Bioinformatics Synapse platform:
by way of a keynote at the O'Reilly Strata Rx conference by the Sage president, Stephen Friend;
http://www.youtube.com/watch?v=m4Pvq4sldbQ
A later talk (apparently not online) by Erich Huang made it clear that their goal is to move research to the open source, open access arena. Erich spoke in terms of putting everything from research journals, to data, to software used for analysis online (e.g. at GitHub), making it available for continuous peer review, evaluation, forking and evolution, to augment the way science is communicated.
That notion seems very much in the spirit of Ward's Wiki Way.
I am still investigating Synapse. I hope to know much more about it soon.
Jack
On Sun, Nov 4, 2012 at 4:57 PM, Ward Cunningham ward@c2.com wrote:
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
While wiki speeds collaboration within a community, the research literature favors long-lasting contributions outside the community. Wiki or wiki-like collaboration tools might be better applied to stimulating and conducting high-quality research that will then usefully feed existing publications.
I assume here that Wikipedia and the like have been sufficiently successful to raise more research questions and to enable more research methods than this community has time and/or ability to pursue. (If topics were scarce then they might be usefully hoarded.)
Of course there are many ways to share methods and directions and lots have already been applied in this community. However, I get the impression that results still take months or years to produce.
Aside: I have built a data mining tool and methodology, Exploratory Parsing, that can read all of Wikipedia in 10 seconds for a useful notion of "read". I have also created a Federated Wiki that promotes wiki-like sharing without need for a common vision or agreed social norms. I intend to combine the two to make an experiment manager where setups are easily shared and where methodological improvements and/or research redirections easily build on other's work in progress.
What would our environment have to be like before our collaborative cycle time is reduced to days?
On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote:
On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc.
I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I just watched Stephen Friend's presentation. 15 minutes well spent. Thanks. -- Ward
On Nov 4, 2012, at 5:13 PM, Jack Park wrote:
Wondering further, I recently became acquainted with the Sage Bioinformatics Synapse platform:
by way of a keynote at the O'Reilly Strata Rx conference by the Sage president, Stephen Friend;
On Sun, Nov 4, 2012 at 4:57 PM, Ward Cunningham ward@c2.com wrote:
Aside: I have built a data mining tool and methodology, Exploratory Parsing, that can read all of Wikipedia in 10 seconds for a useful notion of "read". I have also created a Federated Wiki that promotes wiki-like sharing without need for a common vision or agreed social norms. I intend to combine the two to make an experiment manager where setups are easily shared and where methodological improvements and/or research redirections easily build on other's work in progress.
What would our environment have to be like before our collaborative cycle time is reduced to days?
I would highly encourage researchers interested in collaborative systems to take a look at Federated Wiki. Collaboration among experts is a very interesting potential use case, considering the way the wiki handles data and visualization, as well as the git-like way is allows for collaboration.
Le 5 nov. 2012 à 01:57, Ward Cunningham ward@c2.com a écrit :
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
While wiki speeds collaboration within a community, the research literature favors long-lasting contributions outside the community. Wiki or wiki-like collaboration tools might be better applied to stimulating and conducting high-quality research that will then usefully feed existing publications.
I may sound a bit overractive, but can't we do both? I would easily imagine the following two-way system: *A wiki-laboratory, which hosts quick and less quick projects in progress. That could also include some reactive analysis by social scientists to recent wikimedian and collaborative knowledge phenomena. *A wiki-journal, that should give a definitive and quotable form to the preceding researches and enlarge the discussions to wider disciplinary debates. By publishing external submissions it could give some new food for thought to the wiki-laboratory. The relationships between the two structures would be a relevant analogy to the mutually-rewarding dialogue between the practical and theoretical side of scientific research.
On Nov 4, 2012, at 2:27 PM, Chitu Okoli wrote:
On one hand, I agree that readership is very important. On the other hand, I don't believe it is the most critical issue. I think the most critical matter in starting a new journal is to *attract high-quality submissions* --that is, to get researchers who do high quality work to submit some of their best research to the journal. If that can be achieved, then a high readership is virtually guaranteed. Other benefits should readily follow: high citations, ISI listing, etc.
I've been reading all the comments on this thread carefully, and I see two distinct issues that are being mixed into one: (1) a new journal dedicated to research about wikis, and maybe also about related phenomena; and (2) a journal with new kind of wiki-based peer review system. I think it is best to rather keep these issues distinct.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Mon, Nov 5, 2012 at 1:59 AM, Pierre-Carl Langlais langlais.qobuz@gmail.com wrote:
I may sound a bit overractive, but can't we do both? I would easily imagine the following two-way system: *A wiki-laboratory, which hosts quick and less quick projects in progress. That could also include some reactive analysis by social scientists to recent wikimedian and collaborative knowledge phenomena. *A wiki-journal, that should give a definitive and quotable form to the preceding researches and enlarge the discussions to wider disciplinary debates. By publishing external submissions it could give some new food for thought to the wiki-laboratory. The relationships between the two structures would be a relevant analogy to the mutually-rewarding dialogue between the practical and theoretical side of scientific research.
I think we should be aware of two interesting "countervailing" trends.
(1) Observe that we are STILL having this conversation on a mailing list, despite the existence of a wiki page that is in theory devoted to exploring precisely these issues. (http://meta.wikimedia.org/wiki/Wiki_Research_Ideas)
It would appear that working on a wiki is extra work compared to working on a mailing list (at least for this "exploratory" phase of the conversation). That's not meant to be a normative judgment, it just points to the need for a broader awareness of what wiki-like research might look like. In particular, it may also look like "correspondence". So, let's imagine that the wiki-laboratory was already prototyped and tested on usenet. What should our next steps be?
(2) Observe that the wiki journal idea is connected with a particular research obsession (for some), namely "finding experts". Of course, in order to find an expert, the expert must first have been created or manufactured. In a positive light, this means "education". In a negative light, it means regimes for producing stratification and alienation.
Both, of course, exist already. So again the question is one of next steps, not something "de novo". My sense is that that practicality is usually abhorred (it's expensive, mundane, and you can't "get credit for it"), whereas theory is strongly preferred (it's powerful, efficient, and it can go on your CV).
In my view, the only sensible "solution" is to dissolve the (imagined) separation between practice and theory, and look instead at the actual practices of researchers and knowledge workers, trying to support them (i.e. us), in what we actually do.
I think two things can be done in parallel.
1. Allow folks to create descriptions of research in progress on the wiki, which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working.
2. Have a more formal traditional review process, so that the journal mets the criteria for "reputable" that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary.
I don't think it should be a requirement that you engage in 1) before engaging in 2). But I would like to believe that people who engage in 1) are far more likely to be have an easy acceptance through 2) because their work has already had the benefit of "many eyes".
Where I think this plan is likely to come unstuck relates to the question of authorship of the final papers for the purposes of 2). If someone feels that they have made a lot of contribution to the research through 1) they might feel entitled to author rights. In this regard, it is worth reminding ourselves of the Vancouver protocol, which many journals either mandate or will resort to in the event of authorship disputes:
http://www.authorder.com/index.php?option=com_content&view=article&i... mid=47 (a short version)
http://www.icmje.org/urm_full.pdf (a long version see pages 2-3)
An open way of working will clearly enable people to contribute to the research and the writing in ways that might make them eligible to be an author under the Vancouver protocol. The argument will tend to hinge on the question of whether the contributions were "substantial".
But I don't think this concern is a reason not to enable more collaborative ways of researching. It's just something that everyone should be aware of up-front, that opening your research up to input from others might mean you have to add some co-authors to your work if they make "substantial" contributions.
Kerry
Good points, Kerry, sup!
dj
On Mon, Nov 5, 2012 at 9:05 PM, Kerry Raymond kerry.raymond@gmail.comwrote:
I think two things can be done in parallel.
- Allow folks to create descriptions of research in progress on the wiki,
which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working.
- Have a more formal traditional review process, so that the journal mets
the criteria for "reputable" that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary.
I don't think it should be a requirement that you engage in 1) before engaging in 2). But I would like to believe that people who engage in 1) are far more likely to be have an easy acceptance through 2) because their work has already had the benefit of "many eyes".
Where I think this plan is likely to come unstuck relates to the question of authorship of the final papers for the purposes of 2). If someone feels that they have made a lot of contribution to the research through 1) they might feel entitled to author rights. In this regard, it is worth reminding ourselves of the Vancouver protocol, which many journals either mandate or will resort to in the event of authorship disputes:
http://www.authorder.com/index.php?option=com_content&view=article&i... mid=47 (a short version)
http://www.icmje.org/urm_full.pdf (a long version see pages 2-3)
An open way of working will clearly enable people to contribute to the research and the writing in ways that might make them eligible to be an author under the Vancouver protocol. The argument will tend to hinge on the question of whether the contributions were "substantial".
But I don't think this concern is a reason not to enable more collaborative ways of researching. It's just something that everyone should be aware of up-front, that opening your research up to input from others might mean you have to add some co-authors to your work if they make "substantial" contributions.
Kerry
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I would note that the use of 1) would render double-blind irrelevant in 2). We would all know ...
Sent from my iPad
On 06/11/2012, at 6:05 AM, "Kerry Raymond" kerry.raymond@gmail.com wrote:
I think two things can be done in parallel.
- Allow folks to create descriptions of research in progress on the wiki,
which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working.
- Have a more formal traditional review process, so that the journal mets
the criteria for "reputable" that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary.
I don't think it should be a requirement that you engage in 1) before engaging in 2). But I would like to believe that people who engage in 1) are far more likely to be have an easy acceptance through 2) because their work has already had the benefit of "many eyes".
Where I think this plan is likely to come unstuck relates to the question of authorship of the final papers for the purposes of 2). If someone feels that they have made a lot of contribution to the research through 1) they might feel entitled to author rights. In this regard, it is worth reminding ourselves of the Vancouver protocol, which many journals either mandate or will resort to in the event of authorship disputes:
http://www.authorder.com/index.php?option=com_content&view=article&i... mid=47 (a short version)
http://www.icmje.org/urm_full.pdf (a long version see pages 2-3)
An open way of working will clearly enable people to contribute to the research and the writing in ways that might make them eligible to be an author under the Vancouver protocol. The argument will tend to hinge on the question of whether the contributions were "substantial".
But I don't think this concern is a reason not to enable more collaborative ways of researching. It's just something that everyone should be aware of up-front, that opening your research up to input from others might mean you have to add some co-authors to your work if they make "substantial" contributions.
Kerry
On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham ward@c2.com wrote:
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
+1*
Regarding the existing conversation, if we want a journal, we need to ask what the purpose is.
I'd highly recommend Jason Priem's notion of the "decoupled journal" [1][2]. Jason points out that journals have been used for four main purposes, historically:
1. Registration 2. Archiving 3. Dissemination 4. Certification
Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us?
We can ask:
(1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies?
(2) How can wikis contribute to altmetrics [3] used for certification functions?
(3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need?
(4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...)
I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward.
-Jodi
* of course, reporting and doing research aren't an either/or -- they're closely related and one drives the other
[1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface-quality...
[2] Also a draft article called "Decoupling the scholarly journal" by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue "Beyond open access: visions for open evaluation of scientific papers by post-publication peer review" https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5...
+1 Jodi!
I agree it would be great to experiment on-site as you suggest
Claudia
On Mon, 5 Nov 2012 10:39:34 +0000, Jodi Schneider wrote
On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham ward@c2.com wrote:
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
+1*
Regarding the existing conversation, if we want a journal, we need to ask what the purpose is.
I'd highly recommend Jason Priem's notion of the "decoupled journal" [1][2]. Jason points out that journals have been used for four main purposes, historically:
- Registration
- Archiving
- Dissemination
- Certification
Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us?
We can ask:
(1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies?
(2) How can wikis contribute to altmetrics [3] used for certification functions?
(3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need?
(4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...)
I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward.
-Jodi
- of course, reporting and doing research aren't an either/or -- they're
closely related and one drives the other
[1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface- quality-content-jason-priem-on-the-decoupled-journal-as-the-achievable- future-of-scholarly-communication/
[2] Also a draft article called "Decoupling the scholarly journal" by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue "Beyond open access: visions for open evaluation of scientific papers by post-publication peer review" https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5...
hl=en_US&authkey=CMeCqOYD
thanks & cheers, Claudia koltzenburg@w4w.net
As a side consideration, I think that "science" is elitist, today. Obviously, there are some required rules to assure and assess what is sciencie and what is not, but we have the opportunity to open science to the world.
Until now people just consume science. We are in a historical moment to welcome and engage the entire world in science: proposing stuff to be researched, developing tools, extracting and curating data, and checking and peer-reviewing facts.
In the same way writing the world memory is a task for all us (Wikipedia), sciencie follows the same principles.
Open science, open research, open review, open data.
2012/11/5 Jodi Schneider jschneider@pobox.com
On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham ward@c2.com wrote:
I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
+1*
Regarding the existing conversation, if we want a journal, we need to ask what the purpose is.
I'd highly recommend Jason Priem's notion of the "decoupled journal" [1][2]. Jason points out that journals have been used for four main purposes, historically:
- Registration
- Archiving
- Dissemination
- Certification
Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us?
We can ask:
(1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies?
(2) How can wikis contribute to altmetrics [3] used for certification functions?
(3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need?
(4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...)
I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward.
-Jodi
- of course, reporting and doing research aren't an either/or -- they're
closely related and one drives the other
[1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here:
http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface-quality...
[2] Also a draft article called "Decoupling the scholarly journal" by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue "Beyond open access: visions for open evaluation of scientific papers by post-publication peer review"
https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5...https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5pb8/edit?hl=en_US&authkey=CMeCqOYD
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Following the discussion of yesterday, I have enhanced a bit the design draft: http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Design
It now includes a specific thema for each issue. For instance, I have chosen « Wikipedia Verifiability ».
In order to visualize what kind of content we could get, I have replaced the "Lorem ipsum" stuff with « fake » summaries.
As you can see, the journal may include both social science analyses (cf. the quellenkritik to wikipedia thing), and computer science experiment (the device from wikisource).
Those who are interesting in taking part to the project may add their names here : http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Volunteers
Kind regards,
As a side consideration, I think that "science" is elitist, today. Obviously, there are some required rules to assure and assess what is sciencie and what is not, but we have the opportunity to open science to the world.
Until now people just consume science. We are in a historical moment to welcome and engage the entire world in science: proposing stuff to be researched, developing tools, extracting and curating data, and checking and peer-reviewing facts.
In the same way writing the world memory is a task for all us (Wikipedia), sciencie follows the same principles.
Open science, open research, open review, open data.
2012/11/5 Jodi Schneider jschneider@pobox.com On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham ward@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
+1*
Regarding the existing conversation, if we want a journal, we need to ask what the purpose is.
I'd highly recommend Jason Priem's notion of the "decoupled journal" [1][2]. Jason points out that journals have been used for four main purposes, historically: Registration Archiving Dissemination Certification Decoupling these functions is the way forward for scholarly communication. And it's already been happening -- with ArXiV, SSRN, Math Overflow, ... and new ways of measuring research impact [3]. So which function(s) matter most to us?
We can ask:
(1) What can wikis do for the registration, archiving, and/or dissemination functions, better than existing technologies?
(2) How can wikis contribute to altmetrics [3] used for certification functions?
(3) How can we as a community surface the most interesting and powerful research? What technologies do we need? What social habits do we need?
(4) And, finally, how can our answers to #3 contribute to the certification we value? (Prestige, publications counting for tenure, ...)
I think rather than trying to create a high profile, high impact traditional journal, if we focused on these and similar questions, we would both move wiki research forward, and drive scientific communication itself forward.
-Jodi
- of course, reporting and doing research aren't an either/or --
they're closely related and one drives the other
[1] Jason Priem at Purdue: video http://youtu.be/OM22JuiWYgE slides https://docs.google.com/present/view?id=ddfg787c_362f465q2g5 I've written a short summary here: http://jodischneider.com/blog/2012/11/04/altmetrics-can-help-surface-quality...
[2] Also a draft article called "Decoupling the scholarly journal" by Jason Priem and Bradley M. Hemminger, under review for the Frontiers in Computational Neuroscience special issue "Beyond open access: visions for open evaluation of scientific papers by post- publication peer review" https://docs.google.com/document/d/1xDOy9GXXrUFc9TUIR2C470DTau8JEgZ9k-SMNIx5...
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Emilio J. Rodríguez-Posada http://LibreFind.org - The wiki search engine
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
On Nov 5, 2012, at 2:39 AM, Jodi Schneider wrote:
On Mon, Nov 5, 2012 at 12:57 AM, Ward Cunningham ward@c2.com wrote: I wonder if a better place to innovate might be in the conduct of research, rather than the reporting, review and publication of research?
+1*
Regarding the existing conversation, if we want a journal, we need to ask what the purpose is.
I'd highly recommend Jason Priem's notion of the "decoupled journal" [1][2]. Jason points out that journals have been used for four main purposes, historically: Registration Archiving Dissemination Certification
Having now caught up on Jason Priem I will say that he raises good points about "refactoring" journals.
Jody and others have correctly pointed out:
- of course, reporting and doing research aren't an either/or -- they're closely related and one drives the other
Jason argues that twitter, etc, can be mined to create a new and faster version of "Certification" (I simplify). This seems right but also transient in that online habits change so rapidly. Perhaps they change because we move from service to service looking for the one with the schema (and thus, community) we need at the moment.
Wiki started with almost no schema and remains light in this regard today. This favor's innovation while it complicates interpretation. My own suspicion is that wiki's low-schema is better suited for the "laboratory" than the "journal", at least as a journal is currently conceived. Wiki may be well suited for a "refactored" journal, simply because it makes experiments easy, and this is what makes Jason's premise apropos.
Best regards. -- Ward
I equally see the journal as a way to bridge research as it is, and research as it's likely to become. The main work the editing committee is liable to do is social: enforce usual procedures, keep in touch with the ISI and other certification centers, find partnerships with scholar institutions, ensure communication toward scientific and general public. Is such a work inessential? I am not so sure. During the OA Week meeting at the UNESCO, we all came to agree on one point: the main (if not the sole) obstacle to the generalization of new scientific practices consists in current, unquestioned, hard to get rid off habits. Designing some kind of intermediary between pioneering initiatives and mainstream reflexes may have some relevancy, not in the long-term perspective, but now.
The journal may have therefore two valuable objectives: *It practices an extended version of open access. The mere facts of publishing peer-reviewing debates and getting outsiders to take a small, albeit definite, part in the writing and reviewing process are already quite unusual novelties. If the journal becomes a success wide enough to encourage other journals in imitating it, I think the whole project would have been rewarding. *It's a good way to recruit new scholars, especially for the wiki- laboratory. I know a lot of French academics who are interested in Wikis. Yet they mainly issue solitary analysis without taking part to wider wiki-research.
Le 5 nov. 12 à 19:35, Ward Cunningham a écrit :
Wiki started with almost no schema and remains light in this regard today. This favor's innovation while it complicates interpretation. My own suspicion is that wiki's low-schema is better suited for the "laboratory" than the "journal", at least as a journal is currently conceived. Wiki may be well suited for a "refactored" journal, simply because it makes experiments easy, and this is what makes Jason's premise apropos.
I'm agreeing with you on the refactoring factor. Wiki is a place where people act rather than say. It may not be the most convenient structure to host a journal, although Wikipedia has already imported some tools and behaviors from scientific journals: footnotes, FA/GA reviewing and so on. It is a powerful device to reveal social and textual interactions. As such, it seems to have all the requirements to experiment extended open access, by showing the elaboration of editorial decisions, peer-reviewing discussions, corrections, works in progress from a proposal to a complete article…
PCL
I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the "ridiculous gymnastics" required.
Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify.
Sent from my iPad
On 05/11/2012, at 8:30 AM, "Chitu Okoli" Chitu.Okoli@concordia.ca wrote:
Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are)
Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind).
If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research.
Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven.
~ Chitu
Dariusz Jemielniak a écrit :
actually, with our community, it is not. What other journals die for, we have sort of provided. This is why a Wiki journal may have a better chance than others, but only if it is prepared with the academic career paths and full proper code of conduct nuances considered (double-blind scholarly peer review, proper editorial board, PDFs with page numbers, etc.).
dj
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
The woman discrimination is something the journal should care about. Any idea on how to face it?
2012/11/6 Chitu Okoli Chitu.Okoli@concordia.ca
Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions:
- If there is a "big name" researcher who wants to take advantage of
his/her reputation, he/she would not request double-blind.
- If there is a "big name" researcher who is modest and does not think
highly of himself/herself, he/she would not request double-blind.
- If there is a minority or woman researcher afraid of discrimination, if
he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women.
Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical.
In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals.
~ Chitu
Kerry Raymond a écrit :
I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the "ridiculous gymnastics" required.
Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify.
Sent from my iPad
On 05/11/2012, at 8:30 AM, "Chitu Okoli" Chitu.Okoli@concordia.ca Chitu.Okoli@concordia.ca wrote:
Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are)
Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind).
If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research.
Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven.
~ Chitu
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
just out of curiosity, what could be the reasonable expected purposes for requesting a single-blind review instead of a standard double-blind in your model?
best,
dj
On Tue, Nov 6, 2012 at 3:56 AM, Chitu Okoli Chitu.Okoli@concordia.cawrote:
Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions:
- If there is a "big name" researcher who wants to take advantage of
his/her reputation, he/she would not request double-blind.
- If there is a "big name" researcher who is modest and does not think
highly of himself/herself, he/she would not request double-blind.
- If there is a minority or woman researcher afraid of discrimination, if
he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women.
Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical.
In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals.
~ Chitu
Kerry Raymond a écrit :
I think a compromise position is to use single-blind unless the authors request double-blind and are therefore prepared to undertake the "ridiculous gymnastics" required.
Certainly (in computer science) I would agree that it is very hard for any established researcher publishing in their normal field to successfully disguise their identify.
Sent from my iPad
On 05/11/2012, at 8:30 AM, "Chitu Okoli" Chitu.Okoli@concordia.ca Chitu.Okoli@concordia.ca wrote:
Although in theory double-blind review is superior to single-blind, in practice double-blind vs. single-blind review has very little to do with journal quality. It is much more a matter of disciplinary culture. (Single-blind: authors don't know who the reviewers are, but reviewers do know who authors are; Double-blind: neither authors nor reviewers know who each other are)
Yes, I am certainly aware of studies that show that double-blind reviewing does indeed reduce the bias towards publishing famous researchers and reduces the bias against publishing work by minority researchers and women. So, I believe that double-blind reviewing is indeed meaningful. However, my general observation is that the decision for a journal to be double-blind or single-blind is mainly a matter of disciplinary tradition. To make a very gross generalization, social science journals are generally double-blind, whereas natural science and mathematical science journals are generally single-blind. This observation is very relevant to this discussion, because the wiki-research-l community sits in between this divide. My perception is that 90% of people who post on this list are in information systems (double-blind), computer science (single-blind) or information science (either double- or single-blind).
If we can accept that single-blind journals can be considered as high-quality as well, then I feel a journal with wikified peer review would do much better being single-blind, especially if its subject matter is wiki-related topics. I understand that one of the primary reasons many journals decide against going double-blind is because of the ridiculous gymnastics that have to be undertaken in many cases to try to hide authors' identity. In computer science, where many researchers make their programs available on the Web, and much of their research concerns particular websites that they have developed, double-blinding would often be impossible if attempted--reviewers couldn't properly evaluate the work without knowing who created it. I think this is very much the case for a wiki-based peer review system for much wiki-related research.
Of course, the official reviewers should remain anonymous. (I know that open peer review has often been proposed--authors and reviewers know each others' identities--and has even been experimented with on several occasions, but it does not seem to be a quality improvement--I can post citations if anyone is interested.) It is much easier to hide the identity of the official reviewers than it is to hide that of the authors, and the benefits of single-blinding are substantial and proven.
~ Chitu
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
well, then I think I basically disagree on this one. I think that the fact that the authors CAN be identified after doing some more or less advanced research, does not mean that the reviewers are going to actively seek to break their anonymity (in fact, I'd assume this would be discouraged by most journal policies, and there are many traditional research projects where identifying the authors after some investigation is possible). Double-blind review is a process which is sustained and secured by good-faith participants (both the authors and the reviewers, too). Even if the authors can be guessed with some probability just from the references list, it does not mean that eliminating all elements of doubt serves a good purpose. I, for that matter, would rather avoid checking SSRN/Academia/wiki for the authors' names, to protect the rules of the game, and I would report that i may know the authors if I had known about their project from before hand.
best,
dj
On Tue, Nov 6, 2012 at 3:48 PM, Chitu Okoli Chitu.Okoli@concordia.cawrote:
Here are a few scenarios:
- The research topic concerns a public website. The website identifies the
authors. The paper makes no sense without explicitly identifying the website. Thus, authors should be able to request single-blind review. Note that this scenario very much applies to this entire discussion of a new research journal that uses wiki-based research development. I don't know if you caught Kerry Raymond's comment on this thread (I copy it below), which explains this point very succinctly.
- Authors have posted a working paper which has been on the web for a long
time, and is known to most researchers in that field of interest (i.e. most potential and qualified reviewers for the peer-reviewed version). In this case, I would think that reviewers should not be excluded for no reason other than they know the authors' identity. One of the most backward policies I've ever seen related to this is JIBS's policy to protect double-blind review: "Authors should also not post their submitted manuscript (including working papers and prior drafts) on websites where it could be easily discovered by potential reviewers." [1] Apparently, they consider double-blind review a more sacred ideal than early dissemination of research through working papers.
- The research critically involves a multimedia artifact, such as a video,
that cannot be easily be submitted as supporting materials for peer review. The video is better posted on a website. Here's a case of requested "gymnastics" I've seen in order to protect double-blind peer review even in such cases: "We ask each author to create his/her own account with an open access provider of choice (e.g., linked video could be hosted in Vimeo or YouTube). Please use a pseudo user name in order to maintain anonymity during the review process." [2]
Although I do believe in the benefits of double-blind review (I'll send a separate post with a few citations), in my own research I am increasingly confronted with the fact that new approaches to research that favour openness and mass collaboration are fundamentally in conflict with the idea of anonymity in the identity of the authors of a manuscript submitted for peer review. Personally, I prefer to forge ahead with innovative modes of research conduct, even if double-blind review is sacrificed. For me, a perfect compromise is to default to double-blind, but fall back to single-blind when the nature of the research project calls for it.
~ Chitu
[1] http://www.palgrave-journals.com/jibs/author_instructions.html#Ethical-guide... [2] http://icis2011.aisnet.org/Paper_Submission.html#B
Dariusz Jemielniak a écrit :
just out of curiosity, what could be the reasonable expected purposes for requesting a single-blind review instead of a standard double-blind in your model?
best,
dj
On Tue, Nov 6, 2012 at 3:56 AM, Chitu Okoli Chitu.Okoli@concordia.cawrote:
Actually, I think it is more reasonable to use double-blind by default unless authors request single-blind. If single-blind were the default, it would be difficult to request double-blind as exceptions:
- If there is a "big name" researcher who wants to take advantage of
his/her reputation, he/she would not request double-blind.
- If there is a "big name" researcher who is modest and does not think
highly of himself/herself, he/she would not request double-blind.
- If there is a minority or woman researcher afraid of discrimination, if
he/she requested double-blind, the reviewers would reasonably guess that the author(s) are minorities or women.
Thus, I think double-blind as a default for everyone with single-blind as special exception would be the more practical and fairer general policy. With the increase of preprints and working papers (e.g. arXiv and SSRN), I think author anonymity is becoming increasingly impractical.
In any case, these comments mainly apply to social science journals; I still think that single-blind makes more sense for computer science journals.
~ Chitu
Kerry Raymond a écrit :
I would note that the use of 1) would render double-blind irrelevant in 2). We would all know ...
On 06/11/2012, at 6:05 AM, "Kerry Raymond" kerry.raymond@gmail.com kerry.raymond@gmail.com wrote:
I think two things can be done in parallel.
- Allow folks to create descriptions of research in progress on the wiki,
which can be progressively updated. This enables others to make suggestions on methodology, give feedback on drafts of papers and so forth. Open and collaborative and experimental in the meta-sense. Clearly many on this list desire to experiment with new ways of working.
- Have a more formal traditional review process, so that the journal mets
the criteria for "reputable" that is important for people's CVs, tenure, promotion and so forth. As much as many of this don't like this way of working, it is the reality for earning your salary.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
I cannot speak for other disciplines but double-blind is not standard in Computer Science.
Kerry
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com wrote:
I cannot speak for other disciplines but double-blind is not standard in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com wrote:
I cannot speak for other disciplines but double-blind is not standard in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
agree, ... so it is up to you as a reviewer what you do with your "blindness" :-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they include references to their previous publications to build the new hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in many cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool of researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com wrote:
I cannot speak for other disciplines but double-blind is not standard in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your "blindness" :-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they
include
references to their previous publications to build the new hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in
many
cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com
wrote:
I cannot speak for other disciplines but double-blind is not standard in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
well, any attempts at pressures or bribes could easily be made known, couldn't they?
On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors. In my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your "blindness" :-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they
include
references to their previous publications to build the new hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as in
many
cases you can, as a reviewer, identify the author from the style and argument. This tends to get worse in specialised areas, where the pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com
wrote:
I cannot speak for other disciplines but double-blind is not standard in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
I don't agree. I a hard argument can be considered by some people as a preasure, while other could not.
In fact, what's the gain in knowing who is reviewing a paper?
2012/11/8 koltzenburg@w4w.net
well, any attempts at pressures or bribes could easily be made known, couldn't they?
On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors.
In
my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your "blindness"
:-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they
include
references to their previous publications to build the new
hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as
in
many
cases you can, as a reviewer, identify the author from the style
and
argument. This tends to get worse in specialised areas, where the
pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com
wrote:
I cannot speak for other disciplines but double-blind is not
standard
in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le
pido que
elimine este correo, evitando cualquier divulgación, copia o
distribución
de su contenido, así como desarrollar o ejecutar cualquier acción
basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and
objective.
In case you are not the addressee of the same, I apologize. And I
ask you
to delete this mail, and not to resend, copy or distribute its
content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Manuel asks:
In fact, what's the gain in knowing who is reviewing a paper?
let us look at this from another angle, maybe: As reviewers in open reviewing we get a chance of becoming more aware of our own inclinations in the face of public visibility vis-a-vis objectivity, well-reflected arguments and more transparency in general.
Q: Why should authors of research have to bow to any authority that is hiding its identity and tendencies?
actually, so far I have heard no convincing arguments why in the age of open Wikis any reviewer's identity should stay behind closed doors. Maybe you have another argument to convince me?
Claudia
On Thu, 8 Nov 2012 10:29:25 +0100, Manuel Palomo Duarte wrote
I don't agree. I a hard argument can be considered by some people as a preasure, while other could not.
In fact, what's the gain in knowing who is reviewing a paper?
2012/11/8 koltzenburg@w4w.net
well, any attempts at pressures or bribes could easily be made known, couldn't they?
On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors.
In
my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your "blindness"
:-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they
include
references to their previous publications to build the new
hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as
in
many
cases you can, as a reviewer, identify the author from the style
and
argument. This tends to get worse in specialised areas, where the
pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com
wrote:
> I cannot speak for other disciplines but double-blind is not
standard
> in Computer Science. **** > > ** ** > > Kerry**** > > ** ** > > ** ** > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le
pido que
elimine este correo, evitando cualquier divulgación, copia o
distribución
de su contenido, así como desarrollar o ejecutar cualquier acción
basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and
objective.
In case you are not the addressee of the same, I apologize. And I
ask you
to delete this mail, and not to resend, copy or distribute its
content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede contener información confidencial, dirigida a un destinatario y objetivo específico. Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada en el mismo. -- Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
I guess the scenario you want to protect against is this.
Reviewer is Junior Researcher, the author is a Head of School. Next year Junior Researcher applies for job at that school and doesn't get it or applies for a grant or ....
Kerry
-----Original Message----- From: wiki-research-l-bounces@lists.wikimedia.org [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of koltzenburg@w4w.net Sent: Thursday, 8 November 2012 7:41 PM To: Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] Wiki Research Journal? - Double-blind vs.single-blind review
Manuel asks:
In fact, what's the gain in knowing who is reviewing a paper?
let us look at this from another angle, maybe: As reviewers in open reviewing we get a chance of becoming more aware of our own inclinations in the face of public visibility vis-a-vis objectivity, well-reflected arguments and more transparency in general.
Q: Why should authors of research have to bow to any authority that is hiding its identity and tendencies?
actually, so far I have heard no convincing arguments why in the age of open Wikis any reviewer's identity should stay behind closed doors. Maybe you have another argument to convince me?
Claudia
On Thu, 8 Nov 2012 10:29:25 +0100, Manuel Palomo Duarte wrote
I don't agree. I a hard argument can be considered by some people as a preasure, while other could not.
In fact, what's the gain in knowing who is reviewing a paper?
2012/11/8 koltzenburg@w4w.net
well, any attempts at pressures or bribes could easily be made known, couldn't they?
On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote
I don't thnk opening peer reviewing would be a good idea. Reviewer
must
keep unknown, or she could suffer preasures (even bribes) from
authors.
In
my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your
"blindness"
:-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually
they
include
references to their previous publications to build the new
hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double
blind,
although the effectiveness of it was always a bit questionable,
as
in
many
cases you can, as a reviewer, identify the author from the style
and
argument. This tends to get worse in specialised areas, where
the
pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond
wrote:
> I cannot speak for other disciplines but double-blind is not
standard
> in Computer Science. **** > > ** ** > > Kerry**** > > ** ** > > ** ** > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le
pido que
elimine este correo, evitando cualquier divulgación, copia o
distribución
de su contenido, así como desarrollar o ejecutar cualquier acción
basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and
objective.
In case you are not the addressee of the same, I apologize. And I
ask you
to delete this mail, and not to resend, copy or distribute its
content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido
que
elimine este correo, evitando cualquier divulgación, copia o
distribución
de su contenido, así como desarrollar o ejecutar cualquier acción
basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and
objective.
In case you are not the addressee of the same, I apologize. And I ask
you
to delete this mail, and not to resend, copy or distribute its
content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content, as well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
no. Also, academic world may be quite small in some disciplines. If a reviewer knows that s/he may be evaluated by the author some time in the future (be it in a journal review, or possibly also in career promotion reviews, too) s/he will be definitely motivated not to report any major flaws, especially if the reviewed author is a big shot.
Single blind review is a must. In my view, the advantages of double blind review are also important, even if practically the actual anonymity may not always be preserved. In double blind review the bottom line is that it sometimes may and up as a single blind review, but in many cases it does not and then it is more fair.
best,
dj
On Thu, Nov 8, 2012 at 10:08 AM, koltzenburg@w4w.net wrote:
well, any attempts at pressures or bribes could easily be made known, couldn't they?
On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors.
In
my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your "blindness"
:-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they
include
references to their previous publications to build the new
hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as
in
many
cases you can, as a reviewer, identify the author from the style
and
argument. This tends to get worse in specialised areas, where the
pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com
wrote:
I cannot speak for other disciplines but double-blind is not
standard
in Computer Science. ****
Kerry****
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le
pido que
elimine este correo, evitando cualquier divulgación, copia o
distribución
de su contenido, así como desarrollar o ejecutar cualquier acción
basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and
objective.
In case you are not the addressee of the same, I apologize. And I
ask you
to delete this mail, and not to resend, copy or distribute its
content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
hm, sadly enough I must agree that you seem to be raising important real-life points, Dariusz. But am I getting you correctly that you think that major flaws will only be pointed out in a review if the reviewer can officially stay anonymous?
in your experience, Dariusz, does this mean reviewers feel fine in placing tons of trust in the editors and their helphands who organize the review not to tell authors who was their most brilliant reviewer?
anyway, I think we need a reviewing system where fair and open criticism can flourish.
in my view of the matter, there will be no one-size-fits-all because self-organized communities do have a multicultural tendency to self-organize :-)
one example system that we night also discuss and try out for the Wiki Research Journal is a combination of open and closed peer review: see ACP who, in a highly specialised community, do 8 weeks of post publication public discussion http://www.atmospheric-chemistry-and- physics.net/review/review_process_and_interactive_public_discussion.html
let me suggest we now go to the wiki page http://meta.wikimedia.org/wiki/Wiki_Research_Ideas and continue our debate on the peer review model in a pro/contra/undecided style there
and keep in mind that we are not talking about a traditional journal here but about "a new research journal about Wikis and about research done by using Wikis" http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Volunteers
see you, Claudia koltzenburg@w4w.net
On Thu, 8 Nov 2012 10:44:13 +0100, Dariusz Jemielniak wrote
no. Also, academic world may be quite small in some disciplines. If a reviewer knows that s/he may be evaluated by the author some time in the future (be it in a journal review, or possibly also in career promotion reviews, too) s/he will be definitely motivated not to report any major flaws, especially if the reviewed author is a big shot.
Single blind review is a must. In my view, the advantages of double blind review are also important, even if practically the actual anonymity may not always be preserved. In double blind review the bottom line is that it sometimes may and up as a single blind review, but in many cases it does not and then it is more fair.
best,
dj
On Thu, Nov 8, 2012 at 10:08 AM, koltzenburg@w4w.net wrote:
well, any attempts at pressures or bribes could easily be made known, couldn't they?
On Thu, 8 Nov 2012 09:56:35 +0100, Manuel Palomo Duarte wrote
I don't thnk opening peer reviewing would be a good idea. Reviewer must keep unknown, or she could suffer preasures (even bribes) from authors.
In
my opinion only the editor must communicate with the reviewers
2012/11/8 koltzenburg@w4w.net
agree, ... so it is up to you as a reviewer what you do with your "blindness"
:-)
doesn't this point in the direction of - plus - is + ? I mean: why not do open peer reviewing?
Claudia
On Thu, 8 Nov 2012 09:43:39 +0100, Manuel Palomo Duarte wrote
Even more, you can easily identify the authors because usually they
include
references to their previous publications to build the new
hypothesis ...
2012/11/8 Adam Jenkins adam.jenkins@gmail.com
Most of my reviewing for conference and journals was double blind, although the effectiveness of it was always a bit questionable, as
in
many
cases you can, as a reviewer, identify the author from the style
and
argument. This tends to get worse in specialised areas, where the
pool
of
researchers is necessarily small.
Adam
On 7 November 2012 06:16, Kerry Raymond kerry.raymond@gmail.com
wrote:
> I cannot speak for other disciplines but double-blind is not
standard
> in Computer Science. **** > > ** ** > > Kerry**** > > ** ** > > ** ** > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > >
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le
pido que
elimine este correo, evitando cualquier divulgación, copia o
distribución
de su contenido, así como desarrollar o ejecutar cualquier acción
basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and
objective.
In case you are not the addressee of the same, I apologize. And I
ask you
to delete this mail, and not to resend, copy or distribute its
content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- Prof. Manuel Palomo Duarte, PhD Software Process Improvement and Formal Methods group (SPI&FM). Degree Coordinator for Computer Science. Department of Computer Science. Escuela Superior de Ingenieria. C/ Chile, 1 11002 - Cadiz (Spain) University of Cadiz http://neptuno.uca.es/~mpalomo Tlf: (+34) 956 015483 Mobile phone: (+34) 649 280080 Mobile phone from University network: 45483 Fax: (+34) 956 015139
Aviso legal: Este mensaje (incluyendo los ficheros adjuntos) puede
contener
información confidencial, dirigida a un destinatario y objetivo
específico.
Si usted no es el destinatario del mismo le pido disculpas, y le pido que elimine este correo, evitando cualquier divulgación, copia o distribución de su contenido, así como desarrollar o ejecutar cualquier acción basada
en
el mismo.
Legal Notice: This message (including the attached files) contains confidential information, directed to a specific addressee and objective. In case you are not the addressee of the same, I apologize. And I ask you to delete this mail, and not to resend, copy or distribute its content,
as
well as develop or execute any action based on the same.
thanks & cheers, Claudia koltzenburg@w4w.net
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
dr hab. Dariusz Jemielniak profesor zarządzania kierownik katedry Zarządzania Międzynarodowego i centrum badawczego CROW Akademia Leona Koźmińskiego http://www.crow.alk.edu.pl
hi,
On Thu, Nov 8, 2012 at 11:17 AM, koltzenburg@w4w.net wrote:
hm, sadly enough I must agree that you seem to be raising important real-life points, Dariusz. But am I getting you correctly that you think that major flaws will only be pointed out in a review if the reviewer can officially stay anonymous?
well, not only officially, but also practically. It is an important ethical responsibility of the managing editor to ensure anonymity of the reviewers, so that they can be honest.
in your experience, Dariusz, does this mean reviewers feel fine in placing tons of trust in the editors and their helphands who organize the review not to tell authors who was their most brilliant reviewer?
Yes, that is my experience. In fact, I have never seen the editor revealing the reviewer's identity. I have heard of one such instance, when the author discovered the reviewer's identity simply because of the high praises the reviewer was making for his work, and because of pushing the reviewer's works as suggested needed literature, but the only reaction from the editor was matter-of-factly allowing not to incorporate these suggestions from the review and excluding the reviewer from further process.
and keep in mind that we are not talking about a traditional journal here but about
"a new research journal about Wikis and about research done by using Wikis"
that's true and experimenting with the format is a good idea! I think, for example, that publishing reviews and responses to them, and allowing commenting on them is a good idea. I'm quite convinced though that the anonymity of reviewers helps. Of course, it can be probably be also played with and tested.
best,
dj
On Thu, Nov 8, 2012 at 10:26 AM, Dariusz Jemielniak darekj@alk.edu.plwrote:
in your experience, Dariusz, does this mean reviewers feel fine in placing
tons of trust in the editors and their helphands who organize the review not to tell authors who was their most brilliant reviewer?
Yes, that is my experience. In fact, I have never seen the editor revealing the reviewer's identity.
I'll note that the discussions in this thread remain "abstract" insofar as they focus on journals and editing procedures in general -- not on the specific affordances and workflows associated with wikis, or the specific needs and interests of wikistas. This isn't to say that such discussions are unimportant -- but in order to be make them somewhat more exciting (!) I think it would be good to put aside the frame of "previous experience" and return to the question:
what are we aiming for exactly?
(Because: I think it's actually something very new.)
If the question is only "how to set up a journal" then I wonder if this should be taking place off-list, since that's not really a "wiki research" question. If it is a question about "how to set up a journal that specifically meshes with the socio-technical patterns used by wiki communities", then of course it is appropriate for discussion here. (And obviously I think it's the latter!)
This point from Claudia is important -- «keep in mind that we are not talking about a traditional journal here but about "a new research journal about Wikis and about research done by using Wikis"» -- however, I think it needs expansion, or we'll just end up with some kind of turn-crank solution.
To reframe that:
What's NOT going to be traditional about this journal?
If the question is only "how to set up a journal" then I wonder if this should be taking place off-list, since that's not really a "wiki research" question. If it is a question about "how to set up a journal that specifically meshes with the socio-technical patterns used by wiki communities", then of course it is appropriate for discussion here. (And obviously I think it's the latter!)
the question may also be "how to set up a journal relevant for research specific for wiki-communities, that stands a chance of becoming the leading journal (ranked, listed, prestigious, etc.) in some related fields" and then questions on which traditional academic practices of a journal are necessary, and which are optional, obviously is both important, and of interest for this list. Just saying.
dj
On Thu, Nov 8, 2012 at 10:59 AM, Dariusz Jemielniak darekj@alk.edu.plwrote:
If the question is only "how to set up a journal" then I wonder if this
should be taking place off-list, since that's not really a "wiki research" question. If it is a question about "how to set up a journal that specifically meshes with the socio-technical patterns used by wiki communities", then of course it is appropriate for discussion here. (And obviously I think it's the latter!)
the question may also be "how to set up a journal relevant for research specific for wiki-communities, that stands a chance of becoming the leading journal (ranked, listed, prestigious, etc.) in some related fields" and then questions on which traditional academic practices of a journal are necessary, and which are optional, obviously is both important, and of interest for this list. Just saying.
I don't disagree, but I think the main driving question should be "what's interesting here?". What it says on the wiki page presently is:
«The field of "wiki studies" exists but there is no dedicated journal. This is a problem to be solved.»
So, is the question just a relatively procedural one? (How to create a journal for "Wiki Studies" and edit and populate it using relatively "traditional" means?) I'm certainly not worried about the existence of a "procedural" phase.
My thought is, rather, that at this early stage, there may be some other not so procedural questions, and we shouldn't get lost in the details of process before we've understood what those are.
thanks, Joe, for opening a new wiki page for the peer review model debate
from http://meta.wikimedia.org/wiki/Wiki_Research_Ideas to http://meta.wikimedia.org/wiki/Wiki_Research_Ideas/Review_model
Claudia koltzenburg@w4w.net
wiki-research-l@lists.wikimedia.org