On 04/19/2010 10:46 AM, Nathan wrote:
I wonder if there might be a subtle bias playing into
these reviews.
Perhaps if reviewers begin with the assumption that the article was
written by amateur hobbyists, that influences the outcome. If Lindsey
went back to them and let them know that the articles had been written
or comprehensively reviewed by recognized experts, would that alter
the results?
It's an interesting question, but I think it might influence their
description more than their actual opinion, i.e. that if they knew it
was written by a PhD in their field, they would phrase their
disagreement differently, but might still not like the article. Some of
these comments are almost exactly the comments a survey article will
typically get in peer review! Almost nobody likes the survey article
that someone else has written: it invariably over-emphasizes unimportant
issues, under-emphasizes the key issues, is missing important results in
the field, includes results of questionable reliability, etc.
(Happens with textbooks, too; almost everyone has a gripe about how the
standard textbook in their field misrepresents things.)
-Mark