Delirium wrote:
Stan Shebs wrote:
Actually, it's sort of interesting that this is news. While it's common practice to have peer review for scholarly books, the "peer" part of the term should be a hint that it's not the publisher doing the checking! My guess is that our reverence for the printed page is such that we just assume no one would dare to print without being certain of its correctness.
In many fields the peer-review of even scholarly books is not all that high. In the sciences, journal articles hold much more weight than books, because there's a perception that anybody can get a book published.
I'm not so sure that the peer review of journal articles is necessarily that much better. I recently dealt with a situation in which the submitter of an article was able to specifically request that his article not be forwarded to one of the logical candidates for peer-reviewing it, because he anticipated that this person would give an unfavorable review. Cherry-picking your reviews hardly counts as rigorous scholarship in my book.
The theory of peer review is nice, but even in academia the execution is often shoddy and politically skewed. Practices at different journals vary, of course, so the reputation of the journal needs to be considered beyond just the question of whether it qualifies as peer-reviewed.
--Michael Snow
Michael Snow wrote:
Delirium wrote:
Stan Shebs wrote:
Actually, it's sort of interesting that this is news. While it's common practice to have peer review for scholarly books, the "peer" part of the term should be a hint that it's not the publisher doing the checking! My guess is that our reverence for the printed page is such that we just assume no one would dare to print without being certain of its correctness.
In many fields the peer-review of even scholarly books is not all that high. In the sciences, journal articles hold much more weight than books, because there's a perception that anybody can get a book published.
I'm not so sure that the peer review of journal articles is necessarily that much better. I recently dealt with a situation in which the submitter of an article was able to specifically request that his article not be forwarded to one of the logical candidates for peer-reviewing it, because he anticipated that this person would give an unfavorable review. Cherry-picking your reviews hardly counts as rigorous scholarship in my book.
The theory of peer review is nice, but even in academia the execution is often shoddy and politically skewed. Practices at different journals vary, of course, so the reputation of the journal needs to be considered beyond just the question of whether it qualifies as peer-reviewed.
This past week the CBC has featured the story of Dr. Ranjit Chandra who faked results about multivitamin benefits for the elderly. He has since left the country, but seems to be doing well running a non-existent university in Switzerland, and selling his multivitamins in India. See http://www.cbc.ca/nl/story/nf-chandra-retraction-20050302.html http://www.cbc.ca/national/news/chandra/ http://www.sethroberts.net/ (bibliography) and http://www.sethroberts.net/articles/chandra/1992_Effect_of_Vitamin_and_Trace...
One quotation from the third part of the programme.
The former editor of the British Medical Journal, Richard Smith, says there's a good reason for that.
"Anybody who knows about peer review knows that sometimes it will pick up a fraudulent study, but it's by no means guaranteed to do so," Smith says. "And that's largely because it starts from a position of trust. So if somebody says there were 200 patients, then you assume there were 200 patients. You don't say, 'Well show me their records and show me their photographs, I need to see them.' The whole thing is based on trust."
The pharmaceutical industry in particular has a reputation for making its studies look good.
It is never safe to say that a peer-reviewed article is ipso-facto valid.
Ec
On 2/3/06, Ray Saintonge saintonge@telus.net qoutes:
"Anybody who knows about peer review knows that sometimes it will pick up a fraudulent study, but it's by no means guaranteed to do so," Smith says. "And that's largely because it starts from a position of trust. So if somebody says there were 200 patients, then you assume there were 200 patients. You don't say, 'Well show me their records and show me their photographs, I need to see them.' The whole thing is based on trust."
In other words, "assume good faith".
Ray Saintonge wrote:
Michael Snow wrote:
Delirium wrote:
Stan Shebs wrote:
Actually, it's sort of interesting that this is news. While it's common practice to have peer review for scholarly books, the "peer" part of the term should be a hint that it's not the publisher doing the checking! My guess is that our reverence for the printed page is such that we just assume no one would dare to print without being certain of its correctness.
In many fields the peer-review of even scholarly books is not all that high. In the sciences, journal articles hold much more weight than books, because there's a perception that anybody can get a book published.
I'm not so sure that the peer review of journal articles is necessarily that much better. I recently dealt with a situation in which the submitter of an article was able to specifically request that his article not be forwarded to one of the logical candidates for peer-reviewing it, because he anticipated that this person would give an unfavorable review. Cherry-picking your reviews hardly counts as rigorous scholarship in my book.
The theory of peer review is nice, but even in academia the execution is often shoddy and politically skewed. Practices at different journals vary, of course, so the reputation of the journal needs to be considered beyond just the question of whether it qualifies as peer-reviewed.
This past week the CBC has featured the story of Dr. Ranjit Chandra who faked results about multivitamin benefits for the elderly. He has since left the country, but seems to be doing well running a non-existent university in Switzerland, and selling his multivitamins in India.
<snip>
One quotation from the third part of the programme.
The former editor of the British Medical Journal, Richard Smith, says there's a good reason for that.
"Anybody who knows about peer review knows that sometimes it will pick up a fraudulent study, but it's by no means guaranteed to do so," Smith says. "And that's largely because it starts from a position of trust. So if somebody says there were 200 patients, then you assume there were 200 patients. You don't say, 'Well show me their records and show me their photographs, I need to see them.' The whole thing is based on trust."
The pharmaceutical industry in particular has a reputation for making its studies look good.
It is never safe to say that a peer-reviewed article is ipso-facto valid.
I don't suppose anyone remembers the name of the guy who used the same set of data over and over and over again to fake his results on nanotech research...