On Mon, Dec 1, 2008 at 2:00 PM, Delirium delirium@hackish.org wrote:
phoebe ayers wrote:
Of course, what's interesting and troubling for us is that this is a respected publisher who apparently did all the normal things in setting up an academic journal that is typical of the sort of thing Wikipedia is supposed to use as a "reliable source." But (naturally, I suppose) the academic publishing process is as open to failure as any other publishing or reporting process.* And I can't help but think that in a more open process -- an open access journal, say, or even Wikipedia -- this would not have gone on for so long or played out in the same way.
True, though I think the biggest (and long-standing) problem has actually been books, which in many fields (especially in the humanities) are both the canonical "reliable source", and hugely problematic as sources. Academic presses have a peer-review process, but it isn't intended to make sure the book is representative of consensus in the field, unbiased, or otherwise a good source for writing an encyclopedia article. It's more of a minimal level of reviewing to ensure that the author is making a legitimate contribution to the academic debate, not plagiarizing anyone, etc.---even if the result is a highly polemical book contrary to consensus and accepted by nearly nobody, it may be worth publishing as a contribution to the overall discussion, especially if the author is already well known.
Sure, and maybe this isn't even a problem per se -- it's the job of scholarly discourse to present and discuss new ideas, etc. etc. I am thinking more about a failure of scientific publishing as meaning a (theoretically) respectable journal published by a (theoretically) respectable publisher shouldn't really be an unquestioned soapbox for one guy who may or may not be writing patent nonsense.
Same goes for a Wikipedia article, naturally, and this is something we have certainly been fighting against in obscure topics for years... I seem to recall this even led to a to a policy called "No Original Research" once upon a time. Now of course our problems have shifted more into what materials we cite, though, and why we cite them.
This is all fine if books are read with full knowledge of their status in the field---that they represent the possibly idiosyncratic view of one particular writer. But if their claims are then entered into Wikipedia articles, with a citation to the book to justify them, that's more of a problem. This isn't as rare as people might think either; I'd say the *majority* of academic-press books make at least one significant claim that is controversial in its field, often without even admitting that the claim is controversial.
Maybe we need to put more emphasis on "encyclopedia as a tertiary source" -- let other people do the summarizing and the vetting and sorting out of what ideas are going to stick around for the long-term, and focus away from citing original research directly, which helps side-step the danger of representing obscure or untested theory as canonical truth. This might be particularly be true for new scientific discoveries or new ideas in the humanities. (Different perhaps for events in the news, articles about pop culture, etc).
-- phoebe