On 3/22/06, SJ 2.718281828@gmail.com wrote:
Those reports were wrong, however, because Nature's research was invalid. As our editors and scholarly advisers have discovered by reviewing the research in depth, almost everything about the Nature's investigation was wrong and misleading. Dozens of inaccuracies attributed to the Britannica were not inaccuracies at all, and a number of the articles Nature examined were not even in the Encyclopædia Britannica. The study was so poorly carried out and its findings so error-laden that it was completely without merit.
I'm surprised how vague this is. Later he complains that the source data is not available. But here he just refers to "dozens of inaccuracies"...hmm.
Rearranging, reediting, and excerpting Britannica articles. Severalof the "articles" Nature sent its outside reviewers were only sections of, or excerpts from Britannica entries. Some were cut and pasted together from more than one Britannica article. As a result, Britannica's coverage of certain subjects was represented in the study by texts that our editors never created, approved or even saw.
That seems like a lot of hot air over a small issue. Claiming that their editors "never created, approved or even saw" the particular compilation is really piling on strong words to make an impact. Whereas what they're actually denying is much weaker. Kind of like "I did *not* have sexual relations with that woman..."
Mistakenly identifying inaccuracies. The journal claimed to havefound dozens of inaccuracies in Britannica that didn't exist.
Well, I've had a look at some of their claimed inaccurate inaccuracies and I'm not that sure. It's very much "No it's not / Yes it is".
Reviewing the wrong texts. They reviewed a number of texts that werenot even in the encyclopedia.
Do they get more specific than this? "A number" probably turns out to be 3. Or existed in some previous version of the encyclopaedia that was outdated 4 days before the survey. I'm suspicious.
Failing to check facts. Nature falsely attributed inaccuracies toBritannica based on statements from its reviewers that were themselves inaccurate and which Nature's editors failed to verify.
Well, details, please.
Misrepresenting its findings. Even according to Nature's ownfigures, (which grossly exaggerated the number of inaccuracies in Britannica) Wikipedia had a third more inaccuracies than Britannica. Yet the headline of the journal's report concealed this fact and implied something very different.
I vaguely recall they had some sort of weird logarithmic scale, so I would tend to agree with this one. But actually...wasn't the final number something like "4 errors per article compared to 3 errors per article"? Whether 33% more is a lot or not is purely subjective. I would definitely say it's the same ballpark.
Britannica also made repeated attempts to obtain from Nature the original data on which the study's conclusions were based. We invited Nature's editors and management to meet with us to discuss our analysis, but they declined.
If true, that's very poor on Nature's part.
Whatever may have prompted Nature to do such careless and sloppy research, it's now time for them to uphold their commitment to good science and retract the study immediately. We have urged them strongly to do so.
Uh, a commitment to good science would be publishing the data and their method so that people can attempt to reproduce it. I tend to agree that Nature's method was optimistic. Surely more than one reviewer for each article shoul have been appointed...even better would have been an open forum where many experts could pick through each article and argue amongst themselves.
We have prepared a detailed report that describes Britannica's thorough (7,000 words) analysis of the Nature study. I invite you to download it from our Web site at www.eb.com.
A lot of it seems to be "We reject this criticism. We have asked our adviser, and he says we're right." Not exactly convincing stuff. Particularly when they complain that the original study failed to cite sources - and with few exceptions, the EB rebuttal doesn't either.
Example: "Britannica response: We do not accept these criticisms (which are really just one criticism, not two). We have published a revision of this article that retains the emphasis on supersaturation rather than the transitional stage of saturation."
By this standard, Wikipedia should reply to nature and say "Thanks for the criticisms. Our community rejects them. We stand by our original version". However, we didn't - as I understand, we actually took the criticisms on board and worked with them. Which kind of demonstrates the real strength in Wikipedia. Instead of simply "not accepting" every criticism (as EB does 22 times in their response) to protect our good name, we, without ego, simply make it better.
I also sort of feel that as an attempted model of good science or whatever, EB should not be attacking the entire Nature study as totally "without merit".
Sorry I can't join the IRC chat.
Steve