On 3/22/06, SJ <2.718281828(a)gmail.com> wrote:
Those reports were wrong, however, because
Nature's research was invalid. As
our editors and scholarly advisers have discovered by reviewing the research
in depth, almost everything about the Nature's investigation was wrong and
misleading. Dozens of inaccuracies attributed to the Britannica were not
inaccuracies at all, and a number of the articles Nature examined were not
even in the Encyclopædia Britannica. The study was so poorly carried out and
its findings so error-laden that it was completely without merit.
I'm surprised how vague this is. Later he complains that the source
data is not available. But here he just refers to "dozens of
inaccuracies"...hmm.
* Rearranging, reediting, and excerpting
Britannica articles. Several
of the "articles" Nature sent its outside reviewers were only sections of,
or excerpts from Britannica entries. Some were cut and pasted together from
more than one Britannica article. As a result, Britannica's coverage of
certain subjects was represented in the study by texts that our editors
never created, approved or even saw.
That seems like a lot of hot air over a small issue. Claiming that
their editors "never created, approved or even saw" the particular
compilation is really piling on strong words to make an impact.
Whereas what they're actually denying is much weaker. Kind of like "I
did *not* have sexual relations with that woman..."
* Mistakenly identifying inaccuracies. The
journal claimed to have
found dozens of inaccuracies in Britannica that didn't exist.
Well, I've had a look at some of their claimed inaccurate inaccuracies
and I'm not that sure. It's very much "No it's not / Yes it is".
* Reviewing the wrong texts. They reviewed a
number of texts that were
not even in the encyclopedia.
Do they get more specific than this? "A number" probably turns out to
be 3. Or existed in some previous version of the encyclopaedia that
was outdated 4 days before the survey. I'm suspicious.
* Failing to check facts. Nature falsely
attributed inaccuracies to
Britannica based on statements from its reviewers that were themselves
inaccurate and which Nature's editors failed to verify.
Well, details, please.
* Misrepresenting its findings. Even according
to Nature's own
figures, (which grossly exaggerated the number of inaccuracies in
Britannica) Wikipedia had a third more inaccuracies than Britannica. Yet the
headline of the journal's report concealed this fact and implied something
very different.
I vaguely recall they had some sort of weird logarithmic scale, so I
would tend to agree with this one. But actually...wasn't the final
number something like "4 errors per article compared to 3 errors per
article"? Whether 33% more is a lot or not is purely subjective. I
would definitely say it's the same ballpark.
Britannica also made repeated attempts to obtain from
Nature the original
data on which the study's conclusions were based. We invited Nature's
editors and management to meet with us to discuss our analysis, but they
declined.
If true, that's very poor on Nature's part.
Whatever may have prompted Nature to do such careless
and sloppy research,
it's now time for them to uphold their commitment to good science and
retract the study immediately. We have urged them strongly to do so.
Uh, a commitment to good science would be publishing the data and
their method so that people can attempt to reproduce it. I tend to
agree that Nature's method was optimistic. Surely more than one
reviewer for each article shoul have been appointed...even better
would have been an open forum where many experts could pick through
each article and argue amongst themselves.
We have prepared a detailed report that describes
Britannica's thorough
(7,000 words) analysis of the Nature study. I invite you to download it from
our Web site at
www.eb.com.
A lot of it seems to be "We reject this criticism. We have asked our
adviser, and he says we're right." Not exactly convincing stuff.
Particularly when they complain that the original study failed to cite
sources - and with few exceptions, the EB rebuttal doesn't either.
Example: "Britannica response: We do not accept these criticisms
(which are really just one criticism, not two). We have published a
revision of this article that retains the emphasis on supersaturation
rather than the transitional stage of saturation."
By this standard, Wikipedia should reply to nature and say "Thanks for
the criticisms. Our community rejects them. We stand by our original
version". However, we didn't - as I understand, we actually took the
criticisms on board and worked with them. Which kind of demonstrates
the real strength in Wikipedia. Instead of simply "not accepting"
every criticism (as EB does 22 times in their response) to protect our
good name, we, without ego, simply make it better.
I also sort of feel that as an attempted model of good science or
whatever, EB should not be attacking the entire Nature study as
totally "without merit".
Sorry I can't join the IRC chat.
Steve