I have been preoccupied with other things and have not followed the
list-serve closely, so I apologize if I am missing something (and would
love it if someone would bring me up to speed). But I recently signed onto
a "Forum for Encyclopedic Standards" project, and now see people discussing
peer-review. I'd like to point out that "peer-review" can take many
different forms, and in fact has had different effects.
The form I am most familiar with is an article is sent to two to five
reviewers who comment on and evaluate it, and the editor of a journal makes
recommendations. In some cases, the reviewers put considerable time and
effort into commenting -- the result is something far beyond the casual
hashing out of ideas at a bar or cafe. In many of these cases I know I
have benefited considerably and my article is ultimately much, much better
for it. In other cases, reviewers put little time into the job and often
grind their ax (as some have pointed out) or just insist that the article
under review cite ten more people (surely, friends of the reviewer). In
such cases the result is that those articles that are published are often
mediocre, conform to some current fad, and all sound alike.
But surely, peer review means nothing more than making our words
accountable to the judgement of our peers. Broadly put, isn't this what
Wikipedia is all about, and haven't we been doing this all along?
I agree that there is an issue of concern here. I am currently involved in
an edit war with a couple of users and at the heart of it, I think they
have done no research or very sloppy research. I have read too many
articles where it seems that the primary contributors relied on their
memories of what they learned ten years ago in a college class, or what
they have picked up on other web-sites (and frankly, I think the
information available on the web is mostly mediocre). How do we deal with
this?
Given my own commitments to Wikipedia, I am absolutely against a panel of
people who are charged with approving or vetoing articles or edits.
I have two suggestions. First, I think that we already have established
values that address some of the problems I and others are concerned
with. I am referring specifically to the prohibition on original research,
and the call for verifiability. Most of the edit wars I have been involved
with have involved either NPOV or one of these two values. Frankly, I
think that the Wikipedia community as a whole has focused too much on NPOV
at the exclusion of these two other values. I do not mean this to be a
criticism. I have seen considerable (and necessary, useful) debate over
the past few years on what, exactly, NPOV is -- debate on talk pages,
meta-Wikipedia, and on this listserve. I think the community as a whole
has developed a pretty sophisticated idea of NPOV and how it can be
achieved. I think most veterans are very sensitive about this, and do a
pretty good job of policing articles for violations of NPOV, and educating
newbies. I just think we haven't done as good a job with
"verifiability." I think the explanation we have on the verifiability page
is very good -- but I am talking about the culture of the place. I think it
is time for people actively to look for opportunities -- on article talk
pages, personal pages, and here -- to talk about "verifiability;" how do
you recognize that an article is verifiable; what are the warning signs
that it may not be; how to go about making it more encyclopedic. I think
the culture of Wikipedia is strong and when directed in a thoughtful way, a
powerful tool. I think if the majority of us just started to pay more
attention to this issue on pages, and through debate and dialogue started
evolving a more practical way to explain it to newbies and show people how
to make articles more verifiable, we will not need a panel empowered to
veto articles.
My second suggestion is to try to find a way to appropriate the one good
thing I have found in peer-review -- when a few scholars who really are
experts in a topic give considerable attention to one person's work, and
give them focused feedback. NOTE, the purpose is not to give a stamp of
approval or to veto the work, the purpose is to engage the work on a very
high level, to call attention to the elements of the work that can be
developed, and to suggest alternatives. I am trying to imagine a process
that is more intense than what usually happens on a talk page. What I
suggest is that there be a directory of editors grouped by expertise on
major topics (biology, art, US history, etc). When it is clear that one
editor is writing a new article or substantially rewriting or adding to an
existing article, members of one of these panels can review the changes as
a whole with an eye towards making constructive suggestions and educating
the new editor in Wikipedia values. There need be no threat of veto or
sanction power -- I think if three veterans reviewed an article of mine and
told me "look, deal with our criticisms now, and address these issues, or
sometime over the next year other wikipedians will spot these problems and
delete or change your work at will," it will be strong enough incentive.
These two ideas are off the top of my head and I am sure that even if you
like them they need much more refinement. I hope there are enough other
people out there who get what I am suggesting, and are willing to improve
on the ideas!
Steve
Steven L. Rubenstein
Associate Professor
Department of Sociology and Anthropology
Bentley Annex
Ohio University
Athens, Ohio 45701