On Friday 16 July 2004 20:24, Magnus Manske wrote:
> Meanwhile, the tone of the press concerning wikipedia seems to
> be slowly drifting from "interesting, lots of potential" to "source of
> unreliable information".
[...]
> It seems to me that most of you would agree to a method similar to this:
> * A (logged-in) user can approve a single version of an article.
> * At least two approvals are needed for the "wikipedia seal of approval"
> :-)
Before discussing an approval method we should agree on what the approval is
about. Is it about spelling/grammer correctness, factually accurateness,
license problems, neutrality, ... or all of the above?
What we need for Wikipedia is a complete review in terms of quality similar to
the traditional peer review system. I write similar since we already know
that a traditional peer review system will not work for Wikipedia--too little
incentive, too much work for a single person, basically it comes down to the
same reasons why Nupedia has failed.
A complete review of an article needs all the things described in the
beginning:
1. style - this includes grammer, well-written, image captions, ...
2. legal - in particular check status of images
3. completeness - does the article contain everything "important" about the
subject
4. fact checking - validate every information in the article
For competing against Britannica 1, 3 and 4 are important. No.2 is important
for avoiding legal mess a la SCO.
A simple yes/no approval mechanism is IMHO not the right way in order to
achieve our goals. I don't think that anyone will fact-check an article of
>30.000 characters, this would literally take days of work if done properly.
I even seriously doubt that everyone would read the whole article before
approving. Erik Möller once wrote:
"My experience on FAC [featured articles candidates] indicates that many
people have not even read the articles they support (no surprise, many of
them are 40,000 characters and longer)."
Not to mention other problems:
* everyone can create an arbitrary number of accounts
* we do not know anything about the readers expertise on the field
* approving an article again and again after every (small) edit will become a
pain
The last point IMHO is important. Every review method should be chosen in such
a way that the additional work after several smaller changes have occurred is
small. In particular one (trusted) user should be able to retransfer the
results from a "stable" version to a more recent version.
I am confident that we can solve no.1 and no.2 and perhaps no.3 within the
exiting wiki framework.
[[Wikipedia:Featured article candidates]] is already a good base for no.1 and
it seems to work reasonable well. And btw I never have heard any complaints
in press that grammer and/or writing style of the articles is bad.
In order to avoid legal problems in the long run, in particular after having
printed 1.0, we should at least have someone who is familiar with law and
GFDL who looks through the article. In most cases it probably would be
sufficient to have a closer look at the images. Here it would be helpful to
have a list of trusted "experts" that could help, similar to
[[Wikipedia:Association_of_Members'_Advocates]].
No.3 can become more difficult if the topic is very special. E.g. for the
article [[quantum field theory]] only a small number of our users have the
knowledge to judge if everything important is included.
No.4 IMHO is the most difficult one and concerning the "reliability" aspect
the most important one.
Checking every single statment can't be done by one user (it is too much
work). Thus a simple yes/no for the whole article cannot be the solution. The
hard point IMHO is the following:
* we either check the article in a very short time frame. Then we need an
"expert committee" which can validates the material. This has also has been
discussed many times, the last time
[[meta:Article_validation#Proposition_of_validation_by_a_committee]]. Reading
the proposition reminds me of Nupedia that's why I don't think this will
work.
* the other way is that many person independently validate information in a
longer time frame. The unsolved problem about this ansatz is how the system
remembers which parts already have been validated and which haven't.
(The discussion page won't be sufficient: if we know that A checked section
1,2 of version 125, B checked section 2,3 of version 134 and C of 1,4 of
version 145, it is hard to find out if really everything was validated
because of the edits in between. And this is only an example with three
reviewers where everyone has checked whole paragraphs.)
The edits between reviews make this hard. I have some ideas I'm playing around
with in order to overcome the problem. This is done by allowing users to
validate single sentences and by that allowing the underlying system to tell
for every single sentence how often it was validated. (unfortunately it is
not that easy, but this is another story)
O.k. just my 2 cents, why I think that a simple yes/no unfortunately is not
sufficient for an equivalent of peer review. It's late, therefore I apologize
for any extra grammer / spelling mistakes ;-)
best regards,
Marco