[Wikipedia-l] Waiting for 1.0

[Marco Krohn]

On Friday 16 July 2004 20:24, Magnus Manske wrote:
> Meanwhile, the tone of the press concerning wikipedia seems to
> be slowly drifting from "interesting, lots of potential" to "source of
> unreliable information".
[...]

> It seems to me that most of you would agree to a method similar to this:
> * A (logged-in) user can approve a single version of an article.
> * At least two approvals are needed for the "wikipedia seal of approval"
> :-)

Before discussing an approval method we should agree on what the approval is 
about. Is it about spelling/grammer correctness, factually accurateness, 
license problems, neutrality, ... or all of the above?

What we need for Wikipedia is a complete review in terms of quality similar to 
the traditional peer review system. I write similar since we already know 
that a traditional peer review system will not work for Wikipedia--too little 
incentive, too much work for a single person, basically it comes down to the 
same reasons why Nupedia has failed.

A complete review of an article needs all the things described in the 
beginning:

1. style - this includes grammer, well-written, image captions, ...
2. legal - in particular check status of images
3. completeness - does the article contain everything "important" about the 
subject
4. fact checking - validate every information in the article

For competing against Britannica 1, 3 and 4 are important. No.2 is important 
for avoiding legal mess a la SCO. 

A simple yes/no approval mechanism is IMHO not the right way in order to 
achieve our goals. I don't think that anyone will fact-check an article of 
>30.000 characters, this would literally take days of work if done properly. 
I even seriously doubt that everyone would read the whole article before 
approving. Erik Möller once wrote:

"My experience on FAC [featured articles candidates] indicates that many 
people have not even read the articles they support (no surprise, many of 
them are 40,000 characters and longer)."

Not to mention other problems:
* everyone can create an arbitrary number of accounts
* we do not know anything about the readers expertise on the field
* approving an article again and again after every (small) edit will become a 
pain

The last point IMHO is important. Every review method should be chosen in such 
a way that the additional work after several smaller changes have occurred is 
small. In particular one (trusted) user should be able to retransfer the 
results from a "stable" version to a more recent version.


I am confident that we can solve no.1 and no.2 and perhaps no.3 within the 
exiting wiki framework.  

[[Wikipedia:Featured article candidates]] is already a good base for no.1 and 
it seems to work reasonable well. And btw I never have heard any complaints 
in press that grammer and/or writing style of the articles is bad.

In order to avoid legal problems in the long run, in particular after having 
printed 1.0, we should at least have someone who is familiar with law and 
GFDL who looks through the article. In most cases it probably would be 
sufficient to have a closer look at the images. Here it would be helpful to 
have a list of trusted "experts" that could help, similar to 
[[Wikipedia:Association_of_Members'_Advocates]].

No.3 can become more difficult if the topic is very special. E.g. for the 
article [[quantum field theory]] only a small number of our users have the 
knowledge to judge if everything important is included. 

No.4 IMHO is the most difficult one and concerning the "reliability" aspect 
the most important one.

Checking every single statment can't be done by one user (it is too much 
work). Thus a simple yes/no for the whole article cannot be the solution. The 
hard point IMHO is the following:

* we either check the article in a very short time frame. Then we need an 
"expert committee" which can validates the material. This has also has been 
discussed many times, the last time 
[[meta:Article_validation#Proposition_of_validation_by_a_committee]]. Reading 
the proposition reminds me of Nupedia that's why I don't think this will 
work.

* the other way is that many person independently validate information in a 
longer time frame. The unsolved problem about this ansatz is how the system 
remembers which parts already have been validated and which haven't. 

(The discussion page won't be sufficient: if we know that A checked section 
1,2 of version 125, B checked section 2,3 of version 134 and C of 1,4 of 
version 145, it is hard to find out if really everything was validated 
because of the edits in between. And this is only an example with three 
reviewers where everyone has checked whole paragraphs.)

The edits between reviews make this hard. I have some ideas I'm playing around 
with in order to overcome the problem. This is done by allowing users to 
validate single sentences and by that allowing the underlying system to tell 
for every single sentence how often it was validated. (unfortunately it is 
not that easy, but this is another story)

O.k. just my 2 cents, why I think that a simple yes/no unfortunately is not 
sufficient for an equivalent of peer review. It's late, therefore I apologize 
for any extra grammer / spelling mistakes ;-)

best regards,
   Marco