Dear Andrew Lih,
dear scientific community,
I am a bit disappointed about the available material
that tries to measure the quality of Wikipedia articles.
The quoted newspaper article of the Wall street journal
for example just analyses technical topics but it would
be a dangerous claim to assume that quality is equally
distributed over the different fields and topics.
But you need that claim as condition for the method
of randomly picking articles and conclude for the rest.
There was another attempt to compare the Quality of
Wikipedia with other Encyclopedias in the German Computer
Newspaper C't with the same random approach.
(1) But there is a problem since it is a random way of
choosing articles to compare or to analyse. I see
some problems in non technical fields such as soft
sciences (in social science for example every theory
on society redefines all concepts of society on it's
own: how can an encyclopedia claim to have a definition?).
(2) Political terms are sometimes very complex topics
where the NPOV may not work, because there is no
right nor wrong. It is often a question of opinion
and majority that sometimes changes reality.
I observed a discussion and an edit war on the article
about Direct Democracy (in the Germen Wikipedia:
article "Direkte Demokratie") that led to a loss
of quality: only a minimal and weak consens
survived the different opinions: the evolutionary
process did not improve quality in that case.
(3) The third problem is the tendency of specific groups
that lead to vandalism. There are groups that use
values or ideologies and reject a neutral or scientific
view (moralists, religious groups, nationalists,
neocapitalists etc.). What about articles that are
important for these groups? Are these article tendentious?
My question: Is there a scientific study on the
quality of the Wikipedia ariticles? Does anyone
work on that problems? What methods could be used
to analyse the Quality?
(studies Wikipedia from a social system science view)
Hope to see many of you at Wikimania next week (yes, it's only one week
I want to propose some time is carved out for a BOAF session for wiki
researchers. Seems Friday and Sunday eves are free, or it could be Thursday
before things get started.
Here are some issues I'd love to talk to other folks about, please feel
free to add:
1. Heuristics for recognizing patterns in edit histories. Most pressing is
an algorithm to determine what constitutes an edit war, vandalism or any
other type of "noise" in the system if one's measuring "substantive" edits.
(This is hard - even the "I'll know it when I see it" method is problematic,
as evidenced by the recent dispute with and departure of RickK.) Much of the
research myself, Jakob Voss, Cathy Ma and others do depend on analyzing edit
histories and drawing conclusions about article quality. So far, none of the
research I've seen has "factored out" the effect of edit wars and vandalism.
2. Classifying types of edits, using diffs or edit summaries. There is a
desire to qualitative
lexical (spelling, punctuation), factual (numbers, dates), organizational
(rearranging), prose (style, tense change), etc. What are the best practices
in detecting and classifying these?
3. Comparative approaches to reserach and modelling article "clusters."
Last year, while comparing entries in a print encyclopedia against the
categories in Wikipedia, the toughest part was trying to match up the
taxonomical classifications, and the variation in breakdown into subtopics.
How are people dealing with this mapping?
Please add to the list, and I will help assemble.
University of Hong Kong
Best is to forward it to Wikipedia-l
--- Joachim Schroer
> Date: Tue, 12 Jul 2005 15:55:42 +0200
> From: Joachim Schroer
> To: wiki-research-l(a)Wikimedia.org
> Subject: [Wiki-research-l] Announcement: Survey
> study on the motivation of contributors to Wikipedi
> Dear all,
> we're a team of organizational psychologists at the
> University of
> Wuerzburg (Germany), and at the moment we're
> conducting a survey study
> on the motivation of contributors to Wikipedia.
> The study is available at:
> Our project page is here:
> I have already sent the announcement to the WikiEN
> The questionnaire is in English, but participants
> from other Wikipedias
> are of course invited to take part as well. Do you
> have ideas how to
> make the survey known in other projects?
> We'd also be happy...
> - if you could forward this link to other people who
> might be
> interested, but do not read the mailing lists, and
> - if you might put up links to our survey where they
> Thank you!
> Best wishes from Wuerzburg,
> Joachim Schroer
> Joachim Schroer, Dipl.-Psych.
> University of Wuerzburg
> Department of Psychology II, Industrial and
> Organizational Psychology
> Roentgenring 10
> 97070 Wuerzburg
> Phone: +49 931 31 6062
> Fax: +49 931 31 6063
> Wiki-research-l mailing list
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
we're a team of organizational psychologists at the University of
Wuerzburg (Germany), and at the moment we're conducting a survey study
on the motivation of contributors to Wikipedia.
The study is available at:
Our project page is here:
I have already sent the announcement to the WikiEN list:
The questionnaire is in English, but participants from other Wikipedias
are of course invited to take part as well. Do you have ideas how to
make the survey known in other projects?
We'd also be happy...
- if you could forward this link to other people who might be
interested, but do not read the mailing lists, and
- if you might put up links to our survey where they fit.
Best wishes from Wuerzburg,
Joachim Schroer, Dipl.-Psych.
University of Wuerzburg
Department of Psychology II, Industrial and Organizational Psychology
Phone: +49 931 31 6062
Fax: +49 931 31 6063