Dear Kerry,
Concerning: "I think we need to have some common understanding of what
we mean by quality, before we try to compare it across languages."
When reviewing Wikipedia research we already did that. Our categories
were very much like those you suggested. Under quality
http://wikilit.referata.com/wiki/Category:Quality we have:
1) Comprehensiveness (i.e., completeness or coverage)
2) Currency (i.e., up-to-dateness)
3) Readability and style (spelling and grammar could go here)
4) Reliability (accuracy, e.g., factual errors)
And not really quality per se:
a) Antecedents of quality
b) Featured articles
In our JASIST paper in print '"The sum of all human knowledge": a
systematic review of scholarly research on the content of Wikipedia' we
furthermore has a section called 'verifiability' (i.e., 'use of sources')
In my review ("Wikipedia research and tools: Review and comments") I
have this list: accuracy (no factual errors), coverage, bias,
conciseness, readability, up-to-dateness, usable/suitable
and whether the articles are well-illustrated and
well-sourced.
Note that in our review we distinguish between "real" quality and user
perception of quality. You will see our list of studies on perception of
quality here:
http://wikilit.referata.com/wiki/Category:Reader_perceptions_of_credibility
These studies are discussed in "Wikipedia in the eyes of its beholders:
a systematic review of scholarly research on Wikipedia readers and
readership" (page 14+)
The reviews are (perhaps!?) available from (The webservers have had
problems. If a link does not work try the other one or contact us):
'"The sum of all human knowledge": a systematic review of scholarly
research on the content of Wikipedia'
http://www2.compute.dtu.dk/pubdb/views/edoc_download.php/6784/pdf/imm6784.p…
http://spectrum.library.concordia.ca/978618/1/WikiLit_Content_%2D_open_acce…
http://neuro.compute.dtu.dk/wiki/%22The_sum_of_all_human_knowledge%22:_a_sy…
"Wikipedia research and tools: Review and comments"
http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6012/pdf/imm6012.pdf
"Wikipedia in the eyes of its beholders: a systematic review of
scholarly research on Wikipedia readers and readership"
http://www2.compute.dtu.dk/pubdb/views/edoc_download.php/6785/pdf/imm6785.p…
http://spectrum.library.concordia.ca/978617/1/Wikipedia_Readership_-_JASIST…
http://neuro.compute.dtu.dk/wiki/Wikipedia_in_the_eyes_of_its_beholders:_a_…
best
Finn Årup Nielsen
On 06/11/2014 02:19 AM, Kerry Raymond wrote:
Having followed this thread, I am somewhat confused
about what is meant by
the term "article quality", even in a single language, yet alone multiple
languages.
Sticking just to a single language for the moment ...
Do we mean that the facts presented are correct? That the kings and queens
were born and died on the dates stated?
Do we mean spelling and grammar is correct? Do we mean some kind of logical
structure? Do we mean some kind of narrative flow that "tells the story" of
the topic in a natural and engaging way?
Do we mean the use of citations? Do we mean whether the citation used
actually contains information that supports what is said by the text in the
article with which it is associated?
Do we mean some kind of "completeness" of an article? That is, it has
"all"
the information. If so, what do we do if the topic is split across a number
of articles {{main|...}}}? Do we assess the group of articles? And what do
we mean by "all" anyway?
Do we mean it meets all the WP policies? Notability? Appropriate use of
external links? That the Manual of Style has been carefully followed?
Or do we mean whether it has been assessed as a stub/start/.../good article
by some review process?
Whenever I find myself in a discussion about "quality" (on any subject, not
just Wikipedia), it pretty much always boils down to "fitness for purpose as
perceived by the user". This is why surveying of users is often used to
measure quality. "How well did we serve you today?" If anyone has been
through Singapore Airport recently, you will have encountered the touch
screens asking to rate on a 1-5 scale just about everything you could
imagine, every toilet block, every immigration queue, etc. And it does have
the cleanest toilets and the fastest immigration queues, so maybe there's
something to be said for the approach.
I think we need to have some common understanding of what we mean by
quality, before we try to compare it across languages. And when we do
compare across languages, then we have to observe that the set of users
changes and presumably their needs change too.
It is interesting to note that en.WP page views have dropped consistently
since Google Knowledge (which generally displays the first para from the
en.WP article) was introduced. What this tells us is that a certain
percentage of readers of an article simply want the most basic facts, which
would be delivered even by a stub article. "Suriname is a country on the
northeastern Atlantic coast of South America" certainly met my information
needs adequately (I heard it mentioned on the TV news in connection with a
hurricane). After finding out where it was in the world, I could have gone
on to read about its colonial history, its demographic sexuality, and its
biodiversity, but I didn't because I didn't have a need to know at that
moment. My point here is that while we would not generally regard a stub as
"quality", but a percentage of the readers of a stub are probably completely
satisfied.
Of course, doing surveys of articles with real users is somewhat difficult
for a research project. But it might be useful to see how user perceptions
of quality compare with other metrics (particularly those which can be more
easily generated for a research project). Starting with other metrics,
without knowing that they are a good proxy for user perception, is probably
a waste of time.
Kerry
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l