(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org http://hblog.org/ | @hfordsa http://www.twitter.com/hfordsa
On 10 June 2014 07:58, Anders Wennersten <mail@anderswennersten.se mailto:mail@anderswennersten.se> wrote:
(reposted from Wikimedia-i) I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles). enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?) eswp - a very good version, which in the general discussion are not getting appropriate credit dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered, nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content dawp,fiwp and plwp -Ok but only used by me for articles related to the country (arabic, chinese and japanese I almost never use, too complicated) (I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place) Anders _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten <mail@anderswennersten.se
wrote:
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing listWiki-research-l@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.
And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions
Anders
Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten <mail@anderswennersten.se mailto:mail@anderswennersten.se> wrote:
Thanks for answer Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others? I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp) Anders Heather Ford skrev 2014-06-10 13:09:Hi Anders, Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know! Thanks! Best, heather. [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract [2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual_information_quality_evaluation_-_The_case_of_Arabic_English_and_Korean_Wikipedia/file/60b7d51ae682e9912a.pdf Heather Ford Oxford Internet Institute <http://www.oii.ox.ac.uk> Doctoral Programme EthnographyMatters <http://ethnographymatters.net> | Oxford Digital Ethnography Group <http://www.oii.ox.ac.uk/research/projects/?id=115> http://hblog.org <http://hblog.org/> | @hfordsa <http://www.twitter.com/hfordsa> On 10 June 2014 07:58, Anders Wennersten <mail@anderswennersten.se <mailto:mail@anderswennersten.se>> wrote: (reposted from Wikimedia-i) I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles). enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?) eswp - a very good version, which in the general discussion are not getting appropriate credit dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered, nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content dawp,fiwp and plwp -Ok but only used by me for articles related to the country (arabic, chinese and japanese I almost never use, too complicated) (I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place) Anders _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l-- www.domusaurea.org http://www.domusaurea.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
This thread seems really to be dealing with two distinct issues: coverage and quality.
On the coverage side, the best work I know of is from Brent Hecht and Darren Gergle. Especially [1] but also [2,3]. The English edition may be the greatest in size, but in reality most other languages contain many unique articles not found in English. All the editions to some extent suffer from a self-focus bias where regions where the language is spoken are better represented than regions where the language is not spoken.
Quality of existing articles is completely separate from coverage and is harder to evaluate at scale. I understand that some of the work at GroupLens (Minnesota) around SuggestBot is trying to develop metrics that can measure quality at scale in different languages (and working to develop/use different metrics in different languages as appropriate because not all editions have the same conventions for referencing styles, etc.). I don't have a specific reference here, but would say keep an eye on this area.
~Scott
[1] Hecht, B., & Gergle, D. (2010). The Tower of Babel meets Web 2.0: User-generated content and its applications in a multilingual context. Proceedings of the 28th International Conference on Human Factors in Computing Systems (pp. 291-300). New York, NY, USA: ACM. doi: http://doi.acm.org/10.1145/1753326.1753370
[2] Hecht, B., & Gergle, D. (2009). Measuring self-focus bias in community-maintained knowledge repositories. *Proceedings of the Fourth International Conference on Communities and Technologies* (pp. 11-20). New York, NY, USA: ACM. doi:http://doi.acm.org/10.1145/1556460.1556463
[3] Bao, P., Hecht, B., Carton, S., Quaderi, M., Horn, M., & Gergle, D. (2012). Omnipedia: Bridging the Wikipedia Language Gap. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (pp. 1075-1084). New York, NY, USA: ACM. doi:10.1145/2207676.2208553
On Tue, Jun 10, 2014 at 7:34 AM, Anders Wennersten <mail@anderswennersten.se
wrote:
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.
And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions
Anders
Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten < mail@anderswennersten.se> wrote:
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing listWiki-research-l@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- www.domusaurea.org
Wiki-research-l mailing listWiki-research-l@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Dear Anders,
Thank you for bringing this up. My experience is that there is still a huge gap between computer-based quantity-oriented studies and human-based selective sample quality-oriented studies.
I published in 2009 a paper on "small" or "weak" articles but I am afraid that it was too much a numbers' game. It contained a table that differentiated between large, middle-sized, small and mini Wikipedias, with some assumptions on the quality and the power to cover topics.
Last year I started with a paper but the publishers seemed not to finish their project. I compared the notability criteria of en, de, nl, af (Afrikaans) and fo (Frisian) and found out that they are actually very comparable, as far as they can be compared at all. The often assumed "severity" of de.wp on notability seems to be a myth, maybe based on anti German cliche.
Now I have made for my lectures a table of small and larger encyclopedic articles in order to compare a topic in different reference works. Reason for this is also my contribution to the Historians' Convention (Historikertag) later this year. My basic question is whether Wikipedia is a good starting point for a historic topic, following the research of early deceased Swiss historian Peter Haber.
Haber made his point i.a. at the example of [[de:Frühmittelalter]] (early middle ages) in 2010. That article, he complained, contained no real inaccuracies, but still it was useless for a student. No good structure, some facts put one after the other etc. His explanation: if you want to write an article about a person, say about Henri Dunant, you take some biographies and write from his birth to his death and legacy. That's relatively easy and can be done by any good writer. But for a comprehensive article on the early middle ages, you must be a skilled historian very familiar with the period.
(I now experience the same with a series of Wikipedia articles I write about a certain period in German history. Just following the (older) standard reference works would simply not make me happy, not be a really valuable contribution to Wikipedia. With (nearly) every new work I get from the inter library loan I see that it is good to wait with publication of an article until I have together the set of works I deem necessary. - I consider to write a kind of report about this series.)
It would be great to have a set of criteria for an article typology, based partially on function of the article (overview, or registration of an item in a row etc.) and the inner quality (structure, comprehensiveness, based on literature etc.).
Kind regards Ziko
2014-06-10 14:34 GMT+02:00 Anders Wennersten mail@anderswennersten.se:
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.
And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions
Anders
Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten mail@anderswennersten.se wrote:
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute Doctoral Programme EthnographyMatters | Oxford Digital Ethnography Group http://hblog.org | @hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- www.domusaurea.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Ziko, could you please supply the full reference to Haber's article? Indeed, what I observe in History-related articles is almost a tendency towards positivism, histoire événementielle - hence, for instance, the vast number of battle themes. Discussion of historiographic approaches to concepts is usually quite rare and badly written.
Anyone willing to conduct comparative research on quality of History-related articles, please drop me a note!
Juliana.
On Tue, Jun 10, 2014 at 11:12 AM, Ziko van Dijk zvandijk@gmail.com wrote:
Dear Anders,
Thank you for bringing this up. My experience is that there is still a huge gap between computer-based quantity-oriented studies and human-based selective sample quality-oriented studies.
I published in 2009 a paper on "small" or "weak" articles but I am afraid that it was too much a numbers' game. It contained a table that differentiated between large, middle-sized, small and mini Wikipedias, with some assumptions on the quality and the power to cover topics.
Last year I started with a paper but the publishers seemed not to finish their project. I compared the notability criteria of en, de, nl, af (Afrikaans) and fo (Frisian) and found out that they are actually very comparable, as far as they can be compared at all. The often assumed "severity" of de.wp on notability seems to be a myth, maybe based on anti German cliche.
Now I have made for my lectures a table of small and larger encyclopedic articles in order to compare a topic in different reference works. Reason for this is also my contribution to the Historians' Convention (Historikertag) later this year. My basic question is whether Wikipedia is a good starting point for a historic topic, following the research of early deceased Swiss historian Peter Haber.
Haber made his point i.a. at the example of [[de:Frühmittelalter]] (early middle ages) in 2010. That article, he complained, contained no real inaccuracies, but still it was useless for a student. No good structure, some facts put one after the other etc. His explanation: if you want to write an article about a person, say about Henri Dunant, you take some biographies and write from his birth to his death and legacy. That's relatively easy and can be done by any good writer. But for a comprehensive article on the early middle ages, you must be a skilled historian very familiar with the period.
(I now experience the same with a series of Wikipedia articles I write about a certain period in German history. Just following the (older) standard reference works would simply not make me happy, not be a really valuable contribution to Wikipedia. With (nearly) every new work I get from the inter library loan I see that it is good to wait with publication of an article until I have together the set of works I deem necessary. - I consider to write a kind of report about this series.)
It would be great to have a set of criteria for an article typology, based partially on function of the article (overview, or registration of an item in a row etc.) and the inner quality (structure, comprehensiveness, based on literature etc.).
Kind regards Ziko
2014-06-10 14:34 GMT+02:00 Anders Wennersten mail@anderswennersten.se:
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films etc where svwp are behind most versions but where enwp is
excellent.
The interesting comparisons comes from the next levels of articles that
can
be almost anything, a footballstadium in Kazan Russia, an albanian poet,
a
church in Venize, a specie with unclear taxonomy, the american solider
who
perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.
And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions
Anders
Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about
studies
on Wikipedias other than English. Well, not that I believe that the
Featured
Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a
drop in
the ocean, but I've been fixing some new articles that are translations
from
bad English ones - which look good, but analyzing the content reveals
many
problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten mail@anderswennersten.se wrote:
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp
and
how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the
movement
as they can jeoprdaize the brand name, as they are not living up to
basic Q
(and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting
some
few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias
and
I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The
majority of
quality studies, we find, have been done on English Wikipedia (starting
with
the famous 2005 Nature study) but there have been few studies that
assess of
quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2]
http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute Doctoral Programme EthnographyMatters | Oxford Digital Ethnography Group http://hblog.org | @hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se
wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done,
perhaps
you know more on this list?. before we start the strategy work I
believe we
should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on
sv.wp) (I
list them in order how often I use them to calibrate the svwp
articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if
rather
bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious
subjects
(that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and
the
different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in
other
languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are
uneven. I
now use it even less then Ukrainian and Russian which I use very
seldom as
the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I
am not
sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- www.domusaurea.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Indeed, Juliana, one example: for the complex process with regard to a German emperor in 1848/1849, it would have been possible to write one or several articles on e.g. the debates in the National Assembly. But what did de.wp? Elected with [[Kaiserdeputation]], the Assembly's delegation to the Prussian King, the most visible element of the process.
I now see that https://de.wikipedia.org/wiki/Reichsgr%C3%BCndung is a similar example, also an article suffering from the Frühmittelalter-problem and other issues. "Foundation of the Empire".
* With the picture of von Werner about the proclamation on January 18, it highlights the most visible element.
* "Staatsgründung": Very basically about the (more important) legal proceedings of the parliament
* A huge part about the proclamation
* "Sichtweise der süddeutschen Staaten": good part, but not much connected to the others
* "Folgen und Bewertung": a mix of what followed and a judgement. With good elements, partially two long, some inaccuracies or improper wordings
* Following a paragraph on the Franco-German relations (not quite suitable here, or in larger European context of the event in question, the foundation of the Empire). Following a list and map of the single German states, which we already have elsewhere
* A list with literature, partially not directly related to the topic or dated
* Surprisingly many footnotes, with a certain diversity of sometimes very general works.
Haber is this person: https://de.wikipedia.org/wiki/Peter_Haber_%28Historiker%29
In English short: http://www.hist.net/index.php?id=39&L=1
http://wiki.histnet.ch/index.php/Werkstattgespr%C3%A4ch_Wien_2010
E.g.: http://derstandard.at/1277337531926/Wer-viel-Zeit-hat-hat-bei-Wikipedia-das-...
I can't find anything more specific at the moment.
Kind regards Ziko
2014-06-10 17:07 GMT+02:00 Juliana Bastos Marques domusaurea@gmail.com:
Ziko, could you please supply the full reference to Haber's article? Indeed, what I observe in History-related articles is almost a tendency towards positivism, histoire événementielle - hence, for instance, the vast number of battle themes. Discussion of historiographic approaches to concepts is usually quite rare and badly written.
Anyone willing to conduct comparative research on quality of History-related articles, please drop me a note!
Juliana.
On Tue, Jun 10, 2014 at 11:12 AM, Ziko van Dijk zvandijk@gmail.com wrote:
Dear Anders,
Thank you for bringing this up. My experience is that there is still a huge gap between computer-based quantity-oriented studies and human-based selective sample quality-oriented studies.
I published in 2009 a paper on "small" or "weak" articles but I am afraid that it was too much a numbers' game. It contained a table that differentiated between large, middle-sized, small and mini Wikipedias, with some assumptions on the quality and the power to cover topics.
Last year I started with a paper but the publishers seemed not to finish their project. I compared the notability criteria of en, de, nl, af (Afrikaans) and fo (Frisian) and found out that they are actually very comparable, as far as they can be compared at all. The often assumed "severity" of de.wp on notability seems to be a myth, maybe based on anti German cliche.
Now I have made for my lectures a table of small and larger encyclopedic articles in order to compare a topic in different reference works. Reason for this is also my contribution to the Historians' Convention (Historikertag) later this year. My basic question is whether Wikipedia is a good starting point for a historic topic, following the research of early deceased Swiss historian Peter Haber.
Haber made his point i.a. at the example of [[de:Frühmittelalter]] (early middle ages) in 2010. That article, he complained, contained no real inaccuracies, but still it was useless for a student. No good structure, some facts put one after the other etc. His explanation: if you want to write an article about a person, say about Henri Dunant, you take some biographies and write from his birth to his death and legacy. That's relatively easy and can be done by any good writer. But for a comprehensive article on the early middle ages, you must be a skilled historian very familiar with the period.
(I now experience the same with a series of Wikipedia articles I write about a certain period in German history. Just following the (older) standard reference works would simply not make me happy, not be a really valuable contribution to Wikipedia. With (nearly) every new work I get from the inter library loan I see that it is good to wait with publication of an article until I have together the set of works I deem necessary. - I consider to write a kind of report about this series.)
It would be great to have a set of criteria for an article typology, based partially on function of the article (overview, or registration of an item in a row etc.) and the inner quality (structure, comprehensiveness, based on literature etc.).
Kind regards Ziko
2014-06-10 14:34 GMT+02:00 Anders Wennersten mail@anderswennersten.se:
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.
And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions
Anders
Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten mail@anderswennersten.se wrote:
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2]
http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute Doctoral Programme EthnographyMatters | Oxford Digital Ethnography Group http://hblog.org | @hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- www.domusaurea.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- www.domusaurea.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Thanks a lot for all feedback and links I will look into.
My feeling, though, is that I am more interested in what is "below the belt", ie rottenness creeping into different versions, more then the more acceptable quality aspect we are used looking into.
I will see if I, after reading your links, can formulate what I would like to see (which perhaps I even an do myself) in a kind a "research proposal" and hope I can preesnt it here to get feedback on it
Anders
Ziko van Dijk skrev 2014-06-10 16:12:
Dear Anders,
Thank you for bringing this up. My experience is that there is still a huge gap between computer-based quantity-oriented studies and human-based selective sample quality-oriented studies.
I published in 2009 a paper on "small" or "weak" articles but I am afraid that it was too much a numbers' game. It contained a table that differentiated between large, middle-sized, small and mini Wikipedias, with some assumptions on the quality and the power to cover topics.
Last year I started with a paper but the publishers seemed not to finish their project. I compared the notability criteria of en, de, nl, af (Afrikaans) and fo (Frisian) and found out that they are actually very comparable, as far as they can be compared at all. The often assumed "severity" of de.wp on notability seems to be a myth, maybe based on anti German cliche.
Now I have made for my lectures a table of small and larger encyclopedic articles in order to compare a topic in different reference works. Reason for this is also my contribution to the Historians' Convention (Historikertag) later this year. My basic question is whether Wikipedia is a good starting point for a historic topic, following the research of early deceased Swiss historian Peter Haber.
Haber made his point i.a. at the example of [[de:Frühmittelalter]] (early middle ages) in 2010. That article, he complained, contained no real inaccuracies, but still it was useless for a student. No good structure, some facts put one after the other etc. His explanation: if you want to write an article about a person, say about Henri Dunant, you take some biographies and write from his birth to his death and legacy. That's relatively easy and can be done by any good writer. But for a comprehensive article on the early middle ages, you must be a skilled historian very familiar with the period.
(I now experience the same with a series of Wikipedia articles I write about a certain period in German history. Just following the (older) standard reference works would simply not make me happy, not be a really valuable contribution to Wikipedia. With (nearly) every new work I get from the inter library loan I see that it is good to wait with publication of an article until I have together the set of works I deem necessary. - I consider to write a kind of report about this series.)
It would be great to have a set of criteria for an article typology, based partially on function of the article (overview, or registration of an item in a row etc.) and the inner quality (structure, comprehensiveness, based on literature etc.).
Kind regards Ziko
2014-06-10 14:34 GMT+02:00 Anders Wennersten mail@anderswennersten.se:
My starting point have been the newly created articles on svwp. They will represent the usual bunch of football playser, tv-stars, computergames, films etc where svwp are behind most versions but where enwp is excellent. The interesting comparisons comes from the next levels of articles that can be almost anything, a footballstadium in Kazan Russia, an albanian poet, a church in Venize, a specie with unclear taxonomy, the american solider who perhaps deserted etc. In these cases I only often find a corresponding article in enwp, but also very often (around 20%) I find it in another version and no presence in enwp.
And when enwp is not giving me support, I most often find support in eswp and frwp, sometimes in dewp, but almost never in ptwp. For exemple taxanomical threes with name in native and latin is about the weakest in ptwp. But I can be wrong and I would love to be part in a more complete research on Q comparisons for the different versions
Anders
Juliana Bastos Marques skrev 2014-06-10 14:06:
This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content.
Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems.
Juliana.
On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten mail@anderswennersten.se wrote:
Thanks for answer
Your answer confirm my "fear", that focus is almost completly to en:wp and how it is compared with an ideal perfect Q
My interest and what I believe the movement need before we dig into next round of strategy round is *what versions are dysfunctional. These represent a risk for the movement as they can jeoprdaize the brand name, as they are not living up to basic Q (and NPOV) *what can we learn from each other, why are some better in some aspects and worse in others?
I would recommend a research approach much more basic just collecting some few data on each version (and forget about enwp)
Anders
Heather Ford skrev 2014-06-10 13:09:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute Doctoral Programme EthnographyMatters | Oxford Digital Ethnography Group http://hblog.org | @hfordsa
On 10 June 2014 07:58, Anders Wennersten mail@anderswennersten.se wrote:
(reposted from Wikimedia-i)
I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one
I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles).
enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?)
eswp - a very good version, which in the general discussion are not getting appropriate credit
dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered,
nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages
ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content
dawp,fiwp and plwp -Ok but only used by me for articles related to the country
(arabic, chinese and japanese I almost never use, too complicated)
(I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place)
Anders
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
-- www.domusaurea.org
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Having followed this thread, I am somewhat confused about what is meant by the term "article quality", even in a single language, yet alone multiple languages.
Sticking just to a single language for the moment ...
Do we mean that the facts presented are correct? That the kings and queens were born and died on the dates stated?
Do we mean spelling and grammar is correct? Do we mean some kind of logical structure? Do we mean some kind of narrative flow that "tells the story" of the topic in a natural and engaging way?
Do we mean the use of citations? Do we mean whether the citation used actually contains information that supports what is said by the text in the article with which it is associated?
Do we mean some kind of "completeness" of an article? That is, it has "all" the information. If so, what do we do if the topic is split across a number of articles {{main|...}}}? Do we assess the group of articles? And what do we mean by "all" anyway?
Do we mean it meets all the WP policies? Notability? Appropriate use of external links? That the Manual of Style has been carefully followed?
Or do we mean whether it has been assessed as a stub/start/.../good article by some review process?
Whenever I find myself in a discussion about "quality" (on any subject, not just Wikipedia), it pretty much always boils down to "fitness for purpose as perceived by the user". This is why surveying of users is often used to measure quality. "How well did we serve you today?" If anyone has been through Singapore Airport recently, you will have encountered the touch screens asking to rate on a 1-5 scale just about everything you could imagine, every toilet block, every immigration queue, etc. And it does have the cleanest toilets and the fastest immigration queues, so maybe there's something to be said for the approach.
I think we need to have some common understanding of what we mean by quality, before we try to compare it across languages. And when we do compare across languages, then we have to observe that the set of users changes and presumably their needs change too.
It is interesting to note that en.WP page views have dropped consistently since Google Knowledge (which generally displays the first para from the en.WP article) was introduced. What this tells us is that a certain percentage of readers of an article simply want the most basic facts, which would be delivered even by a stub article. "Suriname is a country on the northeastern Atlantic coast of South America" certainly met my information needs adequately (I heard it mentioned on the TV news in connection with a hurricane). After finding out where it was in the world, I could have gone on to read about its colonial history, its demographic sexuality, and its biodiversity, but I didn't because I didn't have a need to know at that moment. My point here is that while we would not generally regard a stub as "quality", but a percentage of the readers of a stub are probably completely satisfied.
Of course, doing surveys of articles with real users is somewhat difficult for a research project. But it might be useful to see how user perceptions of quality compare with other metrics (particularly those which can be more easily generated for a research project). Starting with other metrics, without knowing that they are a good proxy for user perception, is probably a waste of time.
Kerry
Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts!
btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2].
And thanks, Finn, for the great work you continue to do with Wikilit :)
Best, Heather.
[1] http://scholar.google.co.uk/citations?user=Xhl9P7oAAAAJ&hl=en [2] http://onlinelibrary.wiley.com/doi/10.1002/asi.21531/full
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa
On 11 June 2014 01:19, Kerry Raymond kerry.raymond@gmail.com wrote:
Having followed this thread, I am somewhat confused about what is meant by the term "article quality", even in a single language, yet alone multiple languages.
Sticking just to a single language for the moment ...
Do we mean that the facts presented are correct? That the kings and queens were born and died on the dates stated?
Do we mean spelling and grammar is correct? Do we mean some kind of logical structure? Do we mean some kind of narrative flow that "tells the story" of the topic in a natural and engaging way?
Do we mean the use of citations? Do we mean whether the citation used actually contains information that supports what is said by the text in the article with which it is associated?
Do we mean some kind of "completeness" of an article? That is, it has "all" the information. If so, what do we do if the topic is split across a number of articles {{main|...}}}? Do we assess the group of articles? And what do we mean by "all" anyway?
Do we mean it meets all the WP policies? Notability? Appropriate use of external links? That the Manual of Style has been carefully followed?
Or do we mean whether it has been assessed as a stub/start/.../good article by some review process?
Whenever I find myself in a discussion about "quality" (on any subject, not just Wikipedia), it pretty much always boils down to "fitness for purpose as perceived by the user". This is why surveying of users is often used to measure quality. "How well did we serve you today?" If anyone has been through Singapore Airport recently, you will have encountered the touch screens asking to rate on a 1-5 scale just about everything you could imagine, every toilet block, every immigration queue, etc. And it does have the cleanest toilets and the fastest immigration queues, so maybe there's something to be said for the approach.
I think we need to have some common understanding of what we mean by quality, before we try to compare it across languages. And when we do compare across languages, then we have to observe that the set of users changes and presumably their needs change too.
It is interesting to note that en.WP page views have dropped consistently since Google Knowledge (which generally displays the first para from the en.WP article) was introduced. What this tells us is that a certain percentage of readers of an article simply want the most basic facts, which would be delivered even by a stub article. "Suriname is a country on the northeastern Atlantic coast of South America" certainly met my information needs adequately (I heard it mentioned on the TV news in connection with a hurricane). After finding out where it was in the world, I could have gone on to read about its colonial history, its demographic sexuality, and its biodiversity, but I didn't because I didn't have a need to know at that moment. My point here is that while we would not generally regard a stub as "quality", but a percentage of the readers of a stub are probably completely satisfied.
Of course, doing surveys of articles with real users is somewhat difficult for a research project. But it might be useful to see how user perceptions of quality compare with other metrics (particularly those which can be more easily generated for a research project). Starting with other metrics, without knowing that they are a good proxy for user perception, is probably a waste of time.
Kerry
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
My comment on Google Knowledge relates to a presentation in one of the WMF Metrics meetings and subsequent discussion (all of which can be seen on video since about 2012):
https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings (Heather, this is a better URL than the one I sent you by accident a moment ago)
There are agendas but, while they tell you who will be speaking and about what topic, they don't provide enough info for me to say which meeting was the one when they had discussed Google Knowledge as it was probably part of an agenda item labelled something like "top line metrics".
I find it's worth viewing these videos as they give the best insight into what's happening at WMF at the moment. And of course there is always a metrics presentation, discussion of features in development, in beta, rolled out etc.
Kerry
_____
From: Heather Ford [mailto:hfordsa@gmail.com] Sent: Wednesday, 11 June 2014 6:53 PM To: kerry.raymond@gmail.com; Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] Quality on different language version
Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts!
btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2].
And thanks, Finn, for the great work you continue to do with Wikilit :)
Best,
Heather.
[1] http://scholar.google.co.uk/citations?user=Xhl9P7oAAAAJ http://scholar.google.co.uk/citations?user=Xhl9P7oAAAAJ&hl=en &hl=en
[2] http://onlinelibrary.wiley.com/doi/10.1002/asi.21531/full
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford http://www.oii.ox.ac.uk/research/projects/?id=115 Digital Ethnography Group http://hblog.org http://hblog.org/ | @hfordsa http://www.twitter.com/hfordsa
On 11 June 2014 01:19, Kerry Raymond kerry.raymond@gmail.com wrote:
Having followed this thread, I am somewhat confused about what is meant by the term "article quality", even in a single language, yet alone multiple languages.
Sticking just to a single language for the moment ...
Do we mean that the facts presented are correct? That the kings and queens were born and died on the dates stated?
Do we mean spelling and grammar is correct? Do we mean some kind of logical structure? Do we mean some kind of narrative flow that "tells the story" of the topic in a natural and engaging way?
Do we mean the use of citations? Do we mean whether the citation used actually contains information that supports what is said by the text in the article with which it is associated?
Do we mean some kind of "completeness" of an article? That is, it has "all" the information. If so, what do we do if the topic is split across a number of articles {{main|...}}}? Do we assess the group of articles? And what do we mean by "all" anyway?
Do we mean it meets all the WP policies? Notability? Appropriate use of external links? That the Manual of Style has been carefully followed?
Or do we mean whether it has been assessed as a stub/start/.../good article by some review process?
Whenever I find myself in a discussion about "quality" (on any subject, not just Wikipedia), it pretty much always boils down to "fitness for purpose as perceived by the user". This is why surveying of users is often used to measure quality. "How well did we serve you today?" If anyone has been through Singapore Airport recently, you will have encountered the touch screens asking to rate on a 1-5 scale just about everything you could imagine, every toilet block, every immigration queue, etc. And it does have the cleanest toilets and the fastest immigration queues, so maybe there's something to be said for the approach.
I think we need to have some common understanding of what we mean by quality, before we try to compare it across languages. And when we do compare across languages, then we have to observe that the set of users changes and presumably their needs change too.
It is interesting to note that en.WP page views have dropped consistently since Google Knowledge (which generally displays the first para from the en.WP article) was introduced. What this tells us is that a certain percentage of readers of an article simply want the most basic facts, which would be delivered even by a stub article. "Suriname is a country on the northeastern Atlantic coast of South America" certainly met my information needs adequately (I heard it mentioned on the TV news in connection with a hurricane). After finding out where it was in the world, I could have gone on to read about its colonial history, its demographic sexuality, and its biodiversity, but I didn't because I didn't have a need to know at that moment. My point here is that while we would not generally regard a stub as "quality", but a percentage of the readers of a stub are probably completely satisfied.
Of course, doing surveys of articles with real users is somewhat difficult for a research project. But it might be useful to see how user perceptions of quality compare with other metrics (particularly those which can be more easily generated for a research project). Starting with other metrics, without knowing that they are a good proxy for user perception, is probably a waste of time.
Kerry
_______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Actually, we're still doing work on whether the KnowledgeGraph stuff is having an impact, or whether it's a problem with the data source, or, or..
On 11 June 2014 14:21, Kerry Raymond kerry.raymond@gmail.com wrote:
My comment on Google Knowledge relates to a presentation in one ofthe WMF Metrics meetings and subsequent discussion (all of which can be seen on video since about 2012):
https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings (Heather, this is a better URL than the one I sent you by accident a moment ago)
There are agendas but, while they tell you who will be speaking and about what topic, they don’t provide enough info for me to say which meeting was the one when they had discussed Google Knowledge as it was probably part of an agenda item labelled something like “top line metrics”.
I find it’s worth viewing these videos as they give the best insight into what’s happening at WMF at the moment. And of course there is always a metrics presentation, discussion of features in development, in beta, rolled out etc.
Kerry
*From:* Heather Ford [mailto:hfordsa@gmail.com] *Sent:* Wednesday, 11 June 2014 6:53 PM *To:* kerry.raymond@gmail.com; Research into Wikimedia content and communities *Subject:* Re: [Wiki-research-l] Quality on different language version
Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts!
btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2].
And thanks, Finn, for the great work you continue to do with Wikilit :)
Best,
Heather.
[1] http://scholar.google.co.uk/citations?user=Xhl9P7oAAAAJ&hl=en
[2] http://onlinelibrary.wiley.com/doi/10.1002/asi.21531/full
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org | @hfordsa http://www.twitter.com/hfordsa
On 11 June 2014 01:19, Kerry Raymond kerry.raymond@gmail.com wrote:
Having followed this thread, I am somewhat confused about what is meant by the term "article quality", even in a single language, yet alone multiple languages.
Sticking just to a single language for the moment ...
Do we mean that the facts presented are correct? That the kings and queens were born and died on the dates stated?
Do we mean spelling and grammar is correct? Do we mean some kind of logical structure? Do we mean some kind of narrative flow that "tells the story" of the topic in a natural and engaging way?
Do we mean the use of citations? Do we mean whether the citation used actually contains information that supports what is said by the text in the article with which it is associated?
Do we mean some kind of "completeness" of an article? That is, it has "all" the information. If so, what do we do if the topic is split across a number of articles {{main|...}}}? Do we assess the group of articles? And what do we mean by "all" anyway?
Do we mean it meets all the WP policies? Notability? Appropriate use of external links? That the Manual of Style has been carefully followed?
Or do we mean whether it has been assessed as a stub/start/.../good article by some review process?
Whenever I find myself in a discussion about "quality" (on any subject, not just Wikipedia), it pretty much always boils down to "fitness for purpose as perceived by the user". This is why surveying of users is often used to measure quality. "How well did we serve you today?" If anyone has been through Singapore Airport recently, you will have encountered the touch screens asking to rate on a 1-5 scale just about everything you could imagine, every toilet block, every immigration queue, etc. And it does have the cleanest toilets and the fastest immigration queues, so maybe there's something to be said for the approach.
I think we need to have some common understanding of what we mean by quality, before we try to compare it across languages. And when we do compare across languages, then we have to observe that the set of users changes and presumably their needs change too.
It is interesting to note that en.WP page views have dropped consistently since Google Knowledge (which generally displays the first para from the en.WP article) was introduced. What this tells us is that a certain percentage of readers of an article simply want the most basic facts, which would be delivered even by a stub article. "Suriname is a country on the northeastern Atlantic coast of South America" certainly met my information needs adequately (I heard it mentioned on the TV news in connection with a hurricane). After finding out where it was in the world, I could have gone on to read about its colonial history, its demographic sexuality, and its biodiversity, but I didn't because I didn't have a need to know at that moment. My point here is that while we would not generally regard a stub as "quality", but a percentage of the readers of a stub are probably completely satisfied.
Of course, doing surveys of articles with real users is somewhat difficult for a research project. But it might be useful to see how user perceptions of quality compare with other metrics (particularly those which can be more easily generated for a research project). Starting with other metrics, without knowing that they are a good proxy for user perception, is probably a waste of time.
Kerry
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Dear Kerry,
Concerning: "I think we need to have some common understanding of what we mean by quality, before we try to compare it across languages."
When reviewing Wikipedia research we already did that. Our categories were very much like those you suggested. Under quality http://wikilit.referata.com/wiki/Category:Quality we have:
1) Comprehensiveness (i.e., completeness or coverage)
2) Currency (i.e., up-to-dateness)
3) Readability and style (spelling and grammar could go here)
4) Reliability (accuracy, e.g., factual errors)
And not really quality per se:
a) Antecedents of quality
b) Featured articles
In our JASIST paper in print '"The sum of all human knowledge": a systematic review of scholarly research on the content of Wikipedia' we furthermore has a section called 'verifiability' (i.e., 'use of sources')
In my review ("Wikipedia research and tools: Review and comments") I have this list: accuracy (no factual errors), coverage, bias, conciseness, readability, up-to-dateness, usable/suitable and whether the articles are well-illustrated and well-sourced.
Note that in our review we distinguish between "real" quality and user perception of quality. You will see our list of studies on perception of quality here:
http://wikilit.referata.com/wiki/Category:Reader_perceptions_of_credibility
These studies are discussed in "Wikipedia in the eyes of its beholders: a systematic review of scholarly research on Wikipedia readers and readership" (page 14+)
The reviews are (perhaps!?) available from (The webservers have had problems. If a link does not work try the other one or contact us):
'"The sum of all human knowledge": a systematic review of scholarly research on the content of Wikipedia' http://www2.compute.dtu.dk/pubdb/views/edoc_download.php/6784/pdf/imm6784.pd...
http://spectrum.library.concordia.ca/978618/1/WikiLit_Content_%2D_open_acces...
http://neuro.compute.dtu.dk/wiki/%22The_sum_of_all_human_knowledge%22:_a_sys...
"Wikipedia research and tools: Review and comments" http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6012/pdf/imm6012.pdf
"Wikipedia in the eyes of its beholders: a systematic review of scholarly research on Wikipedia readers and readership"
http://www2.compute.dtu.dk/pubdb/views/edoc_download.php/6785/pdf/imm6785.pd...
http://spectrum.library.concordia.ca/978617/1/Wikipedia_Readership_-_JASIST_...
http://neuro.compute.dtu.dk/wiki/Wikipedia_in_the_eyes_of_its_beholders:_a_s...
best Finn Årup Nielsen
On 06/11/2014 02:19 AM, Kerry Raymond wrote:
Having followed this thread, I am somewhat confused about what is meant by the term "article quality", even in a single language, yet alone multiple languages.
Sticking just to a single language for the moment ...
Do we mean that the facts presented are correct? That the kings and queens were born and died on the dates stated?
Do we mean spelling and grammar is correct? Do we mean some kind of logical structure? Do we mean some kind of narrative flow that "tells the story" of the topic in a natural and engaging way?
Do we mean the use of citations? Do we mean whether the citation used actually contains information that supports what is said by the text in the article with which it is associated?
Do we mean some kind of "completeness" of an article? That is, it has "all" the information. If so, what do we do if the topic is split across a number of articles {{main|...}}}? Do we assess the group of articles? And what do we mean by "all" anyway?
Do we mean it meets all the WP policies? Notability? Appropriate use of external links? That the Manual of Style has been carefully followed?
Or do we mean whether it has been assessed as a stub/start/.../good article by some review process?
Whenever I find myself in a discussion about "quality" (on any subject, not just Wikipedia), it pretty much always boils down to "fitness for purpose as perceived by the user". This is why surveying of users is often used to measure quality. "How well did we serve you today?" If anyone has been through Singapore Airport recently, you will have encountered the touch screens asking to rate on a 1-5 scale just about everything you could imagine, every toilet block, every immigration queue, etc. And it does have the cleanest toilets and the fastest immigration queues, so maybe there's something to be said for the approach.
I think we need to have some common understanding of what we mean by quality, before we try to compare it across languages. And when we do compare across languages, then we have to observe that the set of users changes and presumably their needs change too.
It is interesting to note that en.WP page views have dropped consistently since Google Knowledge (which generally displays the first para from the en.WP article) was introduced. What this tells us is that a certain percentage of readers of an article simply want the most basic facts, which would be delivered even by a stub article. "Suriname is a country on the northeastern Atlantic coast of South America" certainly met my information needs adequately (I heard it mentioned on the TV news in connection with a hurricane). After finding out where it was in the world, I could have gone on to read about its colonial history, its demographic sexuality, and its biodiversity, but I didn't because I didn't have a need to know at that moment. My point here is that while we would not generally regard a stub as "quality", but a percentage of the readers of a stub are probably completely satisfied.
Of course, doing surveys of articles with real users is somewhat difficult for a research project. But it might be useful to see how user perceptions of quality compare with other metrics (particularly those which can be more easily generated for a research project). Starting with other metrics, without knowing that they are a good proxy for user perception, is probably a waste of time.
Kerry
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Dear Heather,
In our WikiLit systematic reviews we found a few publications. I have just made a semantic query on the WikiLit site to give you an overview:
http://wikilit.referata.com/wiki/WikiLit:Quality
There are not that many. You should find them described in our review on research on Wikipedia content: "The sum of all human knowledge": a systematic review of scholarly research on the content of Wikipedia
http://spectrum.library.concordia.ca/978618/1/WikiLit_Content_%2D_open_acces...
http://neuro.compute.dtu.dk/wiki/%22The_sum_of_all_human_knowledge%22:_a_sys...
best Finn Årup Nielsen
On 06/10/2014 01:09 PM, Heather Ford wrote:
Hi Anders,
Yes, it's a great question! Mark Graham and I are currently working on a project around how to determine quality within and between Wikipedias and I've been looking around for literature. I'm only just starting the literature review but I've found some interesting studies by Callahan & Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority of quality studies, we find, have been done on English Wikipedia (starting with the famous 2005 Nature study) but there have been few studies that assess of quality between languages. If you find anything else, let us know!
Thanks!
Best, heather.
[1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract
[2] http://www.researchgate.net/publication/200773220_Issues_of_cross-contextual...
Heather Ford Oxford Internet Institute http://www.oii.ox.ac.uk Doctoral Programme EthnographyMatters http://ethnographymatters.net | Oxford Digital Ethnography Group http://www.oii.ox.ac.uk/research/projects/?id=115 http://hblog.org http://hblog.org/ | @hfordsa http://www.twitter.com/hfordsa
On 10 June 2014 07:58, Anders Wennersten <mail@anderswennersten.se mailto:mail@anderswennersten.se> wrote:
(reposted from Wikimedia-i) I have several times asked for a professional quality study of our different language versions, but not seen it exist or being done, perhaps you know more on this list?. before we start the strategy work I believe we should have basic facts on the table like this one I therefor list here my subjective impression after daily looking into the different version for 5-15 articles (new ones being created on sv.wp) (I list them in order how often I use them to calibrate the svwp articles). enwp- a magnitude better then any other. main weakeness are articles on marginal subjects that seems to be allowed to exist there, even if rather bad, and without templates (noone cares to patrol these?) eswp - a very good version, which in the general discussion are not getting appropriate credit dewp - good when the articles exist, but many serious holes. Is the elitist way of running it, discouraging new editors in non obvious subjects (that after time passes gets very relevant)? frwp - also good, but somewhat scattered quality both in coverage and the different articles (even in same subject area) nlwp - very good coverage in the geographic subjects, decent quality on articles but limited "world" coverage in areas like biographies itwp - good articles but a bit italiancentered, nowp - small but decent articles. Their short focused articletext sometimes give more easyaccessed knowledge then an overly long one in other languages ptwp - the real disappointment. it is among the top ten in volume and accesses but clearly missing a lot, and even existing articles are uneven. I now use it even less then Ukrainian and Russian which I use very seldom as the different alphabet makes it hard to understand the article content dawp,fiwp and plwp -Ok but only used by me for articles related to the country (arabic, chinese and japanese I almost never use, too complicated) (I also use some smaller ones like sqwp , in these versions I have seen serious quality problems not to be found in any of the above ones, I am not sure they even have basic patrolling in place) Anders _________________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.__wikimedia.org <mailto:Wiki-research-l@lists.wikimedia.org> https://lists.wikimedia.org/__mailman/listinfo/wiki-__research-l <https://lists.wikimedia.org/mailman/listinfo/wiki-research-l>
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
wiki-research-l@lists.wikimedia.org