This topic comes in handy for my research on Featured Articles in WP:PT. Maybe some of you may remember my request a little while ago about studies on Wikipedias other than English. Well, not that I believe that the Featured Article requirements are a good evaluation per se, in terms of quality of content. Anders, what are the articles you evaluated? I'm curious to find out what was so bad in the Portuguese Wikipedia. Indeed, there are many problems there, but I'm surprised to hear that it looks so bad. I know it's a drop in the ocean, but I've been fixing some new articles that are translations from bad English ones - which look good, but analyzing the content reveals many problems. Juliana. On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten &lt;mail(a)anderswennersten.se > Thanks for answer > > Your answer confirm my "fear", that focus is almost completly to en:wp and > how it is compared with an ideal perfect Q > > My interest and what I believe the movement need before we dig into next > round of strategy round is > *what versions are dysfunctional. These represent a risk for the movement > as they can jeoprdaize the brand name, as they are not living up to basic Q > (and NPOV) > *what can we learn from each other, why are some better in some aspects > and worse in others? > > I would recommend a research approach much more basic just collecting some > few data on each version (and forget about enwp) > > Anders > > > > Heather Ford skrev 2014-06-10 13:09: > > Hi Anders, > > Yes, it's a great question! Mark Graham and I are currently working on a > project around how to determine quality within and between Wikipedias and > I've been looking around for literature. I'm only just starting the > literature review but I've found some interesting studies by Callahan & > Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority > of quality studies, we find, have been done on English Wikipedia (starting > with the famous 2005 Nature study) but there have been few studies that > assess of quality between languages. If you find anything else, let us > know! > > Thanks! > > Best, > heather. > > [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract > > [2] > http://www.researchgate.net/publication/200773220_Issues_of_cross-contextua… > > > > > > Heather Ford > Oxford Internet Institute <http://www.oii.ox.ac.uk> Doctoral Programme > EthnographyMatters <http://ethnographymatters.net> | Oxford Digital > Ethnography Group <http://www.oii.ox.ac.uk/research/projects/?id=115> > http://hblog.org | @hfordsa <http://www.twitter.com/hfordsa> > > > > > On 10 June 2014 07:58, Anders Wennersten &lt;mail(a)anderswennersten.se> >> (reposted from Wikimedia-i) >> >> I have several times asked for a professional quality study of our >> different language versions, but not seen it exist or being done, perhaps >> you know more on this list?. before we start the strategy work I believe >> we should have basic facts on the table like this one >> >> I therefor list here my subjective impression after daily looking into >> the different version for 5-15 articles (new ones being created on sv.wp) >> (I list them in order how often I use them to calibrate the svwp articles). >> >> enwp- a magnitude better then any other. main weakeness are articles on >> marginal subjects that seems to be allowed to exist there, even if rather >> bad, and without templates (noone cares to patrol these?) >> >> eswp - a very good version, which in the general discussion are not >> getting appropriate credit >> >> dewp - good when the articles exist, but many serious holes. Is the >> elitist way of running it, discouraging new editors in non obvious subjects >> (that after time passes gets very relevant)? >> frwp - also good, but somewhat scattered quality both in coverage and the >> different articles (even in same subject area) >> nlwp - very good coverage in the geographic subjects, decent quality on >> articles but limited "world" coverage in areas like biographies >> itwp - good articles but a bit italiancentered, >> >> nowp - small but decent articles. Their short focused articletext >> sometimes give more easyaccessed knowledge then an overly long one in other >> languages >> >> ptwp - the real disappointment. it is among the top ten in volume and >> accesses but clearly missing a lot, and even existing articles are uneven. >> I now use it even less then Ukrainian and Russian which I use very seldom >> as the different alphabet makes it hard to understand the article content >> >> dawp,fiwp and plwp -Ok but only used by me for articles related to the >> country >> >> (arabic, chinese and japanese I almost never use, too complicated) >> >> (I also use some smaller ones like sqwp , in these versions I have seen >> serious quality problems not to be found in any of the above ones, I am not >> sure they even have basic patrolling in place) >> >> Anders >> >> _______________________________________________ >> Wiki-research-l mailing list >> Wiki-research-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> > > > > _______________________________________________ > Wiki-research-l mailing listWiki-research-l@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > -- www.domusaurea.org

This thread seems really to be dealing with two distinct issues: coverage and quality. On the coverage side, the best work I know of is from Brent Hecht and Darren Gergle. Especially [1] but also [2,3]. The English edition may be the greatest in size, but in reality most other languages contain many unique articles not found in English. All the editions to some extent suffer from a self-focus bias where regions where the language is spoken are better represented than regions where the language is not spoken. Quality of existing articles is completely separate from coverage and is harder to evaluate at scale. I understand that some of the work at GroupLens (Minnesota) around SuggestBot is trying to develop metrics that can measure quality at scale in different languages (and working to develop/use different metrics in different languages as appropriate because not all editions have the same conventions for referencing styles, etc.). I don't have a specific reference here, but would say keep an eye on this area. ~Scott [1] Hecht, B., & Gergle, D. (2010). The Tower of Babel meets Web 2.0: User-generated content and its applications in a multilingual context. Proceedings of the 28th International Conference on Human Factors in Computing Systems (pp. 291-300). New York, NY, USA: ACM. doi: http://doi.acm.org/10.1145/1753326.1753370 [2] Hecht, B., & Gergle, D. (2009). Measuring self-focus bias in community-maintained knowledge repositories. *Proceedings of the Fourth International Conference on Communities and Technologies* (pp. 11-20). New York, NY, USA: ACM. doi:http://doi.acm.org/10.1145/1556460.1556463 [3] Bao, P., Hecht, B., Carton, S., Quaderi, M., Horn, M., & Gergle, D. (2012). Omnipedia: Bridging the Wikipedia Language Gap. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (pp. 1075-1084). New York, NY, USA: ACM. doi:10.1145/2207676.2208553 On Tue, Jun 10, 2014 at 7:34 AM, Anders Wennersten &lt;mail(a)anderswennersten.se > My starting point have been the newly created articles on svwp. They will > represent the usual bunch of football playser, tv-stars, computergames, > films etc where svwp are behind most versions but where enwp is excellent. > The interesting comparisons comes from the next levels of articles that can > be almost anything, a footballstadium in Kazan Russia, an albanian poet, a > church in Venize, a specie with unclear taxonomy, the american solider who > perhaps deserted etc. In these cases I only often find a corresponding > article in enwp, but also very often (around 20%) I find it in another > version and no presence in enwp. > > And when enwp is not giving me support, I most often find support in eswp > and frwp, sometimes in dewp, but almost never in ptwp. For exemple > taxanomical threes with name in native and latin is about the weakest in > ptwp. But I can be wrong and I would love to be part in a more complete > research on Q comparisons for the different versions > > > Anders > > > > > > Juliana Bastos Marques skrev 2014-06-10 14:06: > > This topic comes in handy for my research on Featured Articles in WP:PT. > Maybe some of you may remember my request a little while ago about studies > on Wikipedias other than English. Well, not that I believe that the > Featured Article requirements are a good evaluation per se, in terms of > quality of content. > > Anders, what are the articles you evaluated? I'm curious to find out > what was so bad in the Portuguese Wikipedia. Indeed, there are many > problems there, but I'm surprised to hear that it looks so bad. I know it's > a drop in the ocean, but I've been fixing some new articles that are > translations from bad English ones - which look good, but analyzing the > content reveals many problems. > > Juliana. > > > On Tue, Jun 10, 2014 at 8:29 AM, Anders Wennersten < > mail(a)anderswennersten.se> >> Thanks for answer >> >> Your answer confirm my "fear", that focus is almost completly to en:wp >> and how it is compared with an ideal perfect Q >> >> My interest and what I believe the movement need before we dig into next >> round of strategy round is >> *what versions are dysfunctional. These represent a risk for the movement >> as they can jeoprdaize the brand name, as they are not living up to basic Q >> (and NPOV) >> *what can we learn from each other, why are some better in some aspects >> and worse in others? >> >> I would recommend a research approach much more basic just collecting >> some few data on each version (and forget about enwp) >> >> Anders >> >> >> >> Heather Ford skrev 2014-06-10 13:09: >> >> Hi Anders, >> >> Yes, it's a great question! Mark Graham and I are currently working on a >> project around how to determine quality within and between Wikipedias and >> I've been looking around for literature. I'm only just starting the >> literature review but I've found some interesting studies by Callahan & >> Herring (2011) [1] and Stvilia, Al-Faraj, and Yi (2009) [2]. The majority >> of quality studies, we find, have been done on English Wikipedia (starting >> with the famous 2005 Nature study) but there have been few studies that >> assess of quality between languages. If you find anything else, let us >> know! >> >> Thanks! >> >> Best, >> heather. >> >> [1] http://onlinelibrary.wiley.com/doi/10.1002/asi.21577/abstract >> >> [2] >> http://www.researchgate.net/publication/200773220_Issues_of_cross-contextua… >> >> >> >> >> >> Heather Ford >> Oxford Internet Institute <http://www.oii.ox.ac.uk> Doctoral Programme >> EthnographyMatters <http://ethnographymatters.net> | Oxford Digital >> Ethnography Group <http://www.oii.ox.ac.uk/research/projects/?id=115> >> http://hblog.org | @hfordsa <http://www.twitter.com/hfordsa> >> >> >> >> >> On 10 June 2014 07:58, Anders Wennersten &lt;mail(a)anderswennersten.se&gt; >>> >>> (reposted from Wikimedia-i) >>> >>> I have several times asked for a professional quality study of our >>> different language versions, but not seen it exist or being done, perhaps >>> you know more on this list?. before we start the strategy work I believe >>> we should have basic facts on the table like this one >>> >>> I therefor list here my subjective impression after daily looking into >>> the different version for 5-15 articles (new ones being created on sv.wp) >>> (I list them in order how often I use them to calibrate the svwp articles). >>> >>> enwp- a magnitude better then any other. main weakeness are articles on >>> marginal subjects that seems to be allowed to exist there, even if rather >>> bad, and without templates (noone cares to patrol these?) >>> >>> eswp - a very good version, which in the general discussion are not >>> getting appropriate credit >>> >>> dewp - good when the articles exist, but many serious holes. Is the >>> elitist way of running it, discouraging new editors in non obvious subjects >>> (that after time passes gets very relevant)? >>> frwp - also good, but somewhat scattered quality both in coverage and >>> the different articles (even in same subject area) >>> nlwp - very good coverage in the geographic subjects, decent quality on >>> articles but limited "world" coverage in areas like biographies >>> itwp - good articles but a bit italiancentered, >>> >>> nowp - small but decent articles. Their short focused articletext >>> sometimes give more easyaccessed knowledge then an overly long one in other >>> languages >>> >>> ptwp - the real disappointment. it is among the top ten in volume and >>> accesses but clearly missing a lot, and even existing articles are uneven. >>> I now use it even less then Ukrainian and Russian which I use very seldom >>> as the different alphabet makes it hard to understand the article content >>> >>> dawp,fiwp and plwp -Ok but only used by me for articles related to the >>> country >>> >>> (arabic, chinese and japanese I almost never use, too complicated) >>> >>> (I also use some smaller ones like sqwp , in these versions I have seen >>> serious quality problems not to be found in any of the above ones, I am not >>> sure they even have basic patrolling in place) >>> >>> Anders >>> >>> _______________________________________________ >>> Wiki-research-l mailing list >>> Wiki-research-l(a)lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >>> >> >> >> >> _______________________________________________ >> Wiki-research-l mailing listWiki-research-l@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> >> >> _______________________________________________ >> Wiki-research-l mailing list >> Wiki-research-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l >> >> > > > -- > www.domusaurea.org > > > _______________________________________________ > Wiki-research-l mailing listWiki-research-l@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > > > _______________________________________________ > Wiki-research-l mailing list > Wiki-research-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wiki-research-l > > -- Scott Hale Oxford Internet Institute University of Oxford http://www.scotthale.net/ scott.hale(a)oii.ox.ac.uk

Ziko, could you please supply the full reference to Haber's article? Indeed, what I observe in History-related articles is almost a tendency towards positivism, histoire événementielle - hence, for instance, the vast number of battle themes. Discussion of historiographic approaches to concepts is usually quite rare and badly written. Anyone willing to conduct comparative research on quality of History-related articles, please drop me a note! Juliana. On Tue, Jun 10, 2014 at 11:12 AM, Ziko van Dijk &lt;zvandijk(a)gmail.com&gt; wrote: -- www.domusaurea.org

Thanks a lot for all feedback and links I will look into. My feeling, though, is that I am more interested in what is "below the belt", ie rottenness creeping into different versions, more then the more acceptable quality aspect we are used looking into. I will see if I, after reading your links, can formulate what I would like to see (which perhaps I even an do myself) in a kind a "research proposal" and hope I can preesnt it here to get feedback on it Anders Ziko van Dijk skrev 2014-06-10 16:12:

Totally agree with you, Kerry - that there are *very* different ideas about what constitutes quality. The large diversity in research about quality taking very different variables into account is testament to that. I'm interested in your note about page views after Google Knowledge Graph. According to these stats http://stats.wikimedia.org/EN/SummaryEN.htm. Page views went up from mid-2012 to beginning of 2013 and then they went down quite sharply but seem to start rising again at the end of 2013. But perhaps you're seeing other data? Would love to hear your thoughts! btw, for those asking about historiography, Brendan Luyt [1] has done some great work on how Wikipedia represents dominant and alternative historiographies [e.g. 2]. And thanks, Finn, for the great work you continue to do with Wikilit :) Best, Heather. [1] http://scholar.google.co.uk/citations?user=Xhl9P7oAAAAJ&hl=en [2] http://onlinelibrary.wiley.com/doi/10.1002/asi.21531/full Heather Ford Oxford Internet Institute <http://www.oii.ox.ac.uk> Doctoral Programme EthnographyMatters <http://ethnographymatters.net> | Oxford Digital Ethnography Group <http://www.oii.ox.ac.uk/research/projects/?id=115> http://hblog.org | @hfordsa <http://www.twitter.com/hfordsa> On 11 June 2014 01:19, Kerry Raymond &lt;kerry.raymond(a)gmail.com&gt; wrote:

Actually, we're still doing work on whether the KnowledgeGraph stuff is having an impact, or whether it's a problem with the data source, or, or.. On 11 June 2014 14:21, Kerry Raymond &lt;kerry.raymond(a)gmail.com&gt; wrote: -- Oliver Keyes Research Analyst Wikimedia Foundation

Dear Heather, In our WikiLit systematic reviews we found a few publications. I have just made a semantic query on the WikiLit site to give you an overview: http://wikilit.referata.com/wiki/WikiLit:Quality There are not that many. You should find them described in our review on research on Wikipedia content: "The sum of all human knowledge": a systematic review of scholarly research on the content of Wikipedia http://spectrum.library.concordia.ca/978618/1/WikiLit_Content_%2D_open_acce… http://neuro.compute.dtu.dk/wiki/%22The_sum_of_all_human_knowledge%22:_a_sy… best Finn Årup Nielsen On 06/10/2014 01:09 PM, Heather Ford wrote: