Hi:
Do we have tools, metrics or traces about the evolution of quality in articles? Or something like that. Not sure if the ORES technology is appropriate for it.
Hey Ismael, Thanks for the question. Thoughts below:
Do we have tools, metrics or traces about the evolution of quality in
articles? This is my attempt at a brief summary at some of the aspects of your question though I have written a good bit more about this here: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Article...
** How is quality measured? ** There are two main sources of quality data: community assessments and synthetic model predictions. The former are actual assessments by Wikipedians and so can capture much more nuance -- e.g., not just length of the article but how well is it written? how important are images or references to that language community? The downside of these scores is that it's near impossible to keep up with the rate of change of Wikipedia and so they can be sparse and outdated. Also, the talk-page template/PageAssessment https://www.mediawiki.org/wiki/Extension:PageAssessments ecosystem for recording these scores generally just allows for recording of the actual quality class without clear details about what it would take to improve that score. To fill these gaps, many folks have worked on developing models for assessing quality on Wikipedia. There are many ways to distinguish the models but the one that is most salient to me is how may languages does the model support? I personally developed a model for assessing article quality in any language of Wikipedia (details https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Language-Agnostic_Quality#V2) that is pretty simple but seems to do a good job and is heavily inspired by WikiRank https://meta.wikimedia.org/wiki/WikiRank. The caveat is that the scores are not directly comparable across language editions (which is going to be true of most models and community assessments for that matter - quality is complex).
** Evolution over time ** Gathering / generating quality data over time is a scaling headache. I personally have not worked on this part but will point to an initiative by one of my colleagues Diego Sáez-Trumper and collaborator Paramita Das that worked on scalability: https://meta.wikimedia.org/wiki/Research:Cross-lingual_article_quality_asses...
** Tools / metrics ** There is an ongoing project that would provide dashboards that, among other things, would allow you to explore the evolution of article quality across various important knowledge gaps areas on Wikipedia. It's not ready yet but you can read up, let us know if you have specific use-cases, and track progress: https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Measurement
Hope that helps!
Best, Isaac
On Fri, Jan 13, 2023 at 5:27 AM Ismael Olea ismael@olea.org wrote:
Hi:
Do we have tools, metrics or traces about the evolution of quality in articles? Or something like that. Not sure if the ORES technology is appropriate for it.
--
Ismael Olea
http://olea.org/diario/ _______________________________________________ Analytics mailing list -- analytics@lists.wikimedia.org To unsubscribe send an email to analytics-leave@lists.wikimedia.org