Hey Ismael,
Thanks for the question. Thoughts below:
Do we have tools, metrics or traces about the
evolution of quality in
articles?
This is my attempt at a brief summary at some of the aspects of your
question though I have written a good bit more about this here:
https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articl…
** How is quality measured? **
There are two main sources of quality data: community assessments and
synthetic model predictions. The former are actual assessments by
Wikipedians and so can capture much more nuance -- e.g., not just length of
the article but how well is it written? how important are images or
references to that language community? The downside of these scores is that
it's near impossible to keep up with the rate of change of Wikipedia and so
they can be sparse and outdated. Also, the talk-page template/PageAssessment
<https://www.mediawiki.org/wiki/Extension:PageAssessments> ecosystem for
recording these scores generally just allows for recording of the actual
quality class without clear details about what it would take to improve
that score. To fill these gaps, many folks have worked on developing models
for assessing quality on Wikipedia. There are many ways to distinguish the
models but the one that is most salient to me is how may languages does the
model support? I personally developed a model for assessing article quality
in any language of Wikipedia (details
<https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Language-Agnostic_Quality#V2>)
that is pretty simple but seems to do a good job and is heavily inspired by
WikiRank <https://meta.wikimedia.org/wiki/WikiRank>. The caveat is that the
scores are not directly comparable across language editions (which is going
to be true of most models and community assessments for that matter -
quality is complex).
** Evolution over time **
Gathering / generating quality data over time is a scaling headache. I
personally have not worked on this part but will point to an initiative by
one of my colleagues Diego Sáez-Trumper and collaborator Paramita Das that
worked on scalability:
https://meta.wikimedia.org/wiki/Research:Cross-lingual_article_quality_asse…
** Tools / metrics **
There is an ongoing project that would provide dashboards that, among other
things, would allow you to explore the evolution of article quality across
various important knowledge gaps areas on Wikipedia. It's not ready yet but
you can read up, let us know if you have specific use-cases, and track
progress:
https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Measurement
Hope that helps!
Best,
Isaac
On Fri, Jan 13, 2023 at 5:27 AM Ismael Olea <ismael(a)olea.org> wrote:
>
> Hi:
>
Do we have tools, metrics or traces about the
evolution of quality in
> articles? Or something like that. Not sure if the ORES
technology is
> appropriate for it.
>
> --
>
> Ismael Olea
>
>
http://olea.org/diario/
> _______________________________________________
> Analytics mailing list -- analytics(a)lists.wikimedia.org
> To unsubscribe send an email to analytics-leave(a)lists.wikimedia.org
>
--
Isaac Johnson (he/him/his) -- Senior Research Scientist -- Wikimedia
Foundation