[Wikimedia-l] AFT5: what practical benefits has it had?

[Dario Taraborelli]

> Thank you for enabling it again. I had read about the blind tests in <https://meta.wikimedia.org/wiki/Research:Article_feedback/Quality_assessment> before but I see some major changes in the graphs, which are a bit hard to understand.
> 1) In "Daily moderation actions (percentage)" there's a huge spike of helpful/unhelpful after C (July), did those flags even exist before? Or did helpfulness increase after wider usage according to the finding «the average page receives higher quality feedback than pages picked for their popularity/controversial topic»? (There's no change between 5 and 10 % though.)
> *They did; the spike is most probably caused by a deployment from 0.6 percent of articles to 5 percent of articles, with a resulting "ooh, shiny! Lets take a look" reaction.

the spike actually follows the 5% deployment combined with the CentralNotice announcement (see annotation D in the first plot), the latter is almost certainly what caused the spike
> 
> 2) "Unique daily articles with feedback moderated" shows a spike and then a stabilization, but I don't know what the graphs actually is about. For instance, can feedback be moderated per article ("feedback semi/full protection" or so) or only per item, etc. Do you know if moderation happens on the same articles and if stricter moderation increases helpfulness of feedback also on non-moderated articles?
>  
> *So, I believe it means "the number of distinct articles which have had feedback moderated that day", regardless of whether people use the article-specific page or the centralised page, but I'm not sure - some clarification from Dario would be awesome :). Ditto your other questions, particularly on the distribution of articles. 

what Oliver said, we are not keeping track of the source of moderation activity at the moment but I agree it would be a very important piece of data to analyze. After consulting with Fabrice, I've opened this ticket on bugzilla so we can assess the effort needed to implement this via the logging table: https://bugzilla.wikimedia.org/show_bug.cgi?id=41061

Dario