Thank you for enabling it again. I had read about the
blind tests in
<https://meta.wikimedia.org/wiki/Research:Article_feedback/Quality_assessment>
before but I see some major changes in the graphs, which are a bit hard to understand.
1) In "Daily moderation actions (percentage)" there's a huge spike of
helpful/unhelpful after C (July), did those flags even exist before? Or did helpfulness
increase after wider usage according to the finding «the average page receives higher
quality feedback than pages picked for their popularity/controversial topic»? (There's
no change between 5 and 10 % though.)
*They did; the spike is most probably caused by a deployment from 0.6 percent of articles
to 5 percent of articles, with a resulting "ooh, shiny! Lets take a look"
reaction.
the spike actually follows the 5% deployment combined with the CentralNotice announcement
(see annotation D in the first plot), the latter is almost certainly what caused the
spike
2) "Unique daily articles with feedback moderated" shows a spike and then a
stabilization, but I don't know what the graphs actually is about. For instance, can
feedback be moderated per article ("feedback semi/full protection" or so) or
only per item, etc. Do you know if moderation happens on the same articles and if stricter
moderation increases helpfulness of feedback also on non-moderated articles?
*So, I believe it means "the number of distinct articles which have had feedback
moderated that day", regardless of whether people use the article-specific page or
the centralised page, but I'm not sure - some clarification from Dario would be
awesome :). Ditto your other questions, particularly on the distribution of articles.
what Oliver said, we are not keeping track of the source of moderation activity at the
moment but I agree it would be a very important piece of data to analyze. After consulting
with Fabrice, I've opened this ticket on bugzilla so we can assess the effort needed
to implement this via the logging table:
https://bugzilla.wikimedia.org/show_bug.cgi?id=41061
Dario