[Foundation-l] Office Hours on the article feedback tool

[WereSpielChequers]

> ------------------------------
>
> Message: 6
> Date: Wed, 26 Oct 2011 11:11:57 +0100
> From: Oliver Keyes <scire.facias at gmail.com>
> Subject: Re: [Foundation-l] Office Hours on the article feedback tool
> To: Wikimedia Foundation Mailing List
>        <foundation-l at lists.wikimedia.org>
> Message-ID:
>        <CAPYupWA34CujYan_vV_cHgYxWfCT3EJnb4d-Nrav_U20QejZ1A at mail.gmail.com
> >
> Content-Type: text/plain; charset=ISO-8859-1
>
> No, the data will remain; you can find it at
> http://toolserver.org/~catrope/articlefeedback/ (we really need to
> advertise
> that more widely, actually).
>
> To be clear, we're not talking about junking the idea; we will still have
> an
> "Article Feedback Tool" that lets readers provide feedback to editors. The
> goal is more to move away from a subjective rating system, and towards
> something the editors can look at and go "huh, that's a reasonable
> suggestion as to how to fix the article, I'll go do that" or "aw, that's
> really nice! I'm glad they liked it so much"
>
> O.
>
>
As someone who was never exactly a fan of the Article Feedback Tool I'm glad
to hear that the current version is to be canned. The sort of subjective
ratings it could produce were never going to be useful at improving
articles, certainly not useful enough to justify the screen space. My fear
was that it might divert people from improving articles to complaining about
them. Since we skipped a key stage in the testing we will never know whether
it did that. I didn't realise at the time that it was going to abuse our
readers trust by collecting shed loads of data that we weren't going to use.

We took a big risk in implementing the Article Feedback Tool without first
testing to see whether it would do more harm than good. It is hard to tell
in hindsight whether it has been negative or neutral in effect. Yes
recruitment of new editors has fallen sharply - September's new editors on
EN wiki are down to levels not seen since 2005
http://stats.wikimedia.org/EN/TablesWikipediaEN.htm#editdistribution but
things were on the decline anyway so we don't know whether and to what
extent the Article Feedback tool exacerbated the trend. My concern about
turning it into something that collects more meaningful comments is that
this could exacerbate the pernicious trend from improving articles to
tagging them for others to improve. I appreciate that there are various
competing theories as to why the community went off the boil circa 2007, but
for me and anyone else who considers that the trend to template rather than
improve articles has been a major cause of community decline, an "improved"
version of the Article Feedback Tool is a worrying prospect.

Can we make sure that any new generation Article Feedback tool is properly
tested, and that testing includes:

   1. Implementing it on a random group  of articles and comparing them with
   a control sample to see which group of articles had the more edits from
   newbies;
   2. Whether the collecting of feedback on ways to improve the article
   generates additional comments or diverts some editors away from actually
   fixing the article.
   3. Which group of articles recruited the most new editors to the pedia.

Please don't implement it if the testing shows that it diverts people from
fixing articles to pointing out things that others can fix.

On a broader note I suggested some time ago that for the community to give
meaningful input into article development we need a process for the
community to give feedback on the priority of various potential
developments. Wikimania does something like that in the way the program is
put together. The image filter "referendum" came close in that it asked
people to rate the image filter for importance, unfortunately it didn't
include other proposals so that people could put them in order of relevant
importance (we also need a quite separate question for whether you think
something is worth doing at all). In your new role as liaison between the
community and the development team please could you initiate something like
that, so that those of us who would give a higher priority to global
watchlists or enhancing catalot so that it works on uncategorised articles
can say so?

Regards

WereSpielChequers