Hi Yaroslav,
Yes it was certainly my hope that one of the roles of RCom would be to
identify issues in the community where independent research would be
really useful.
As for the specific issue, I've raised concerns about AFT on more than
one occasion, and I've had assurances that they will keep an eye on
whether it is doing more harm than good. Though my preferred option of
measuring the 100,000 test articles against a control sample of
100,000 and seeing which articles received more improvement doesn't
seem to have happened.
AFT is an interesting test of several of what some of us believe to be
cornerstones of the wiki; Firstly accountability, the theory that
people with registered accounts edit positively in order to conserve a
reputation that they acquire through having their edits logged to
them. So it isn't just the practical concern that racist or homophobic
rating of articles can't be dealt with as easily as blocking editors
who make racist or homophobic edits, there is the theoretical
possibility that anonymous ratings would give otherwise goodfaith
editors a chance to vent their spleen against editors they've clashed
with by giving poor ratings to their articles. Now this wouldn't make
much difference if we were only displaying ratings where dozens of
different IP addresses had rated the article, but for most of our
articles hoping for a dozen ratings would be optimistic.
Secondly, and in my view far more seriously, if like me you buy the
idea that the decline of "sofixit" and the rise of the templating
culture in the last four years is a major cause of the community
peaking and starting to decline, then such a major attempt to further
shift people from improving articles to merely critiquing them is very
unhelpful.
Thirdly and much more variably by language, we need to remember that
for many of our editors a significant motive for editing Wikipedia is
the opportunity to practice and improve a language they are learning.
I suspect this will be a bigger issue for the English language wiki
than for most others. Assuming that AFT will exacerbate the trend of
shifting people from fixing things to critiquing them, then it would
be worthwhile getting some research to see how much this demotivates
those of our editors who are not native speakers. If you contribute
something and someone else fixes typos and grammar you are liable to
learn something, if they just rate it as poorly written you are left
wondering why.
Of course it's possible that the extra editors this brings in will
offset the damage, at least for a short period before people get blasé
with the rating template and start to tune it out.
So this could be a great opportunity for an independent researcher to
test and prove or disprove the criticisms.
WereSpielChequers
On 30 July 2011 08:01, Yaroslav M. Blanter <putevod(a)mccme.ru> wrote:
Dear all,
I always thought that the role of the Rcom would be not only managing the
incoming requests for research (in a broad sense), but also identifying
what research results we would like to have in order to facilitate the
future development of the WMF projects.
Recently, AFT has been deployed on every article in en.wp, which caused
some controversy. Users are unsure on what the rankings actually mean (for
instance, if the readers actually answer the questions they are asked, or
instead give rankings based on their perception of the subject; whether it
would be a good idea to invite the readers also to provide text input, and
a number of other issues).
To me it looks like a good test case. It would be good for us to have some
research results on the issue, and so far (from what I know) nobody
volunteered to perform such research. Should WMF then indicate that they
want these results? Probably at this stage we are not prepared to order it
(I mean to pay for it), but it could be broadly advertised in certain
places.
Just to make sure, what I asked is not one but two separate questions:
1) Do we (badly) want the research on AFT?
Even if the answer is no (not needed, too early, unrealistic etc), it
makes sense in my opinion to discuss another question:
2) Is is appropriate and/or meaningful for us as Rcom to compile a
priority list of needed research topics and try to advertise the necessity
and urgency of research carried out on these topics?
Note that even if the answer to 1 is yes the answer to 2 can still be no
(as a matter of principle).
Cheers
Yaroslav
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l