Thanks Dan, to be clear, the proposal is not to develop another manual rating system (such as the AFT or the project rating systems), it’s to develop some automated quality assessments. Those might include some manual elements as inputs particularly for any machine learning approach, but generating new methods there is not the aim of the project.

Cheers

Simon

From: wikimediauk-l-bounces@lists.wikimedia.org [mailto:wikimediauk-l-bounces@lists.wikimedia.org] On Behalf Of Deskana
Sent: 16 April 2014 19:24
To: UK Wikimedia mailing list
Subject: Re: [Wikimediauk-l] [Wikimedia-l] Rating Wikimedia content (was Our next strategy plan-Paid editing)

Something similar was tried at the Wikimedia Foundation, with the Article Feedback Tool. This all happened before I joined the WMF, and I wasn't aware of it in my time as a volunteer, so I'm mostly reciting what others have told me here. Take what I say with a pinch of salt.

There were two major problems that occurred with the Article Feedback Tool:

1) Way, way more feedback was generated than could be handled.

2) People often ended up rating the subject of the page rather than the content in it (e.g. [[Justin Bieber]] got lots of 1s and 5s).

It would be wise to think about ways to mitigate these problems from the very start, so that they don't occur again. I'm happy to work with you on these, as much as my time allows.

Thanks,

Dan

On 16 April 2014 10:25, Michael Maggs <Michael@maggs.name> wrote:

On 26 Mar 2014, at 21:35, Andrew Gray <andrew.gray@dunelm.org.uk> wrote:

<snip>

It would be great if this sort of rating was being systematically checked -
but at a vague estimate of thirty seconds to scan, grade, and tag,
aggregated across all pages on enwiki, that's about fifteen or twenty
person-years of work to do it as a once-off, much less a rolling process.

Andrew.

On 25 March 2014 23:35, Pete Forsyth <peteforsyth@gmail.com> wrote:

Philippe,

The Public Policy Initiative produced strong validation for the Wikipedia
1.0 approach to assessing article quality. Was Amy Roth's research ever
published, and are there any plans to repeat it with a larger sample size
etc.? I'd say we're closer than you think to having a good way to measure
article quality.

Pete
[[User:Peteforsyth]]

There is at present no comprehensive automated tool that can be used to measure article and media file quality. Measuring quantity is easy; quality much more difficult.

At the Wikimedia Conference over the weekend I presented some thoughts about a possible software project, to be lead by Wikimedia UK, to tackle this.

A review of the presentation, and slides, can be seen at: https://meta.wikimedia.org/wiki/Wikimedia_Conference_2014/Documentation/24#Michael_Maggs_.28WMUK.29_-_WikiRate:_rating_Wikimedia

The WMUK wiki page is here:  https://wikimedia.org.uk/wiki/Technology_Committee/Project_requests/WikiRate_-_rating_Wikimedia

Comments and feedback are most welcome.  In particular, we would like to know whether creating such tools would be considered a useful thing to do by the community.

Best regards

Michael

____________

Michael Maggs

Chair, Wikimedia UK 

_______________________________________________
Wikimedia UK mailing list
wikimediauk-l@wikimedia.org
http://mail.wikimedia.org/mailman/listinfo/wikimediauk-l
WMUK: https://wikimedia.org.uk