[WikiEN-l] Article Feedback - Ramp up to 10% of Articles

[WereSpielChequers]

Re Ray's comment:

> It's difficult to see any logical connection between an article rating
> system, and encouraging new editors.

I'm not convinced that we fully understand all the different things
that made Wikipedia work, and especially what are the elements that
didn't motivate us individually but are important to others. Two
aspects that really attracted me to the pedia were firstly the SoFixIt
approach, rather than write a paragraph somewhere explaining why I
thought something was wrong I could just make a change and see if
others accepted it. Secondly the correction of my errors. Instead of
someone red-penning my work I much prefer that they just fix isolated
errors - if I'm watchlisting the article I have the opportunity to see
what gets changed.  Similarly I don't whinge at people who make the
typos that I fix, I just fix them. I suspect this is part of the
motivation for many of our editors who edit in a language other than
their native one, editing Wikipedia gives them an opportunity to
practice a language in a collaborative environment where their
mistakes are fixed in a non-judgmental way. I'm not convinced that
such editors would benefit if there was a switch from collaborative
editing where they have a chance to improve their English to article
assessment where they are marked down without getting specific
feedback as to how to improve.

So for me there are two close and logical connections between article
rating and editing. But I appreciate that others may see them as
unconnected, and I agree that without testing we can't easily find out
how important these connections are or indeed whether this will divert
people from collaborative editing to critiquing or simply attract
additional involvement..

My concern about the article rating system is that it could undermine
two important parts of what I perceive to be our foundations. I'm
reassured that there will be testing to see whether this does in
practice what I suspect it will, and indeed whether we can do
sufficient call to actions to persuade some of our new article
assessors to actually edit.

WSC

On 15 July 2011 10:28, Ray Saintonge <saintonge at telus.net> wrote:
> On 07/14/11 5:56 AM, WereSpielChequers wrote:
>> Do we have stats yet that measure whether this is encouraging editing,
>> or diverting even more people from improving the pedia to critiquing
>> it?
>
> It's difficult to see any logical connection between an article rating
> system, and encouraging new editors.
>> Remember there is a risk that this could exacerbate the templating
>> trend. Just as we need to value edits that fix problems and remove
>> templates above edits that add to the hundreds of thousands of
>> maintenance templates on the pedia;
>
> If templates were subject to a similar rating system as articles we
> would soon see which are being ignored by users, and are thus of no value.
>
>> So we need to value a talkpage
>> comment that explains why someone has a specific concern about an
>> article over a bunch of "feedback" that says people like or dislike an
>> article without indicating why. Better still we should be encouraging
>> readers to improve articles that they see as flawed.
>
> This dream has been around since the stone age.
>
>> So we need to
>> measure this tool in terms of its success at getting readers to edit,
>> not in terms of its success at getting readers to rate articles. I
>> hope it is successful, and I'm happy to take the long view and measure
>> a trial over months to see how effectively we convert article raters
>> into article editors.
>
> I seriously doubt that it will head in that direction.
>> But we do need to be prepared to remove this if
>> it has a net effect of diverting potential editors into merely rating
>> articles for others to fix.
>
> It's not a problem if they do. If many readers do this for a single
> article it's worth paying attention to these articles. A claim from a
> single person can be suspected eccentric.
>> We also need to be careful how we compare this 374k to the other
>> "90%", not least because with 3,682,158 articles on En wiki as I
>> write, 374k is about 6k more than a random 10% sample would be.
>
> It's all a matter of statistical trends, and for this a 100-point scale
> would have been more useful than a 5-point scale. I actually suggested a
> 10-point scale many years ago. The first statistical measure that should
> develop is a cumulative rating for all articles. The mean in that will
> be the measure of the average article, and any article falling within a
> certain deviation from that could be judged average.  As overall quality
> of WP increases so too will the average rating, but only extremely
> slowly. Other measures could be developed from there.
>> We also need to learn from one of the lessons of the Strategy wiki
>> where we had a similar rating system. Many of the proposals there had
>> so few ratings that they were close to being individual views and few
>> had sufficient responses to be genuinely collective to the point where
>> one maverick couldn't skew them - even without sockpuppetry. On
>> average our articles get one or two edits a month, many get far less.
>> I would not be surprised if 100,000 of the 374k in the trial had less
>> than ten ratings even if trialled for a couple of months.
>
> This isn't a problem either.  The number of ratings given is just as
> important as what those ratings are.  It should be reported right along
> with the rating on the article page  Users could then be reminded that a
> small number of ratings is just not statistically significant; they
> could even be color-coded to that effect. Short samples are also more
> volatile.  They would easily be driven into the top or bottom decile of
> the data, and that alone would bring attention to them.
>> Lastly we need to be prepared for sockpuppetry, especially as these
>> are random unsigned votes with no rationale. Can we have assurances
>> that something is being built into the scheme to combat this?
>
> This FUD gives undue weight to sockpuppetry or other hostile editing.
> Ideally such practices should be marginalised to a point where they
> don't matter. Mounting a successful campaign to influence the rating of
> an article would take a tremendous amount of sustained effort. I played
> with trying to affect the page views of one of the Bomis girl articles
> in the early days by going repeatedly to that page; the effects were
> minimal. Now, with a much bigger encyclopedic corpus this would be
> proportionally more difficult. "Random unsigned votes" are perfectly
> consistent with wikiness, and will also trend toward statistical norms.
> Building safeguards against agenda based ratings would be a waste of
> time and effort.
>
> Ec
>
> _______________________________________________
> WikiEN-l mailing list
> WikiEN-l at lists.wikimedia.org
> To unsubscribe from this mailing list, visit:
> https://lists.wikimedia.org/mailman/listinfo/wikien-l
>