[Wikiquality-l] Implicit vs Explicit metadata

[Luca de Alfaro]

Aaron's message is very good.  I just added a few comments, especially on
the part "no one commits to check text when editing".
Luca

On Dec 2, 2007 11:13 AM, Aaron Schulz <jschulz_4587 at msn.com> wrote:

>  *Bah, I meant to send this here, not to just one person...*
>
> OK, in order to talk about pros vs. cons, we need to consider the uses
> first. Some main tasks are:
> 1) selected an unvandalized version (for AT, this will do "worse part"
> checks and such)
> 2) selecting a quality, fact-checked version (German Wikipedia wants this)
> 3) selecting a consensus version/marking featured pages
> 4) selecting the best version and displaying it be *default*
>
> *Selecting an unvandalized version*
> Flagged Revisions (pros):
> 1) Templates and images are part of the review process, so vandalism to
> them will not show for reviewed pages
> 2) Users with review rights get the gratification of setting the latest
> unvandalized/"sighted" version
> 3) New users can look forward to getting these rights in short order,
> after being considered trusted
> 4) Edits by reviewers can be autoreviewed if they are to a page where the
> stable and current are synced.
> 5) If 4 above is not possible, a diff of the changes to the stable are
> shown after edit to reviewers with a review form with the tags preselected.
> It shouldn't take long at all to glance over the changes and click "review".
>
> Flagged Revisions (cons)
> 1) Initial review takes noticeable time for non-stub pages
>
> 2) Revisions can fall out of date if not maintained, so people clicking to
> see a stable version may get a really old one. This is integrated with the
> RC patrolling system and with the autoreviewing/quick diffs to help, but it
> is still a possibility.
>
> Article Trust(pros)
> 1) No workload added, all automatic. This is very nice.
> 2) Fast and fluid since calculations for the sighted version are done on
> every edit without anyone having to do anything
> 3) Accounts for consensus, so no rouge reviewer can easily flag garbage.
> Still, a "trusted" user can go rouge and add garbage, even in several edits
> to bump the trust.
>
> Article Trust(cons)
> 1) Template and image vandalism is still a problem
> 2) Bot and AWB edits flying through pages automatically make the trust of
> large chunks of text increase
>

True, but  it would not be hard to fix this in the code.

>
> 3) No direct control over it by anyone -> incentive loss
>
> *Selecting a quality, fact-checked version*
> Flagged Revisions (pros):
> 1) Trusted users, who have some respect for consensus as well, can
> directly mark off solid revisions
>
> Flagged Revisions (cons):
> 1) If the user goes rouge they can flag garbage. Not as bad as rouge
> admins, likely rare, but something to think about...
> 2) A roughish user may ignore consensus and reasonable fact disputes. This
> could result in a small user or cabal having a monopoly over the "best
> version". Good policy standards and respect should be enforced to avoid
> this.
>
> Article Trust(pros):
> 1) Nearly all "white" pages have a good chance of being reasonable
> accurate
> 2) No work required
> 3) Harder to form cabals/monopolies over the "best version"
>
> Article Trust(cons):
> 1) Again, bots and such
>
Again, bots are not so hard to remedy.

>
> 2) You cannot edit anything without vouching for it (bad for fixing typos)
> and the text around it. Either people get afraid to edit or dubious text
> gets more and more "trusted"
>

Yes, this is indeed a problem. Any suggestion?  One could add a checkbox,
when committing an edit, that says "I checked the content"; If one does not
check it, the trust of the text would increase much more slowly.  I have no
idea how using this idea would change the overall trust distribution of the
text, we would have to try it on a live wiki so see how it works.

>
> 3) No one necessarily committed to having fact-checked anything
>
>
Yes, indeed; see above.


>
> *Selecting a consensus version/Marking feature pages*
> Flagged Revisions (pros):
> 1) Trusted reviewers (higher flagging rights than normal reviewers), like
> bureaucrats, look at debates and see if a consensus for a community selected
> version exists
> 2) As long as the trusted reviewer acts like most "bureaucrats" on
> Wikipedia and just measure consensus, it is not easy to game
>
> Flagged Revisions (cons):
> 1) Rouge trusted reviewer...blah...could end up at arbcom
> 2) Not automatic...I mean look at how slow WP:FA stuff is...
>
> Article Trust(pros):
> 1) It would take a lot of users to try to edit war to push the "consensus
> version" around since it is automatic, and that would just make a bunch of
> red text to the current revision which would cause it not to be selected.
> 2) Automatic, account for all edits, not just those "voting" on some talk
> page
> 3) Generally waaay faster
>
> Article Trust(cons):
> 1) If there is consesus for a version clearly demontrated on a talk page,
> a small group of editors can still edit war over it and drop the trust. It
> would be nice for some trusted reviewer to be able to see this and
> expediently flag it.
>
> *Selecting the best version and displaying it be *default**
> Flagged Revisions (pros):
> 1) Again, template/image vandalism won't be such a problem since those are
> set for each reviewed revision based on how it was when reviewed.
> 2) For bios of living people, we can easily and immediatly set the stable
> version without having to fiddle around getting it autotrusted.
> 3) The incentive issue again, reviewer can set this, and editors can look
> forward to becoming reviewers.
>
> Flagged Revisions (cons):
> 1) Rouge trusted reviewer...blah...could end up at arbcom
> 2) Spelling/grammar errors can get stuck if no one is around to review
> corrections (though reviewers spelling fixes could be autoreviewed
> sometimes)
>
> Article Trust(pros):
> 1) The default is BY FAR the most important revision, so giving direct
> control over gives incentives to form evil cabals :)
> 2) As this is important, it helps to stay up to date with the workload,
> this requires none...so that's pretty easy...
>
> Article Trust(cons):
> 1) Spelling/grammar errors can get stuck since it's hard to directly
> control
>

You mean, the article that is automatically selected can still contain a
grammar/spelling mistake which escaped attention of previous visitors?
True.

>
> 2) The "highest least trusted" and "max age" and other heurestics will be
> confusing to new users. Default page selection will feel kind of random
>

Yes, true, there will be a lot of people who will wonder "why was A rather
than B selected", and more importantly, who will try to influence which of
various equivalently high trust versions happens to be selected.  I hope
this would not instigate "selection wars" or schemes to game the system, but
I admit I don't know.


> This is still an imcomplete list probably...I should probably save this
> somewhere and build on it.
>

Great idea.  Also, I am very interested in ways to improve article trust,
and its uses, so I am open to all suggestions (and the code will be
available this week).

Luca


> Also, for default revisions selection on page view, I am just comparing
> the methods of selection by the two extensions. We could have it where
> Flagged Revisions does the overriding of the default revision, but that it
> grabs the Article Trust "most trusted" version rather than some reviewed
> one. This would just be to avoid duplicated code though.
>
> -Aaron Schulz
>
>
> ------------------------------
> Connect and share in new ways with Windows Live. Connect now!<http://www.windowslive.com/connect.html?ocid=TXT_TAGLM_Wave2_newways_112007>
>
> _______________________________________________
> Wikiquality-l mailing list
> Wikiquality-l at lists.wikimedia.org
> http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/wikiquality-l/attachments/20071202/1b667fe5/attachment.htm