Hello Luca,
> 1. The trust coloring rightly colored orange (low-trust) some
> unreliable content,
Yes I was lost in translation. ;-)
> 2. and the Wikipedia people were quick in reverting it.
Yes.
> Note that we also highlight as low trust text that is by anonymous
> contributors. The text will then gain trust as it is revised.
One possible weakness came into my mind after I also read your paper. Your
algorithm is perhapes a bit vulnerable to "sock puppets". Imagine person A
with one account and person B with two accounts. Both have a medium
reputation value for their accounts. User A edits an article with his account
4 times. All 4 subsequent edits are taken together and the article has a
maximum trust value according to the user's reputation. User B makes as well
4 edits to an article but switches between his accounts and thus "reviews"
his own edits. If I understand your algorithm correctly the sock puppeted
article is trusted more than the other one.
Quite some time ago I reflected how to avoid incentives for sock puppets in
karma systems without even knowing which accounts are sock puppets:
http://meta.wikimedia.org/wiki/Meritokratischer_Review (sadly in German ;-).
The system described there differs from your approach but the idea on how to
avoid incentives for sock puppets without even knowing who a sock puppet is
could perhapes adapted to your system.
The basic idea for a sock puppet proof metric is is that a person has only a
limited amount of time for editing (I don't consider bots cause they are
easily detectable by humans). A single person needs the same time for
e.g. 4
edits (in the following I assume each edit has the same length in bytes)
regardless how much accounts are used but two different people with each 2
edits only need half of the (imaginary) time (you don't need to measure any
time untits at all).
So the maximum possible reliability person B can apply to the article with its
two accounts (let us say each acount has 2 edits = 4 total edits) has to be
the same as the one which is possible with person A's single account (4
edits). So in general two accounts with each X edits should never be able to
add more trust to an article than one person with 2*X edits (note: edit count
number is only for illustration, you can take another appropriate
contribution unit).
> About 2, I am very glad that bad edits are quickly reverted; this is the
> whole reason Wikipedia has worked up to now.
> Still, it might be easier for editors to find content to check via the
> coloring, rather than by staring at diffs.
That's certainly true for articles not on your watchlist (or bad edits that
were forgotten and are still the latest version).
> - Finding when flagged revisions are out of date (there may be a new
> high-trust version later)
Well as I said I'd love to see flagged revisions and your system combined (in
a way described by my previous mail). An automated system probably always has
some weaknesses some clever people can abuse but it is very fast, while a
hand crafted system depends on the speed of individual persons but is much
harder to fool.
> BTW, as the method is language-independent, we look forward to doing the
> same for wikipedias in other languages.
Good to know. :-)
Arnomane
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l