[Wikiquality-l] Wikipedia colored according to trust

Thu Dec 27 22:00:09 UTC 2007

"The bot would have to IP hop over several A-classes", this is whats
called a tor-network.

One very problematic type of vandalism that usually goes undetected are
introduction of nonexisting iw-links. Another are wrong categories. The
list of possible options are fairly long.

Note that I don't say this isn't a good tool, I say that it should be
carefully inspected for ways to game the system, and if possible, adding
features that makes it impossible to use those flaws.

John E Blad

Aaron Schulz skrev:
> That would still be quite a feat. The bot would have to IP hop over
> several A-classes. Make some random vandalism that looks like random
> vandalism (rather than the same set of garbage or random
> characters/blanking). The IP would have to change enough for it to not
> look like one bot on RC. The good bot would have to give a time pause
> before reverting to let other beat it to the punch. Also, if all it
> did was revert bad bot, then as I said, the bad bot's IP would really
> have to be all over the place, unless the bot made random accounts
> too. It would still take a while to build up trust this way...and even
> with the max, it will still take several edits to get bad content
> white. This would have to be good enough to set the "most trusted"
> version. And even if this is done on some pages, it will get reverted
> and the user blocked, and they have to use another "good bot".
>
> It certainly is always better to be less easily spoofed, and if there
> are good practical things that can stop this without making the
> software too underinclusive, I'm all for it. I said earlier that users
> in the formal bot group should not count as adding credibility. In
> this vein, we could do the opposite and have groups that increase the
> max trust (credits) a user can have. That way the default max trust
> can be lowered to deal further with stuff like good/bad bot networks.
> This would work if integrated with FlaggedRevs, as the 'editor' group
> could have a higher max trust limit.
>
> -Aaron Schulz
>
> > Date: Sat, 22 Dec 2007 00:42:27 +0100
> > From: john.erling.blad at jeb.no
> > To: wikiquality-l at lists.wikimedia.org
> > Subject: Re: [Wikiquality-l] Wikipedia colored according to trust
> >
> > Note that I don't say clique detection should be implemented, I say this
> > is a very interesting option to get running if possible. There are
> > basically two possible solution, or rather I'm not aware of any other
> > that is possible to implement. One is to use identified cooperation
> > between two or more persons to be able to edit an otherwise locked
> > article, the other is to identify cooperation by checking how two or
> > more users interact. If two persons starting to interact in a positive
> > way at any one article it is highly unlikely the goes to war on another
> > article. War between users can be readily detected by simple statistical
> > means (aka automatic classification of statements).
> >
> > Now the second problem. In trust-accumulating systems you don't have to
> > pile up edits in a single article to game the system. You use two bots.
> > One bad-bot introduces some identifiable error in many articles, and
> > then another good-bot fix those errors. The good-bot will quickly gain
> > high reputation. Even so if the bad-bot changes ip-address often so it
> > goes undetected. This has to be solved somehow, and to say it involves
> > to much work to do something like that is plainly a wrong solution to
> > the problem. I think it is solvable, and I think at least one solution
> > is the fact that all such systems will produce highly unusual usage
> > metrics. It is then mostly a matter of detecting the unusual metrics and
> > sound an alarm bell. If it is possible to incorporate "physical means"
> > that makes it impossible to game the system it is even better. For
> > example, a radio communication system can be jammed, but using frequency
> > hopping radios makes them susceptible to jamming. If they are hopping
> > fast enough they stays ahead of the jammer. If they are hopping faster
> > than the radio waves can reach the jammer then it is physically
> > impossible to use smart jammers.
> >
> > Don't create a system that can be easilly fooled by more or less smart
> > bots. Make it physically impossible to fool the system with a bot.
> >
> > 1) decrease credits when time between edits goes down
> > 2) never give more credits to revert an edit then given when making the
> > original edit
> > 3) weight the edits according to the amount of contribution, yet
> > different styles of contributions should give the same net result in
> credits
> > 4) decrease credits when cooperation are detected and the reason behind
> > the cooperation can't be detected
> > 5) increase credits when there are a positive cooperation
> >
> > A slight modification of 2 is to release it if there goes sufficient
> time.
> >
> > John E Blad
> >
> > Aaron Schulz skrev:
> > > How do you tell if users interact? Cross user talk page edits? If you
> > > go by edits or edits to talk, what is the difference between people in
> > > a clique and people that disagree all the time from the program's
> > > standpoint.
> > >
> > > For identifying non-vandalized versions, cliques don't seem to pose
> > > much of a threat. The only way for that to be a problem is for people
> > > to edit stack, similar to the issue of users not using preview and
> > > making a ton of edits in a row. This could game the system sometimes,
> > > but it is also very obvious behavior when the same 2-3 new (otherwise
> > > thy would have probably been blocked by now) people pile on 8 edits
> > > after each other. Not only will it stand out, but all it would do is
> > > influence the "most trusted"/"likely unvandalized" stable version or
> > > the trust for the current revision in their favor...until someone just
> > > reverts it anyway. It's too much work for too little plus likely
> > > getting caught in the process.
> > >
> > > The system will not be bullet proof. Admins have gone rouge before.
> > > Even for FlaggedRevs, "editor"/"surveyor" status may be abused by a
> > > very small number of people. By default it checks for email, a
> > > userpage, 150 edits spread out well, account age. Still, some people
> > > could get through and then troll. The thing is though, that it is just
> > > not worth, as they would have the rights removed and possibly be
> > > blocked immediately. And after what? Sighting some vandalism, which
> > > would just get reverted and fixed in a few minutes. To do it again
> > > would require another IP and another account back at square one again.
> > > That's just not worth it for a troll. We get vandals from people
> > > testing and joking around as well as people that just whip new
> > > accounts out their ass every minutes because it is so easy.
> > > FlaggedRevs makes it not worth it. Likewise, trying to game Article
> > > Trust does seem to very "worth it" much either.
> > >
> > > Think of how long login didn't have captchas for such a large site.
> > > That's because nobody cared to sit there guessing around or writing
> > > bots then. If things are too hard to game, people won't care. The
> > > "rewards" of gaming this system is even way less than hacking an
> > > account. It's a bit better with FlaggedRevs because you commit to
> > > having done reviews, so reviewing vandalism goes straight against you.
> > > But still, tweaking junk edits 15 times or having the same two editors
> > > cluster 8 tiny edits after each other accomplishes little, is
> > > noticeable, and IMO, not really worth it. Also, we don't have to use
> > > AT or FR exclusively, and some combination of both (quality > sighted
> > > > autotrust > current) could avoid the multiple edit gaming issue
> > > altogether.
> > >
> > > -Aaron Schulz
> > >
> > > > Date: Fri, 21 Dec 2007 21:49:30 +0100
> > > > From: john.erling.blad at jeb.no
> > > > To: wikiquality-l at lists.wikimedia.org
> > > > Subject: Re: [Wikiquality-l] Wikipedia colored according to trust
> > > >
> > > > I think that kind of weighting is correct as long as it is
> symmetrical
> > > > (ie. do-undo -pairs weights approx the same). Clique-building is
> > > > interesting although when you are able to analyze which ones
> interacts.
> > > > Then you can punish people when their friends do bad edits. Its
> not as
> > > > bad as it sounds, the reasoning is that often someone writes the
> article
> > > > under guidance of some other. You don't want to punish only one
> user in
> > > > such situations but both.
> > > > John E
> > > >
> > > > Luca de Alfaro skrev:
> > > > > I am not sure I am replying to the correct point, but, the system
> > > > > weighs an author feedback as a function of the reputation of the
> > > author.
> > > > > Reputation is "linear" in the sense that new feedback is
> simply added
> > > > > to the reputation.
> > > > > A user of reputation r gives weight log(1 + r) to hiers feedback.
> > > > > We use this logarithmic scaling to prevent long-time editors from
> > > > > forming a clique that is essentially impervious to feedback
> from the
> > > > > rest of the community (will this kind of comments get me
> skinned? :-)
> > > > >
> > > > > Luca
> > > > >
> > > > > On Dec 21, 2007 11:41 AM, John Erling Blad
> <john.erling.blad at jeb.no
> > > > > <mailto:john.erling.blad at jeb.no>> wrote:
> > > > >
> > > > > It is wise to make a note about the fact that such systems make it
> > > > > possible to deduce earlier in the mean that someone is a vandal or
> > > > > not,
> > > > > but it can't replace a good reader that responds to an error. This
> > > > > creates the rather annoying situation where a response from a
> casual
> > > > > reader should be weighted more than non-beginners, but this
> makes the
> > > > > system suceptible to users wanting to skew its metrics on specific
> > > > > users.
> > > > >
> > > > > John E
> > > > >
> > > > > Aaron Schulz skrev:
> > > > > > Right. Also, we need to be clear what we want this to do. It
> will
> > > > > > never be great at determining fact-checked material. What it
> is good
> > > > > > at is spotting the more dubious stuff, like possible vandalism.
> > > > > This
> > > > > > makes the possibility of having "most trusted" stable version as
> > > > > > discussed earlier. Small changes not only can be big in
> meaning, but
> > > > > > they still attest to the trust.
> > > > > >
> > > > > > If I read a sentence to change some minor thing, I still read
> > > > > it. If a
> > > > > > wrongly says "he identifies himself as bisexual" or "born in
> 1885"
> > > > > > rather than 1985 in a page when I edit, I am going to revert
> if I
> > > > > > catch it. Even if just making some grammar/syntax cleanup.
> So each
> > > > > > time people look at stuff if still attest to the page a
> little bit,
> > > > > > from a vandalism perspective.
> > > > > >
> > > > > > The algorithms can be made more strict to catch more general
> dubious
> > > > > > info better, but it is not that bad at that already, and the
> > > > > stricter
> > > > > > it gets, the more it gets under inclusive as to what is
> considered
> > > > > > unlikely to be vandalized.
> > > > > >
> > > > > > -Aaron Schulz
> > > > > >
> > > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > > Date: Fri, 21 Dec 2007 10:34:47 -0800
> > > > > > From: luca at dealfaro.org <mailto:luca at dealfaro.org>
> > > > > > To: wikiquality-l at lists.wikimedia.org
> > > > > <mailto:wikiquality-l at lists.wikimedia.org>
> > > > > > Subject: Re: [Wikiquality-l] Wikipedia colored according to
> > > > > trust
> > > > > >
> > > > > > If you want to pick out the malicious changes, you need to flag
> > > > > > also small changes.
> > > > > >
> > > > > > "Sen. Hillary Clinton did *not* vote in favor of war in Iraq"
> > > > > >
> > > > > > "John Doe, born in *1947*"
> > > > > >
> > > > > > The ** indicates changes.
> > > > > >
> > > > > > I can very well make a system that is insensitive to small
> > > > > > changes, but then the system would also be insensitive to many
> > > > > > kinds of malicious tampering, and one of my goals was to make it
> > > > > > hard for anyone to change without leaving at laest a minimal
> > > > > trace.
> > > > > >
> > > > > > So it's a matter of goals, really.
> > > > > >
> > > > > > Luca
> > > > > >
> > > > > > On Dec 21, 2007 10:01 AM, Jonathan Leybovich
> > > > > <jleybov at gmail.com <mailto:jleybov at gmail.com>
> > > > > > <mailto: jleybov at gmail.com <mailto:jleybov at gmail.com>>> wrote:
> > > > > >
> > > > > > One thing that stood out for me in the small sample of
> > > > > articles I
> > > > > > examined was the flagging of innocuous changes by casual
> > > > > users to
> > > > > > correct spelling, grammar, etc. Thus a "nice-to-have"
> > > > > would be a
> > > > > > "smoothing" algorithm that ignores inconsequential changes
> > > > > > such as
> > > > > > spelling corrections, etc. or the reordering of
> > > > > > semantically-contained
> > > > > > units of text (for example, reordering the line items in a
> > > > > > list w/o
> > > > > > changing the content of any particular line item, etc.,
> > > > > or the
> > > > > > reordering of paragraphs and perhaps even sentences.) I
> > > > > think
> > > > > > this
> > > > > > would cover 90% or more of changes that are immaterial
> > > > > to an
> > > > > > article's
> > > > > > credibility.
> > > > > >
> > > > > > _______________________________________________
> > > > > > Wikiquality-l mailing list
> > > > > > Wikiquality-l at lists.wikimedia.org
> > > > > <mailto:Wikiquality-l at lists.wikimedia.org>
> > > > > > <mailto:Wikiquality-l at lists.wikimedia.org
> > > > > <mailto:Wikiquality-l at lists.wikimedia.org>>
> > > > > > http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > > > Get the power of Windows + Web with the new Windows Live. Get it
> > > > > now!
> > > > > >
> > > > >
> > >
> <http://www.windowslive.com?ocid=TXT_TAGHM_Wave2_powerofwindows_122007>
> > > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Wikiquality-l mailing list
> > > > > > Wikiquality-l at lists.wikimedia.org
> > > > > <mailto:Wikiquality-l at lists.wikimedia.org>
> > > > > > http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Wikiquality-l mailing list
> > > > > Wikiquality-l at lists.wikimedia.org
> > > > > <mailto:Wikiquality-l at lists.wikimedia.org>
> > > > > http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > > > <http://lists.wikimedia.org/mailman/listinfo/wikiquality-l>
> > > > >
> > > > >
> > > > >
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > _______________________________________________
> > > > > Wikiquality-l mailing list
> > > > > Wikiquality-l at lists.wikimedia.org
> > > > > http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > > >
> > >
> > >
> ------------------------------------------------------------------------
> > > i’m is proud to present Cause Effect, a series about real people
> > > making a difference. Learn more
> > > <http://im.live.com/Messenger/IM/MTV/?source=text_Cause_Effect>
> > >
> ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > Wikiquality-l mailing list
> > > Wikiquality-l at lists.wikimedia.org
> > > http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > >
>
> ------------------------------------------------------------------------
> Share life as it happens with the new Windows Live. Share now!
> <http://www.windowslive.com/share.html?ocid=TXT_TAGHM_Wave2_sharelife_122007>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Wikiquality-l mailing list
> Wikiquality-l at lists.wikimedia.org
> http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
>   
-------------- next part --------------
A non-text attachment was scrubbed...
Name: john.erling.blad.vcf
Type: text/x-vcard
Size: 181 bytes
Desc: not available
Url : http://lists.wikimedia.org/pipermail/wikiquality-l/attachments/20071227/00aa8319/attachment-0001.vcf 