"The bot would have to IP hop over several A-classes", this is whats
called a tor-network.
One very problematic type of vandalism that usually goes undetected are
introduction of nonexisting iw-links. Another are wrong categories. The
list of possible options are fairly long.
Note that I don't say this isn't a good tool, I say that it should be
carefully inspected for ways to game the system, and if possible, adding
features that makes it impossible to use those flaws.
John E Blad
Aaron Schulz skrev:
That would still be quite a feat. The bot would have
to IP hop over
several A-classes. Make some random vandalism that looks like random
vandalism (rather than the same set of garbage or random
characters/blanking). The IP would have to change enough for it to not
look like one bot on RC. The good bot would have to give a time pause
before reverting to let other beat it to the punch. Also, if all it
did was revert bad bot, then as I said, the bad bot's IP would really
have to be all over the place, unless the bot made random accounts
too. It would still take a while to build up trust this way...and even
with the max, it will still take several edits to get bad content
white. This would have to be good enough to set the "most trusted"
version. And even if this is done on some pages, it will get reverted
and the user blocked, and they have to use another "good bot".
It certainly is always better to be less easily spoofed, and if there
are good practical things that can stop this without making the
software too underinclusive, I'm all for it. I said earlier that users
in the formal bot group should not count as adding credibility. In
this vein, we could do the opposite and have groups that increase the
max trust (credits) a user can have. That way the default max trust
can be lowered to deal further with stuff like good/bad bot networks.
This would work if integrated with FlaggedRevs, as the 'editor' group
could have a higher max trust limit.
-Aaron Schulz
Date: Sat, 22 Dec 2007 00:42:27 +0100
From: john.erling.blad(a)jeb.no
To: wikiquality-l(a)lists.wikimedia.org
Subject: Re: [Wikiquality-l] Wikipedia colored according to trust
Note that I don't say clique detection should be implemented, I say this
is a very interesting option to get running if possible. There are
basically two possible solution, or rather I'm not aware of any other
that is possible to implement. One is to use identified cooperation
between two or more persons to be able to edit an otherwise locked
article, the other is to identify cooperation by checking how two or
more users interact. If two persons starting to interact in a positive
way at any one article it is highly unlikely the goes to war on another
article. War between users can be readily detected by simple statistical
means (aka automatic classification of statements).
Now the second problem. In trust-accumulating systems you don't have to
pile up edits in a single article to game the system. You use two bots.
One bad-bot introduces some identifiable error in many articles, and
then another good-bot fix those errors. The good-bot will quickly gain
high reputation. Even so if the bad-bot changes ip-address often so it
goes undetected. This has to be solved somehow, and to say it involves
to much work to do something like that is plainly a wrong solution to
the problem. I think it is solvable, and I think at least one solution
is the fact that all such systems will produce highly unusual usage
metrics. It is then mostly a matter of detecting the unusual metrics and
sound an alarm bell. If it is possible to incorporate "physical means"
that makes it impossible to game the system it is even better. For
example, a radio communication system can be jammed, but using frequency
hopping radios makes them susceptible to jamming. If they are hopping
fast enough they stays ahead of the jammer. If they are hopping faster
than the radio waves can reach the jammer then it is physically
impossible to use smart jammers.
Don't create a system that can be easilly fooled by more or less smart
bots. Make it physically impossible to fool the system with a bot.
1) decrease credits when time between edits goes down
2) never give more credits to revert an edit then given when making the
original edit
3) weight the edits according to the amount of contribution, yet
different styles of contributions should give the same net result in
credits
4) decrease credits when cooperation are detected
and the reason behind
the cooperation can't be detected
5) increase credits when there are a positive cooperation
A slight modification of 2 is to release it if there goes sufficient
time.
John E Blad
Aaron Schulz skrev:
> How do you tell if users interact? Cross user talk page edits? If you
> go by edits or edits to talk, what is the difference between people in
> a clique and people that disagree all the time from the program's
> standpoint.
>
> For identifying non-vandalized versions, cliques don't seem to pose
> much of a threat. The only way for that to be a problem is for people
> to edit stack, similar to the issue of users not using preview and
> making a ton of edits in a row. This could game the system sometimes,
> but it is also very obvious behavior when the same 2-3 new (otherwise
> thy would have probably been blocked by now) people pile on 8 edits
> after each other. Not only will it stand out, but all it would do is
> influence the "most trusted"/"likely unvandalized" stable
version or
> the trust for the current revision in their favor...until someone just
> reverts it anyway. It's too much work for too little plus likely
> getting caught in the process.
>
> The system will not be bullet proof. Admins have gone rouge before.
> Even for FlaggedRevs, "editor"/"surveyor" status may be abused
by a
> very small number of people. By default it checks for email, a
> userpage, 150 edits spread out well, account age. Still, some people
> could get through and then troll. The thing is though, that it is just
> not worth, as they would have the rights removed and possibly be
> blocked immediately. And after what? Sighting some vandalism, which
> would just get reverted and fixed in a few minutes. To do it again
> would require another IP and another account back at square one again.
> That's just not worth it for a troll. We get vandals from people
> testing and joking around as well as people that just whip new
> accounts out their ass every minutes because it is so easy.
> FlaggedRevs makes it not worth it. Likewise, trying to game Article
> Trust does seem to very "worth it" much either.
>
> Think of how long login didn't have captchas for such a large site.
> That's because nobody cared to sit there guessing around or writing
> bots then. If things are too hard to game, people won't care. The
> "rewards" of gaming this system is even way less than hacking an
> account. It's a bit better with FlaggedRevs because you commit to
> having done reviews, so reviewing vandalism goes straight against you.
> But still, tweaking junk edits 15 times or having the same two editors
> cluster 8 tiny edits after each other accomplishes little, is
> noticeable, and IMO, not really worth it. Also, we don't have to use
> AT or FR exclusively, and some combination of both (quality > sighted
> > autotrust > current) could avoid the multiple edit gaming issue
> altogether.
>
> -Aaron Schulz
>
> > Date: Fri, 21 Dec 2007 21:49:30 +0100
> > From: john.erling.blad(a)jeb.no
> > To: wikiquality-l(a)lists.wikimedia.org
> > Subject: Re: [Wikiquality-l] Wikipedia colored according to trust
> >
> > I think that kind of weighting is correct as long as it is
symmetrical
> > (ie. do-undo -pairs weights approx the
same). Clique-building is
> > interesting although when you are able to analyze which ones
interacts.
> > Then you can punish people when their
friends do bad edits. Its
not as
> > bad as it sounds, the reasoning is that
often someone writes the
article
> > under guidance of some other. You
don't want to punish only one
user in
> > such situations but both.
> > John E
> >
> > Luca de Alfaro skrev:
> > > I am not sure I am replying to the correct point, but, the system
> > > weighs an author feedback as a function of the reputation of the
> author.
> > > Reputation is "linear" in the sense that new feedback is
simply added
> > > to the reputation.
> > > A user of reputation r gives weight log(1 + r) to hiers feedback.
> > > We use this logarithmic scaling to prevent long-time editors from
> > > forming a clique that is essentially impervious to feedback
from
the
> > > rest of the community (will this
kind of comments get me
skinned? :-)
> > >
> > > Luca
> > >
> > > On Dec 21, 2007 11:41 AM, John Erling Blad
<john.erling.blad(a)jeb.no
> > >
<mailto:john.erling.blad@jeb.no>> wrote:
> > >
> > > It is wise to make a note about the fact that such systems make it
> > > possible to deduce earlier in the mean that someone is a vandal or
> > > not,
> > > but it can't replace a good reader that responds to an error. This
> > > creates the rather annoying situation where a response from a
casual
> > > reader should be weighted more
than non-beginners, but this
makes the
> > > system suceptible to users wanting
to skew its metrics on specific
> > > users.
> > >
> > > John E
> > >
> > > Aaron Schulz skrev:
> > > > Right. Also, we need to be clear what we want this to do. It
will
> > > > never be great at determining
fact-checked material. What it
is good
> > > > at is spotting the more
dubious stuff, like possible vandalism.
> > > This
> > > > makes the possibility of having "most trusted" stable
version as
> > > > discussed earlier. Small changes not only can be big in
meaning, but
> > > > they still attest to the
trust.
> > > >
> > > > If I read a sentence to change some minor thing, I still read
> > > it. If a
> > > > wrongly says "he identifies himself as bisexual" or
"born in
1885"
> > > > rather than 1985 in a page
when I edit, I am going to revert
if I
> > > > catch it. Even if just making
some grammar/syntax cleanup.
So each
> > > > time people look at stuff if
still attest to the page a
little bit,
> > > > from a vandalism
perspective.
> > > >
> > > > The algorithms can be made more strict to catch more general
dubious
> > > > info better, but it is not
that bad at that already, and the
> > > stricter
> > > > it gets, the more it gets under inclusive as to what is
considered
> > > > unlikely to be vandalized.
> > > >
> > > > -Aaron Schulz
> > > >
> > > >
> > >
>
------------------------------------------------------------------------
> > >
> > > > Date: Fri, 21 Dec 2007 10:34:47 -0800
> > > > From: luca(a)dealfaro.org <mailto:luca@dealfaro.org>
> > > > To: wikiquality-l(a)lists.wikimedia.org
> > > <mailto:wikiquality-l@lists.wikimedia.org>
> > > > Subject: Re: [Wikiquality-l] Wikipedia colored according to
> > > trust
> > > >
> > > > If you want to pick out the malicious changes, you need to flag
> > > > also small changes.
> > > >
> > > > "Sen. Hillary Clinton did *not* vote in favor of war in
Iraq"
> > > >
> > > > "John Doe, born in *1947*"
> > > >
> > > > The ** indicates changes.
> > > >
> > > > I can very well make a system that is insensitive to small
> > > > changes, but then the system would also be insensitive to many
> > > > kinds of malicious tampering, and one of my goals was to make it
> > > > hard for anyone to change without leaving at laest a minimal
> > > trace.
> > > >
> > > > So it's a matter of goals, really.
> > > >
> > > > Luca
> > > >
> > > > On Dec 21, 2007 10:01 AM, Jonathan Leybovich
> > > <jleybov(a)gmail.com <mailto:jleybov@gmail.com>
> > > > <mailto: jleybov(a)gmail.com
<mailto:jleybov@gmail.com>>> wrote:
> > > >
> > > > One thing that stood out for me in the small sample of
> > > articles I
> > > > examined was the flagging of innocuous changes by casual
> > > users to
> > > > correct spelling, grammar, etc. Thus a "nice-to-have"
> > > would be a
> > > > "smoothing" algorithm that ignores inconsequential changes
> > > > such as
> > > > spelling corrections, etc. or the reordering of
> > > > semantically-contained
> > > > units of text (for example, reordering the line items in a
> > > > list w/o
> > > > changing the content of any particular line item, etc.,
> > > or the
> > > > reordering of paragraphs and perhaps even sentences.) I
> > > think
> > > > this
> > > > would cover 90% or more of changes that are immaterial
> > > to an
> > > > article's
> > > > credibility.
> > > >
> > > > _______________________________________________
> > > > Wikiquality-l mailing list
> > > > Wikiquality-l(a)lists.wikimedia.org
> > > <mailto:Wikiquality-l@lists.wikimedia.org>
> > > > <mailto:Wikiquality-l@lists.wikimedia.org
> > > <mailto:Wikiquality-l@lists.wikimedia.org>>
> > > >
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > >
> > > >
> > > >
> > > >
> > >
>
------------------------------------------------------------------------
> > > > Get the power of Windows +
Web with the new Windows Live. Get it
> > > now!
> > > >
> > >
>
<http://www.windowslive.com?ocid=TXT_TAGHM_Wave2_powerofwindows_122007>
> > > >
> > >
>
------------------------------------------------------------------------
> > >
> > > >
> > > > _______________________________________________
> > > > Wikiquality-l mailing list
> > > > Wikiquality-l(a)lists.wikimedia.org
> > > <mailto:Wikiquality-l@lists.wikimedia.org>
> > > >
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > >
> > >
> > > _______________________________________________
> > > Wikiquality-l mailing list
> > > Wikiquality-l(a)lists.wikimedia.org
> > > <mailto:Wikiquality-l@lists.wikimedia.org>
> > >
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > > <http://lists.wikimedia.org/mailman/listinfo/wikiquality-l>
> > >
> > >
> > >
>
------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > Wikiquality-l mailing list
> > > Wikiquality-l(a)lists.wikimedia.org
> > >
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
> > >
>
>
------------------------------------------------------------------------
> i’m is proud to present Cause Effect, a
series about real people
> making a difference. Learn more
> <http://im.live.com/Messenger/IM/MTV/?source=text_Cause_Effect>
>
------------------------------------------------------------------------
>
> _______________________________________________
> Wikiquality-l mailing list
> Wikiquality-l(a)lists.wikimedia.org
>
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l
>
------------------------------------------------------------------------
Share life as it happens with the new Windows Live. Share now!
<http://www.windowslive.com/share.html?ocid=TXT_TAGHM_Wave2_sharelife_122007>
------------------------------------------------------------------------
_______________________________________________
Wikiquality-l mailing list
Wikiquality-l(a)lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/wikiquality-l