Hi, I talked with a developer of cluebot and we have had an interesting idea. It would be really cool if tools like huggle, clue bot and others could communicate so that for instance people would not attempt to revert same edit in same time, could report to cluebot new vandalism so that it could update the definitions or mark edits as valid, share whitelist and such. For this it's necessary to create some communication protocol which tool would be using.
My original proposal was to use irc for this because it's not so complicated, but Ryan told me that using the http://burrow.openstack.org/ would be much better for this purpose, what do you think about it, is there someone who would like to discuss some standardised protocol we could implement to the tools for vandalism help?
My idea was to setup a simple irc network on labs where the tools would be able to send messages, each project would have own channel and tools would use standardised messages including various data. However this has some disadvantages, the port 6667 could be firewalled and it doesn't offer so many features. Also it's quite possible that someone could "hack" into communication channel in order to cause harm.
So maybe the first question to discuss is which system to use, if irc or queues? The next step would be probably to set up a wiki page with details so that developers could update specification of this protocol.
If believe we have/had evil plans to move away from IRC for things like this to something like XMPP[Citation Needed].
Wouldn't a system like this slow down the vandal checking, because it would need to check the feed, do its business and then check again before attempting to save. Or would it check the list and then mark that it is going to attempt the edit, do it and then mark it done?
Hi K.
Thank you for reply, I don't think it would slow down anything, it would actually make all tools a lot faster, it wouldn't require tools to wait for each other but it would allow them to communicate in a way so that it avoid for instance:
* reverting the same edit * reviewing edit which is correct * allow them to notify others on suspicious edits
it would be an optional feature not a required thing, I don't see any reason why it should cause any troubles.
On Mon, Dec 26, 2011 at 9:24 AM, K. Peachey p858snake@gmail.com wrote:
If believe we have/had evil plans to move away from IRC for things like this to something like XMPP[Citation Needed].
Wouldn't a system like this slow down the vandal checking, because it would need to check the feed, do its business and then check again before attempting to save. Or would it check the list and then mark that it is going to attempt the edit, do it and then mark it done?
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
MediaWiki already has a native inteface meant for marking edits as patrolled. That system is ideal for anti-vandalism communication, and is already used as such on many wikis (other than en.wikipedia.org)
It works like this: * Tool gets edit-feed from irc.wikimedia.org, or recentchanges API or Special:RecentChanges (depending on whether it is a standalone app, IRC bot, standalone gadget or enhancement on top of Special:RecentChanges) * Tool gets diff or links to it * User marks it as patrolled OR User fixes edit / warns user and then marks as patrolled * Patrolling is done by either following a link to action=markpatrolled from within the wiki or the tool uses the action=patrol API
Right now tools using the recentchanges API or Special:RecentChanges to retrieve the edit list already have a way to filter out patrolled edits (such as RTRC [1]). Tools using the IRC feed can't do it as easily, although they could parse Special:Log/patrol actions (which are also sent to IRC) and identify the rc_id of the markpatrolled action with the edit action previously recorded and then hide that edit it from their live queue.
On en.wikipedia.org the markpatrolled feature was disabled in 2005 because the red exlaimation marks that some admins didn't like (note that at the time admins were the only users with the ability to see the patrol marks) - and as a result of the way things were in 2005 it was disabled by default globally and still is and wikis have to request it to be enabled individually, 57 wikis have done so already including Wikimedia Commons and most Dutch, German, French and Italian projects.
Using the patrol feature for this makes sense for more reasons, as it is also used internally by MediaWiki. It it also integrated into the Watchlist and user right 'patrol' is assigned to allow users to know whether a revision is patrolled or not and allow them to mark it as such (which can potentially replace the duplication of 'trusted user' lists, simply check if the user has this user right on the target wiki).
Krinkle
[1] https://meta.wikimedia.org/wiki/User:Krinkle/Tools/Real-Time_Recent_Changes
Hi, that's a nice feature. But if it's disabled I don't think it's useful then... Anyway this communication protocol would allow us to share way more info, we could share the list of potentially problematic edits and such, Clue Bot is skipping a lot of edits which may be a vandalism but it's not able to detect it correctly.
On Tue, Dec 27, 2011 at 10:28 AM, Krinkle krinklemail@gmail.com wrote:
MediaWiki already has a native inteface meant for marking edits as patrolled. That system is ideal for anti-vandalism communication, and is already used as such on many wikis (other than en.wikipedia.org)
It works like this:
- Tool gets edit-feed from irc.wikimedia.org, or recentchanges API or
Special:RecentChanges (depending on whether it is a standalone app, IRC bot, standalone gadget or enhancement on top of Special:RecentChanges)
- Tool gets diff or links to it
- User marks it as patrolled OR User fixes edit / warns user and then marks
as patrolled
- Patrolling is done by either following a link to action=markpatrolled from
within the wiki or the tool uses the action=patrol API
Right now tools using the recentchanges API or Special:RecentChanges to retrieve the edit list already have a way to filter out patrolled edits (such as RTRC [1]). Tools using the IRC feed can't do it as easily, although they could parse Special:Log/patrol actions (which are also sent to IRC) and identify the rc_id of the markpatrolled action with the edit action previously recorded and then hide that edit it from their live queue.
On en.wikipedia.org the markpatrolled feature was disabled in 2005 because the red exlaimation marks that some admins didn't like (note that at the time admins were the only users with the ability to see the patrol marks) - and as a result of the way things were in 2005 it was disabled by default globally and still is and wikis have to request it to be enabled individually, 57 wikis have done so already including Wikimedia Commons and most Dutch, German, French and Italian projects.
Using the patrol feature for this makes sense for more reasons, as it is also used internally by MediaWiki. It it also integrated into the Watchlist and user right 'patrol' is assigned to allow users to know whether a revision is patrolled or not and allow them to mark it as such (which can potentially replace the duplication of 'trusted user' lists, simply check if the user has this user right on the target wiki).
Krinkle
[1] https://meta.wikimedia.org/wiki/User:Krinkle/Tools/Real-Time_Recent_Changes _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 27.12.2011, 14:12 Petr wrote:
Hi, that's a nice feature. But if it's disabled I don't think it's useful then... Anyway this communication protocol would allow us to share way more info, we could share the list of potentially problematic edits and such, Clue Bot is skipping a lot of edits which may be a vandalism but it's not able to detect it correctly.
It can be re-enabled as easily as it was disabled. We just need real use cases where it can be useful and communicate to communities clearly why we're making such a change. We could even disable the display of those annoying exclamation marks in RC by default.
On Tue, Dec 27, 2011 at 11:12 AM, Petr Bena benapetr@gmail.com wrote:
Hi, that's a nice feature. But if it's disabled I don't think it's useful then... Anyway this communication protocol would allow us to share way more info, we could share the list of potentially problematic edits and such, Clue Bot is skipping a lot of edits which may be a vandalism but it's not able to detect it correctly.
Well, I don't believe in a new "big all-in-one" system. We have many that we should build upon to avoid wasting a lot of efforts.
MediaWiki has a "patrol" flag on each recent change. That should simply be used. Otherwise double work occurs as other extensions or gadgets do use that. It also has a stable API that can be relied on, and any extensions written to utilize this MediaWiki feature will automatically work.
By not using that, double work occurs and time is being wasted.
Aside from MediaWiki in general, for the Wikimedia Foundation wikis there is an unofficial organization named "Countervandalism Network" (CVN), which pretty manages communication protocols and centralization in countervandalism efforts, I'd recommend staying in close touch with them[5] as well.
The CVN already has a "global" countervandalism database that contains much useful information that is being kept up to date with recent events[6]: * a global blacklist (list of usernames with an expiration date for the entry and a reason) * a global whitelist (basically a mirror listing all users that are 'patroller' or 'sysop' on one or wikis) * a global and a per-wiki watchlist (page names, expiration date and reason)
These are currently used by all CVNBots and SWBots in the #cvn-* channels on irc.freenode.net such as #cvn-commons and #cvn-wp-nl as well as #cvn-wp-en. In those channels edits by blacklisted users and/or on watched pages are highlighted. So it's a "common watchlist" for all CVN vandal-fighters. This database is replicated to the Toolserver and has a primitive API [3] (could be expanded) allowing gadgets to use this information as well. For example RTRC [1] and CVN_SimpleOverlay [2] use this API.
Some ideas I've been having for the long term:
* Switch to a machine-readable push notification service for recent changes (i.e. the best of the API (XML/JSON) and the best of IRC (push/subscribe). Something like WebSockets / PubSubHubBub / Jabber.
* Replace monitoring systems such as irc-vandalism bots, Huggle, STiki and the like with a Web-based framework (e.g. Extension:ActivityMonitor providing Special:ActivityMonitor) that allows following this stream live and ability to filter things out (e.g. patrolled edits) and highlight stuff (things that are on your watchlist, things on the CVN blacklist / watchlist) and other things we would expect from a monitor tool.
* Move the CVN database to WMF, making it available through the API.
* Enable cross-domain AJAX whitelist on WMF (CORS)
* Option in SpecialActivityMonitor to monitor multiple wikis (right now the CVN has an IRC channel #cvn-sw that monitors 600 small wikis that don't have a stable anti-vandalism team - monitored mostly by stewards and global sysops as well as global rollbackers, it is also the key to catching cross-wiki vandalism)
* In additional to monitoring a stream and hand-picking interesting edits, we should also implement a workflow like WikiHow's Special:RCPatrol [4] for wikis that are capable of patrolling all edits. Right now they do so through Special:RecentChanges by hiding unpatrolled edits and start at the back and work their way to the top, but that workflow sucks a lot and often causes double work as people click the same links.
I could go on like this but I better stop now. Hop in on #countervandalism for more.
Krinkle
[1] https://meta.wikimedia.org/wiki/User:Krinkle/Tools/Real-Time_Recent_Changes
[2] https://meta.wikimedia.org/wiki/User:Krinkle/Scripts/CVNSimpleOverlay
[3] https://wiki.toolserver.org/view/CVN_API
[4] http://www.wikihow.com/Patrol-Recent-Changes-on-wikiHow
[5] "them", or "us", as I'm one of them - https://meta.wikimedia.org/wiki/CVN
[6] CVN has several bots. For example users that are blocked by admins on a wiki are automatically added to the global blacklist so that if a user is blocked on one wiki, edits by that account on other wikis will be highlighted (this is currently the only working defense that I know of against cross-wiki vandalism). CVN also has an irc-bridge with ClueNet, any user flagged in the ClueNet stream (which is populated by by ClueBot_NG) is also added to the CVN blacklist for the 2*duration that ClueBot monitors it.
complete off-topic
I has ben following this interesting thread, and some links connected here. Doing that I have found this:
http://code.google.com/p/php-console/
Is a chrome extension + a php class that created a "out of band" communication from the php error generation to the desktop, opening notifications for the programmer. Is really cool, If you have chrome open, and you refresh a firefox window that generate error's, you still get the same notifications popups. I don't know how it work, I suppose the php class create a socket and do some sort of "machine-readable push notifications" to chrome.
HTML5 even have a notification api. http://www.html5rocks.com/en/tutorials/notifications/quick/
The future is going to be fun :D
end offtopic.
wikitech-l@lists.wikimedia.org