http://www-users.cs.umn.edu/~lam/papers/lam_group2010_wikipedia-group-decisi... :
"We also found that there have been two bots (computer programs that edit Wikipedia)—BJBot and Jayden54Bot—that automatically automatically notified article editors about AfD discussions and recruited them to participate per the established policy. These bots performed AfD notifications for several months, and offer us an opportunity to study the effect of recruitment that is purely policy driven. We use a process like one described above to detect successful instances of bot-initiated recruitment: if a recruitment bot edited a user’s talk page, and that user !voted in an AfD within two days, then we consider that user to have been recruited by the bot. Using the above processes, we identified 8,464 instances of successful recruiting. Table 2 shows a summary of who did the recruiting, and how their recruits !voted. We see large differences in !voting behavior, which suggests that there is bias in who people choose to recruit. (From these data we cannot tell whether the bias is an intentional effort to influence consensus, or the result of social network homophily [14].) Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier."
This is a nicely competent paper. Thanks for the heads up! SJ
On Fri, May 20, 2011 at 10:12 AM, Gwern Branwen gwern0@gmail.com wrote:
http://www-users.cs.umn.edu/~lam/papers/lam_group2010_wikipedia-group-decisi... :
"We also found that there have been two bots (computer programs that edit Wikipedia)—BJBot and Jayden54Bot—that automatically automatically notified article editors about AfD discussions and recruited them to participate per the established policy. These bots performed AfD notifications for several months, and offer us an opportunity to study the effect of recruitment that is purely policy driven. We use a process like one described above to detect successful instances of bot-initiated recruitment: if a recruitment bot edited a user’s talk page, and that user !voted in an AfD within two days, then we consider that user to have been recruited by the bot. Using the above processes, we identified 8,464 instances of successful recruiting. Table 2 shows a summary of who did the recruiting, and how their recruits !voted. We see large differences in !voting behavior, which suggests that there is bias in who people choose to recruit. (From these data we cannot tell whether the bias is an intentional effort to influence consensus, or the result of social network homophily [14].) Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier."
-- gwern http://www.gwern.net
WikiEN-l mailing list WikiEN-l@lists.wikimedia.org To unsubscribe from this mailing list, visit: https://lists.wikimedia.org/mailman/listinfo/wikien-l
On Fri, May 27, 2011 at 9:19 PM, Samuel Klein meta.sj@gmail.com wrote:
This is a nicely competent paper. Thanks for the heads up! SJ
Re-reading, I'm not sure I understand what the results mean. To continue the above quote:
Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier.
To see what effect participant recruitment has on decision quality, we introduce four binary variables: BotRecruit, NomRecruit, DeleteRecruit, and KeepRecruit. These variables indicate whether a bot, the AfD nominator, a delete !voter, or a keep !voter successfully recruited somebody to the group, respectively.
Looking back to table 1, we find that regardless of the decision, none of the first three variables has a statistically significant effect. On the other hand, when a keep !voter recruited someone to the discussion, we see a significant effect: delete decisions are more likely to be reversed.
So:
1. people recruited by a !keep voter (KeepRecruit) also tend to vote !keep 2. people recruited by a !delete voter (DeleteRecruit) tend neither way, both !delete and !keep 2.5. likewise for people recruited (NomRecruit) by the nominator (almost always a !delete voter, obviously) 3. people recruited by a bot (BotRecruit), like 2 & 2.5, have no 'statistically significant effect'
This is a little troubling for anyone who wants to argue that deleting fewer articles is the will of the people - the BotRecruits should then have been more likely to be !keepers.
We offer two possible explanations: the first is that recruitment by keep !voters, biased as it may appear, is a sign of positive community interest, and suggests that the article should be kept. If the community decides otherwise and deletes the article, then decision quality suffers. An alternative explanation is that keep !voter recruitment is a sign of activism among those who prefer to keep the article. These proponents may be especially persistent in maintaining the article’s existence in Wikipedia, even if it requires working to reverse a delete decision."
Obviously I prefer the first interpretation. With that one, the story becomes an article in an obscure niche is put for deletion by a boorish deletionist; in come the specialists who are not ignorant of the topic and literature and save it. If I saw an anime article that should not be deleted up for deletion, I wouldn't ask random Wikipedians to help, I'd go to what pass for anime experts on Wikipedia like Timothy Perper, who can look through the academic literature and have better access to media both English and Japanese. Looks like bias, smells like homophily, but really just the system working.
Interesting,
If I understand this correctly the bots are notifying the wikiprojects based on the wikiproject tags, and in my experience the !voters that attracts will tend to have a good idea as to who is and isn't notable to their project. So I'm not surprised they are a mix of delete and keep.
The Keep !voters are I would hope being selective, and only bothering to try and rescue what they believe is worth rescuing and possible to rescue. So I'm not surprised that they get more keep voters than the average AFD "recruiter".
Amongst the people notified by the nominator I assume you have the author and major editors, and they I would expect to tend to vote keep. Otherwise you have the people who voted in previous AFDs on that subject - that can range from ones who are likely to vote keep because the previous AFD closed as keep, to the probably rarer scenario where the article was deleted, has been recreated but doesn't meet G4. To counterbalance that I'm not sure who else the nominator or a delete voter would be informing who would tend to vote delete.
It would be interesting to see the voting tendency of the other two major groups - regular AFD voters who weren't specifically canvassed but who vote in many AFDs; and watchlisters who for whatever reason have watchlisted the article or the talkpage of its creator. I know I've found myself at AFD trying to rescue an article after seeing an AFD notification on a usertalk page I've been watchlisting.
As for the argument that "deleting fewer articles is the will of the people" I'm not sure we get that many mistakes at AFD. I'm more worried about CSD, and to a lesser extent prods and turning articles into redirects.
WereSpielChequers
On 1 June 2011 20:07, Gwern Branwen gwern0@gmail.com wrote:
On Fri, May 27, 2011 at 9:19 PM, Samuel Klein meta.sj@gmail.com wrote:
This is a nicely competent paper. Thanks for the heads up! SJ
Re-reading, I'm not sure I understand what the results mean. To continue the above quote:
Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier.
To see what effect participant recruitment has on decision quality, we introduce four binary variables: BotRecruit, NomRecruit, DeleteRecruit, and KeepRecruit. These variables indicate whether a bot, the AfD nominator, a delete !voter, or a keep !voter successfully recruited somebody to the group, respectively.
Looking back to table 1, we find that regardless of the decision, none of the first three variables has a statistically significant effect. On the other hand, when a keep !voter recruited someone to the discussion, we see a significant effect: delete decisions are more likely to be reversed.
So:
- people recruited by a !keep voter (KeepRecruit) also tend to vote !keep
- people recruited by a !delete voter (DeleteRecruit) tend neither
way, both !delete and !keep 2.5. likewise for people recruited (NomRecruit) by the nominator (almost always a !delete voter, obviously) 3. people recruited by a bot (BotRecruit), like 2 & 2.5, have no 'statistically significant effect'
This is a little troubling for anyone who wants to argue that deleting fewer articles is the will of the people - the BotRecruits should then have been more likely to be !keepers.
We offer two possible explanations: the first is that recruitment by keep !voters, biased as it may appear, is a sign of positive community interest, and suggests that the article should be kept. If the community decides otherwise and deletes the article, then decision quality suffers. An alternative explanation is that keep !voter recruitment is a sign of activism among those who prefer to keep the article. These proponents may be especially persistent in maintaining the article’s existence in Wikipedia, even if it requires working to reverse a delete decision."
Obviously I prefer the first interpretation. With that one, the story becomes an article in an obscure niche is put for deletion by a boorish deletionist; in come the specialists who are not ignorant of the topic and literature and save it. If I saw an anime article that should not be deleted up for deletion, I wouldn't ask random Wikipedians to help, I'd go to what pass for anime experts on Wikipedia like Timothy Perper, who can look through the academic literature and have better access to media both English and Japanese. Looks like bias, smells like homophily, but really just the system working.
-- gwern http://www.gwern.net
WikiEN-l mailing list WikiEN-l@lists.wikimedia.org To unsubscribe from this mailing list, visit: https://lists.wikimedia.org/mailman/listinfo/wikien-l