Hi Felipe, I can’t follow your reasoning how bots are insignificant. Just as Ziko pointed out, the matrix of bot contributions (and our general experience) tells otherwise. On larger wikipedias bots account for 5-30% of edits on smaller wikis anything up to 50-70% or even more in rare cases. Think of the bots that add interwiki links as primary example of activities that account for massive amount of edits. These may be insignificant on popular articles with 1000’s of edits, but most articles have very few edits, ‘the long tail’ one might call it and there it adds up. Cheers, Erik From: wiki-research-l-bounces(a)lists.wikimedia.org [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Ziko van Dijk Sent: Thursday, November 13, 2008 23:37 To: glimmer_phoenix(a)yahoo.es; Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] "Regular contributor" Hello Felipe, Maybe we speak about different things now. At http://stats.wikimedia.org/EN/BotActivityMatrix.htm de <http://stats.wikimedia.org/EN/TablesWikipediaDE.htm> ja <http://stats.wikimedia.org/EN/TablesWikipediaJA.htm> fr <http://stats.wikimedia.org/EN/TablesWikipediaFR.htm> it <http://stats.wikimedia.org/EN/TablesWikipediaIT.htm> pl <http://stats.wikimedia.org/EN/TablesWikipediaPL.htm> es <http://stats.wikimedia.org/EN/TablesWikipediaES.htm> nl <http://stats.wikimedia.org/EN/TablesWikipediaNL.htm> pt <http://stats.wikimedia.org/EN/TablesWikipediaPT.htm> ru <http://stats.wikimedia.org/EN/TablesWikipediaRU.htm> zh <http://stats.wikimedia.org/EN/TablesWikipediaZH.htm> sv <http://stats.wikimedia.org/EN/TablesWikipediaSV.htm> fi <http://stats.wikimedia.org/EN/TablesWikipediaFI.htm> 8% 6% 22% 25% 26% 15% 29% 30% 26% 15% 23% 22% The bot share of all edits is not that insignificant. Ziko 2008/11/13 Felipe Ortega &lt;glimmer_phoenix(a)yahoo.es&gt; Hi, Erik, and all. IMHO, it would be a good idea...but not definitely an urgent one. In our analyses on the top-ten Wikipedias, we found that bots contributions introduced very few noise in data (to be precise statistically, it was not significant at all). You also have the additional problem that some bots are not identified in the users_group table. My "practical impression" is that when you deal with overall figures, then bots are irrelevant. However, if you want to focus in special metrics like concentration indexes then their contribution DOES MATTER, since a very active bot in one month may ruin your measurments. Regards, Felipe. --- El mié, 22/10/08, Erik Zachte &lt;erikzachte(a)infodisiac.com&gt; escribió: _______________________________________________ Wiki-research-l mailing list Wiki-research-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Ziko van Dijk NL-Silvolde

Hi all. Maybe tomorrow, if I'm not too busy, I will have a detailed graph of what's going on, at least in the top-10 (or top-20) editions. In the meantime, some basic spells on MySQL show this for eswiki: http://pastebin.com/m19859c24 Where perc_logged_revs shows % of bots edits over total number of logged users edits (by month) and perc_revs shows % of bots edits over total number of revs (by month, including annonymous users). The magic of numbers is that they are (most of times) completely objective. Erik, your numbers seems to be fine, also looking at the monthly trends. Maybe my impression (and other past graphs) are only valid for enwiki, but I begin to have my doubts looking at these results. Tomorrow, we will see. All the same, we all agree in that bots should be filtered out, but it was a curious question for me. Best, F.

Felipe, about you second argument, that not all bots are registered as such that (or not anymore, it may change): yes that is a problem. I can only hope that really active bots are ‘caught’ and registered on large wikis. Many bots that are active on many wikis are not registered as such on smaller wikis. Therefore I treat any user name that is registered as bot on 10+ wikis as bot on all wikis. It is of course again an correction which is not 100% accurate, but close I might hope. Single User Logon can help in this respect some day. In theory we could spot some bots by their behavior, say a user that edits 24 hours per day, of manages 5 updates per second for a long time, or added thousands of articles in a short period. But I’m not sure it would be worth the effort, and it would low priority in any case. Erik From: wiki-research-l-bounces(a)lists.wikimedia.org [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Ziko van Dijk Sent: Thursday, November 13, 2008 23:37 To: glimmer_phoenix(a)yahoo.es; Research into Wikimedia content and communities Subject: Re: [Wiki-research-l] "Regular contributor" Hello Felipe, Maybe we speak about different things now. At http://stats.wikimedia.org/EN/BotActivityMatrix.htm de <http://stats.wikimedia.org/EN/TablesWikipediaDE.htm> ja <http://stats.wikimedia.org/EN/TablesWikipediaJA.htm> fr <http://stats.wikimedia.org/EN/TablesWikipediaFR.htm> it <http://stats.wikimedia.org/EN/TablesWikipediaIT.htm> pl <http://stats.wikimedia.org/EN/TablesWikipediaPL.htm> es <http://stats.wikimedia.org/EN/TablesWikipediaES.htm> nl <http://stats.wikimedia.org/EN/TablesWikipediaNL.htm> pt <http://stats.wikimedia.org/EN/TablesWikipediaPT.htm> ru <http://stats.wikimedia.org/EN/TablesWikipediaRU.htm> zh <http://stats.wikimedia.org/EN/TablesWikipediaZH.htm> sv <http://stats.wikimedia.org/EN/TablesWikipediaSV.htm> fi <http://stats.wikimedia.org/EN/TablesWikipediaFI.htm> 8% 6% 22% 25% 26% 15% 29% 30% 26% 15% 23% 22% The bot share of all edits is not that insignificant. Ziko 2008/11/13 Felipe Ortega &lt;glimmer_phoenix(a)yahoo.es&gt; Hi, Erik, and all. IMHO, it would be a good idea...but not definitely an urgent one. In our analyses on the top-ten Wikipedias, we found that bots contributions introduced very few noise in data (to be precise statistically, it was not significant at all). You also have the additional problem that some bots are not identified in the users_group table. My "practical impression" is that when you deal with overall figures, then bots are irrelevant. However, if you want to focus in special metrics like concentration indexes then their contribution DOES MATTER, since a very active bot in one month may ruin your measurments. Regards, Felipe. --- El mié, 22/10/08, Erik Zachte &lt;erikzachte(a)infodisiac.com&gt; escribió: _______________________________________________ Wiki-research-l mailing list Wiki-research-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l -- Ziko van Dijk NL-Silvolde

Felipe Ortega wrote: He has returned, ~500 edits this week ;) The problem is that here you need the edits *per user*, not per page. I understand from the WikiXRay page that you're recreating the mediawiki tables. It'd just to query each user contributions and check the time difference. With indexes in place, you would get a time good enough. When it may get terribly slow is if applying to all users, as you would make the algorithm quadratic.