Unique registered bots drop in February 2013 - Analytics - lists.wikimedia.org

List overview All Threads
Download

Unique registered bots drop in February 2013

Y jjjjuyug

Doing housekeeping work on...

Dario Taraborelli

17 Jan 2014 17 Jan '14

4:09 a.m.

I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams-fyukCIlMdH… Thanks, Dario

Reply

Show replies by thread

Steven Walling

17 Jan 17 Jan

4:15 a.m.

On Thursday, January 16, 2014, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:

I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams-fyukCIlMdH… Thanks, Dario

Do we have usernames/ids for those bots that stopped?

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org <javascript:;> https://lists.wikimedia.org/mailman/listinfo/analytics

-- Steven Walling, Product Manager https://wikimediafoundation.org/

Reply

Dario Taraborelli

4:18 a.m.

yes, I’m just pulling some year-long data on bot activity and wanted to exclude something trivial (like some Wikidata bot activity in early 2013 that may have suddenly ceased in February?) On Jan 16, 2014, at 8:15 PM, Steven Walling <swalling(a)wikimedia.org> wrote:

On Thursday, January 16, 2014, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote: I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams-fyukCIlMdH… Thanks, Dario Do we have usernames/ids for those bots that stopped? _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics -- Steven Walling, Product Manager https://wikimediafoundation.org/ _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Dario Taraborelli

6:51 a.m.

Kunal sent me this note off-list explaining the observed drop in unique bot activity:

I'm not on the analytics list, but I saw your email about bot activity dropping in Feb 2013 on enwiki (http://lists.wikimedia.org/pipermail/analytics/2014-January/001515.html), and the reason is that all interwiki bots were shutoff due to the deployment of Wikidata.https://en.wikipedia.org/wiki/Wikipedia:Bot_owners%27_noticeboard/… is the relevant thread with some info.

Dario On Jan 16, 2014, at 8:18 PM, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:

yes, I’m just pulling some year-long data on bot activity and wanted to exclude something trivial (like some Wikidata bot activity in early 2013 that may have suddenly ceased in February?) On Jan 16, 2014, at 8:15 PM, Steven Walling <swalling(a)wikimedia.org> wrote:

On Thursday, January 16, 2014, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote: I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams-fyukCIlMdH… Thanks, Dario Do we have usernames/ids for those bots that stopped? _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics -- Steven Walling, Product Manager https://wikimediafoundation.org/ _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Denny Vrandečić

7:33 a.m.

Wikidata. No interwiki bots needed anymore :) On Thu Jan 16 2014 at 8:09:41 PM, Dario Taraborelli < dtaraborelli(a)wikimedia.org> wrote:

I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams- fyukCIlMdHZPbGlaMXdkbjdYZ0RKWnUxWTBXUVE#gid=0 Thanks, Dario _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Federico Leva (Nemo)

6:20 p.m.

I don't know what a "registered bot" is, but we'll see a drop in new bot accounts, flagged accounts as well as active accounts. On wikis where bureaucrats are less eager to "retire" bots you won't see mass deflags, of course, only the rest. Nemo

Reply

Dario Taraborelli

11:20 p.m.

Indeed, I should have looked up the Wikidata milestone timeline. I added a series of total daily edit counts by registered bots and it spikes right after the bulk shutdown of interwiki bots. On Jan 16, 2014, at 11:33 PM, Denny Vrandečić <vrandecic(a)gmail.com> wrote:

Wikidata. No interwiki bots needed anymore :) On Thu Jan 16 2014 at 8:09:41 PM, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote: I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams-fyukCIlMdH… Thanks, Dario _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Dario Taraborelli

11:27 p.m.

I call a “registered bot" a registered user of a given project who is member of user_groups.ug_group = "bot". We’re currently working on evaluating different methods for bot categorization, both historically and across projects, as part of an effort to standardize key participation metrics. We’re also considering the possibility of a dedicated bot group purely for analytics purposes, i.e. not associated with any permission [1]. We’ll post the results of the analysis on Meta and we will solicit feedback from people on this list on the merits of different proposals/definitions. [1] https://meta.wikimedia.org/wiki/Research:Bot On Jan 17, 2014, at 10:20 AM, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote:

I don't know what a "registered bot" is, but we'll see a drop in new bot accounts, flagged accounts as well as active accounts. On wikis where bureaucrats are less eager to "retire" bots you won't see mass deflags, of course, only the rest. Nemo _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Reply

Andrew Gray

18 Jan 18 Jan

8:27 a.m.

The spike is also expected, I think. Wikidata and bots went like this: * Status quo: dozens of interwiki bots making rare maintenance edits to lots of articles * Wikidata appears; the interwiki bots are no longer needed. The regular interwiki maintenance jobs are stopped, meaning dozens of bots stop doing anything, mostly overnight. * Because Wikidata is now working, 95%+ of in-text wikilinks are obsolete and can be removed. A couple of bots do this over a few weeks, editing the vast majority of articles across all projects to remove the text interwikis - so the overall bot edits go up despite number of bots going down. * New status quo: no mass removal needed any more, and no regular maintenance jobs going on. Andrew. On Friday, 17 January 2014, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:

Indeed, I should have looked up the Wikidata milestone timeline. I added a series of total daily edit counts by registered bots and it spikes right after the bulk shutdown of interwiki bots. On Jan 16, 2014, at 11:33 PM, Denny Vrandečić <vrandecic@gmail.com<javascript:_e({}, 'cvml', 'vrandecic(a)gmail.com');>> wrote: Wikidata. No interwiki bots needed anymore :) On Thu Jan 16 2014 at 8:09:41 PM, Dario Taraborelli < dtaraborelli(a)wikimedia.org <javascript:_e({}, 'cvml', 'dtaraborelli(a)wikimedia.org');>> wrote:

I noticed that the number of daily unique registered bots with at least 1 ns0 edit on enwiki saw a steep drop in February 2013. Does anybody have a clue about the cause, before I dig further into this data? https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Ams- fyukCIlMdHZPbGlaMXdkbjdYZ0RKWnUxWTBXUVE#gid=0 Thanks, Dario _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org <javascript:_e({}, 'cvml', 'Analytics(a)lists.wikimedia.org');> https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org <javascript:_e({}, 'cvml', 'Analytics(a)lists.wikimedia.org');> https://lists.wikimedia.org/mailman/listinfo/analytics

-- - Andrew Gray andrew.gray(a)dunelm.org.uk

Reply

Federico Leva (Nemo)

9:54 a.m.

Dario Taraborelli, 18/01/2014 00:27:

I call a “registered bot" a registered user of a given project who is member of user_groups.ug_group = "bot".

Ok. Standard terminology for that is "flagged bot".

We’re currently working on evaluating different methods for bot categorization, both historically and across projects, as part of an effort to standardize key participation metrics. We’re also considering the possibility of a dedicated bot group purely for analytics purposes, i.e. not associated with any permission [1]. We’ll post the results of the analysis on Meta and we will solicit feedback from people on this list on the merits of different proposals/definitions. [1] https://meta.wikimedia.org/wiki/Research:Bot

I commented there. Nemo

Reply

3750

days inactive

3751

days old

analytics@lists.wikimedia.org

Manage subscription

9 comments

5 participants

tags (0)

participants (5)

Andrew Gray
Dario Taraborelli
Denny Vrandečić
Federico Leva (Nemo)
Steven Walling