Hey On 04/06/2013 03:16 PM, Lydia Pintscher wrote: This was kind of solution I also thought of. I didn't do it, because I was unsure if it's a bug. Cheers Marco

As the dispatch lag is getting bigger again (nearly one day) should wikidata maybe stop all botwork for one day an discuss stricter rules for bots? Or will there be software solution to handle all edits on wikidata? Sk!d *Severin Wünsch* On Sat, Apr 6, 2013 at 6:44 PM, Jeremy Baron <jeremy(a)tuxmachine.com> wrote:

Lydia Pintscher, 07/04/2013 23:59: The bot was blocked and backlog grows by 400.000 items every 24h, so I doubt that's the point. Nemo

Lydia Pintscher, 08/04/2013 00:14: But they're not editing faster that in the last few months, so something is broken. There was that change to preserve en.wiki's DB after all, it's not unrealistic that things slowed down. Nemo

The problem wasn't visible before, so nothing suddenly got broken. Last sunday I counted at one interval 600 edits/min and a lot of those was across several client sites. If I remember correct Daniel said a week or two ago that we could handle about 7K changes per minute. If our present edits are split on individual sites, then it is not unlikely we are simply way above what we can handle. Simplest way to solve this is to use a queue order that works for the bots. Implement a maxlag for change dispatching and set it to 5 minutes as default for bots, or something like that. All editing modules should check the maxlag. A bot that hits the maxlag should incrementally add some delay to its edits, and then slow down. A successful edit allow the bot to decrease the delay. This is slightly different to how other modules are doing this. All bots will then adapt to a maximum throughput with an acceptable lag. An alternative is to not service an edit request before the delay (or some delay) is imposed serverside. There are several variations. A version to enforce the delay is to serve back a special token, a waiting ticket, the bot can use to get its request handled in those cases. Without the token the bot will sit on wait forever. If more iron is added later the throughput will increase and the maxlag will be hit more and more seldom.