Hello all,
As you may know, over the past months we’ve been struggling with more and more bots editing Wikidata at a very high rate, causing infrastructure issues having an impact on the Query Service that couldn’t keep up with the changes and on tools such as Pywikibot https://phabricator.wikimedia.org/T242081.
Over the years, we tried different things (like Add Wikidata query service lag to Wikidata maxlag https://phabricator.wikimedia.org/T221774, increase maxlag or factor https://phabricator.wikimedia.org/T243701#5864130, limit the edit rate for all accounts https://phabricator.wikimedia.org/T184948). Wikidata admins also approached individual bot owners to ask them to comply with the bot policy’s limit, sometimes without success.
Recently, we discussed removing the noratelimit feature for the bots group https://phabricator.wikimedia.org/T258354. This would have as an effect to *limit the edits to 90 edits/min for most of the bots* ; the few bots that need an unlimited rate to function (for example MassMessage) can be added to the existing accountcreator group (with the possibility to rename this group).
Many thanks to bot owners who gave input in the comments of the ticket https://phabricator.wikimedia.org/T258354 and helped us frame this solution. If you want to continue the discussion, please have a look at these comments, so we can build from them and avoid restarting the discussion from scratch.
We hope that this solution will allow a fair access to mass editing Wikidata, while preserving the existing infrastructure and avoiding hitting too hard on the Query Service. In the meantime we are working together with the Search Team at the WMF and others on improvements to the Query Service scalability and alternatives to it so some load can be redirected to other systems, as well as general infrastructure improvements.
Unless there’s a strong opposition from the community to this change, we will implement it on October 20th.
If you have any questions or need more information, feel free to add a comment in the ticket https://phabricator.wikimedia.org/T258354 or onwiki https://www.wikidata.org/wiki/Wikidata:Project_chat#Remove_noratelimit_for_bots to avoid too many messages on these mailing-lists.
Thanks,