Hello! Thanks for trying to not overload the service! There is some minimal documentation on the throttling done by Wikidata Query Service [1], but it clearly needs to be improved. High level overview: Throttling is done by "client". Where client in this case is identified by user-agent and IP address (yes, it is a flawed definition of client, but it mostly works for throttling purpose). Limits are set on the query execution time and on the number of errors raised by the client. When the limits are reached, an HTTP 429 response is sent to the client, with a "Retry-After" HTTP header. This header contains an estimate of how long a client should wait before retrying a request (in seconds). If we see a client that seems to ignore HTTP 429 for long enough, that client is going to be banned for 24 hours. What you can do: * don't execute more than one request in parallel * set a user-agent specific to your application (see [2] for some documentation on the user-agent policy) * when receiving an HTTP 429 response, stop for the duration of the Retry-After header or for 1 minute If you follow all that, you should be good. If you still see throttling / ban, let us know. If you give me the User-Agent of your script and the time at which you received the throttling / ban response, and I can have a look into the logs. Note that we might have some degenerated behaviour when the service is already overloaded (I don't think so, but who knows). Good luck! Guillaume [1] https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual#Query_lim… [2] https://meta.wikimedia.org/wiki/User-Agent_policy On Sat, Nov 2, 2019 at 11:37 AM Andra Waagmeester <andra(a)micel.io> wrote: -- Guillaume Lederrey Engineering Manager, Search Platform Wikimedia Foundation UTC+2 / CEST