Andre Engels wrote:
2007/2/21, Tim Starling
<tstarling(a)wikimedia.org>rg>:
Jeremy Cushman wrote:
Right now on the English Wikipedia bots are
limited to 6 edits per
minute.
Is it time to raise the permitted edit rate? Do
we have the server
capacity
now to handle rapid editing?
If there is a
technical problem with high rate editing then we will limit
the rate by technical means. We can't rely on policy to maintain good
performance. Recommended bot edit rates are there for non-technical
reasons, such as to allow time for review.
Currently there is a technically imposed limit of 8 edits per minute on
IPs and new accounts, to mitigate bot-driven vandalism. There is no limit
on established accounts.
Does this mean I can disable the get-throttling of the bots? Or does this
statement only hold for the edit limit?
Only for the edit limit. And only then because I doubt anyone will try a
high enough edit rate to cause serious load on the server. If someone
starts trying 100 reads per second, it's going to cause problems. And more
so for 100 writes per second.
As a general rule, all bots should be single threaded. This limits the
performance impact they can have on the server -- cheap requests can be
done often, and expensive requests less often. If you run requests in
parallel, then you're using more than your fair share of server resources.
Even single-threaded bots can cause problems if the requests are
particularly long running. If you want to be polite and sleep between
requests, you should throttle them in proportional to the service time --
a duty cycle, in other words. Say if a request takes 0.5s to service, you
could use a duty cycle of 25%, corresponding to one request every 2
seconds. Think about your load on the server in terms of the number of
threads you're tying up, on average.
The worst offenders we've discovered in recent times are the edit counters
-- web based scripts that send requests to the servers often lasting
minutes, with unrestricted parallelism. We were very tempted to block them
all. We would have blocked them for much less if we weren't afraid having
an angry mob of hundreds of Wikipedians obsessed with edit counts,
descending on our door.
-- Tim Starling