"Well-behaved" is, I fear, in the eye of the beholder. I dearly want
to be
well-behaved, and a good citizen in all ways, but I am still not sure
I know
what the considerations are. Was I 403'ed because the actual volume of
transactions was too high, or simply because I was using "remote
loading"?
Generally, if your volume's big enough to be noticed leeching page views
and searches off our main servers, you've got a fair chance of being
blocked. (I seem to remember hearing something about the search engine
being hit in this case, but I may be thinking of something else.)
results in this request on Wikipedia:
209.68.1.165 - - [12/Jun/2005:03:29:09 +0000] "GET