Hi,
I have an application that every five minutes or so comes up with a list of 50-500 Wikipedia individual documents that it wants to request via curl. It wants to get them as quickly and as completely as possible with minimal missed documents or dropped connections.
1) what is the etiquette for how to do this politely?
2) what are the best practice for using curl to submit API requests?
* is it best to use a single curl command with a long list of URLs to get, or a long list of individual curl commands?
* compressed or not?
* retry values?
* max-redirs?
* other parameters?
FredZ
-----------------------------------------------------