Jimmy Wales wrote:
My thought is that a well behaved application should produce no more load than people surfing our site with a traditional browser. But it is possible (likely, even) that some applications will not be well-behaved.
If we have a single generic interface where everyone pulls in exactly the same way, then we have no way to block abusive applications without blocking everyone.
Something as simple as requiring a "user agent" string might be enough?
This issue is quite similar to the issue of people pulling pages from our site "live" to make a mirror. It's not a good thing to do if done abusively. But with web hosts, it is easy enough to simply block them if they misbehave. A misbehaving application will come in through many ip numbers.
I have been thinking a little bit as well about a model with full blown keys (like the Google API) -- the point could be that for free software, we can give out free keys and support those users at our own expense. But for proprietary software, we can charge money for the keys.
For server-side systems (mirrors and other sites including our bits) something like the Google API's keys would work ok. A relatively small number of server admins will be setting things up, and the keys will be 'secret', kept on that server.
Things are a bit different with the client-side software integration envisioned by the KDE folks; like a web browser these apps will be distributed to thousands of users and should "just work". A per-user key setup would probably not be acceptable, while a non-secret key distributed with the app (like a user-agent string) can be easily ripped and faked by someone less scrupulous.
We probably want client-side apps to be able to act like a browser (easy, no manual setup), while trying to avoid server abuse by someone who decides to snarf a million pages a day; a per-IP rate limit on the 'free software key' is probably sufficient to automatically curb server-side abuse.
A separate policy for proprietary client-side apps would have to be policed on the honor system; someone could always have their client claim to be amaroK and we'd never be the wiser. :)
Blocking a client-side app that identifies itself with a unique user-agent or key, but behaves badly, would be easy. Blocking an app that behaves badly _and_ pretends to be legitimate software is a harder problem. (Sometimes you can distinguish between the real and the fake app by its behavior or a quirk of its HTTP submission formatting, sometimes that might not be easy.)
-- brion vibber (brion @ pobox.com)