On 01/03/2013 16:46, Asher Feldman wrote:
I don't think a custom daemon would actually be needed.
While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010 was orders of magnitude greater than the rate of edits across all wmf projects. Publishing to a redis pubsub channel does grow in cost as the number of subscribers increases but I don't see a problem at our scale. If so, there are ways around it.
This is basically the way the new Disqus platforms works also - with a thin http proxy infront and the client side (public, javascript) doing 'sort of long' polling.
Quite an interesting talk from europython - https://www.youtube.com/watch?v=PeVB5DNptD4 - if you're into that sort of thing.
- Damian