Whoa, Confluent (Kafka folks) just packaged up everything we've been building over the last two years:

http://blog.confluent.io/2015/02/25/announcing-the-confluent-platform-1-0/

http://confluent.io/docs/current/platform.html

http://blog.confluent.io/2015/02/25/stream-data-platform-1/