The schema registry is especially needed if you want to do Avro in streaming context. In a streaming context, you don’t have the schema shipped along with the data, only the data. So, I’m pretty sure that they store a message schema id in the Kafka message key, so that when you read from kafka you can easily look up the schema associated with a message.

On Feb 25, 2015, at 15:23, Toby Negrin <tnegrin@wikimedia.org> wrote:

The schema manager is _really_ interesting. Can we take it for a spin?

-Toby

On Wed, Feb 25, 2015 at 12:07 PM, Andrew Otto <aotto@wikimedia.org> wrote:
Whoa, Confluent (Kafka folks) just packaged up everything we've been building over the last two years:

http://blog.confluent.io/2015/02/25/announcing-the-confluent-platform-1-0/

http://confluent.io/docs/current/platform.html

http://blog.confluent.io/2015/02/25/stream-data-platform-1/

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics