We don’t do much with Avro data…yet, if/when we do, for sure!
The schema registry is especially needed if you want to do Avro in streaming context. In a streaming context, you don’t have the schema shipped along with the data, only the data. So, I’m pretty sure that they store a message schema id in the Kafka message key, so that when you read from kafka you can easily look up the schema associated with a message.
On Feb 25, 2015, at 15:23, Toby Negrin tnegrin@wikimedia.org wrote:
The schema manager is _really_ interesting. Can we take it for a spin?
-Toby
On Wed, Feb 25, 2015 at 12:07 PM, Andrew Otto <aotto@wikimedia.org mailto:aotto@wikimedia.org> wrote: Whoa, Confluent (Kafka folks) just packaged up everything we've been building over the last two years:
http://blog.confluent.io/2015/02/25/announcing-the-confluent-platform-1-0/ http://blog.confluent.io/2015/02/25/announcing-the-confluent-platform-1-0/
http://confluent.io/docs/current/platform.html http://confluent.io/docs/current/platform.html
http://blog.confluent.io/2015/02/25/stream-data-platform-1/ http://blog.confluent.io/2015/02/25/stream-data-platform-1/
Analytics mailing list Analytics@lists.wikimedia.org mailto:Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics