-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow the specification of the schema-registry subject when creating avro tables/stream #1994
Comments
Hello, Error:
If I get subjects from schema registry, test-topic-value subject doesn't exist, so I think I have to set the schema-registry subject when creating avro tables/stream. |
This would be a really interesting feature, that will open up many possibilities, especially if we consider that (as per the excellent confluentinc/schema-registry#680), schema registry already supports the publishing of different AVRO objects on a topic. What I would like to do in KSQL CLI is something like the following: CREATE STREAM xxx AS SELECT ... WITH (KAFKA_TOPIC='topic1', VALUE_FORMAT='AVRO', SCHEMA='topic1-com.mypackage.MyMessage") which creates a stream that is filtered on the specific type of message, even when the topic hosts a plethora of different types of objects. Ok, I perfectly know that a possible answer could be "you can create as many topics as the objects you want to handle", but there are cases in which it is important to publish different events (related to the same entity, that is, events that share the same partition key) in a topic, in order to keep the ordering of events. |
Actually, this is a duplicate of #1267. Up to you to close it or link it to the original one. |
I don't think this is a duplicate of #1267 as that one is about multiple schemas per topic and here I understand that we want a way to reuse an already existing AVRO schema when creating a new KSQL stream. But this issue may be a duplicate of #2427 where I have proposed something similar to what @darkwings proposed above (a SCHEMA argument on the WITH clause to set which AVRO schema should be used). For me it would be great to reuse AVRO schemas on KSQL Streams. Or at least there should be a way to add documentation on the auto-generated AVRO schemas create by KSQL. |
related to #3634 |
Currently when we create tables and streams we infer the avro subject from the topic name. This works great if everything has been published via the confluent stack, however, there are cases where this might not be the case and people want to share a schema across multiple topics. It would be nice to add an extra value to the
WITH
clause such that an operator can set the subject to a different valueThe text was updated successfully, but these errors were encountered: