-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove alert for pre-existing new columns while merging realtime schema #16989
Remove alert for pre-existing new columns while merging realtime schema #16989
Conversation
"Realtime schema merge: datasource [%s] is missing columns to be updated. " | ||
+ "ExistingSignature [%s], deltaSchema [%s], missingUpdateColumns [%s].", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The alert description should be small. Don't include big objects like the existingSignature
or deltaSchema
in it. You may add these fields separately using .addData()
.
existingNewColumns | ||
).emit(); | ||
if (!missingUpdateColumns.isEmpty()) { | ||
log.makeAlert("Datasource schema is missing columns to be updated in delta realtime segment schema.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The message is confusing. Could you try to rephrase this to say exactly what went wrong and what can be done to remediate?
boolean missingUpdateColumns = false; | ||
// new column to be added is already present in the existing schema | ||
boolean existingNewColumns = false; | ||
Set<String> missingUpdateColumns = new HashSet<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit:
Set<String> missingUpdateColumns = new HashSet<>(); | |
final Set<String> missingUpdateColumns = new HashSet<>(); |
…ma (apache#16989) Currently, an alert is thrown while merging datasource schema with realtime segment schema when the datasource schema already has update columns from the delta schema. This isn't an error condition since the datasource schema can have those columns from a different segment. One scenario in which this can occur is when multiple replicas for a task is run.
Parent issue: #14989
Currently, an alert is thrown while merging datasource schema with realtime segment schema when the datasource schema already has update columns from the delta schema.
This isn't an error condition since the datasource schema can have those columns from a different segment.
One scenario in which this can occur is when multiple replicas for a task is run.
This PR has: