-
Notifications
You must be signed in to change notification settings - Fork 188
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support compression in OTel sources (trace, logs, metrics) #1176
Comments
Thank you for making this feature request. This is a valuable feature especially since the OTel Collector is using compression by default. We need to determine if our current Armeria version supports compression or if we need to update. We are currently blocked on updating Armera per #924 . |
There is probably no out-of-the-box protobuf compression supported in Armeria: https://javadoc.io/doc/com.linecorp.armeria/armeria-javadoc/latest/com/linecorp/armeria/server/grpc/GrpcServiceBuilder.html#enableUnframedRequests(boolean). Will confirm with their community. |
From community help, EncodingService might help resolve the issue:
|
Thanks @chenqi0805 for looking into this. It does appear that this class is in Armeria 1.9.2 which are currently on. So this might be something we can do even without updating Armeria (though we still should do that). |
Actually DecodingService is what we need. Right now it only supports gzip decoding and does not require armeria upgrade. |
Please support in all three sources:
|
Is your feature request related to a problem? Please describe.
By default, OpenTelemetry Collector compress its requests body (along with Content-Encoding header). Otel-trace-source is not supporting this (responding with
INTERNAL: Invalid protobuf byte sequence
) and it needs to be turned off in the OpenTelemetry Collector configuration.Describe the solution you'd like
Data Prepper should accept Content-Encoding header and adjust to it.
Describe alternatives you've considered (Optional)
Alternatively, it can let users configure which compression they intend to use, however it requires coordinated changes on both Collector and Prepper, so following HTTP header guidance is preferable.
Additional context
Found while reporting #1152
The text was updated successfully, but these errors were encountered: