Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capture request and response bodies #857

Open
pavolloffay opened this issue Oct 6, 2020 · 15 comments
Open

Capture request and response bodies #857

pavolloffay opened this issue Oct 6, 2020 · 15 comments
Assignees

Comments

@pavolloffay
Copy link
Member

What are you trying to achieve?

I want to capture request and response bodies. The content can be used for business transaction monitoring and security.

To move this forward we need:

  • data semantics e.g. http.request.body/http.response.body or be transport agnostic e.g. request.body, response.body.
  • a configuration for auto instrumentations e.g. OTEL_INSTRUMENTATION_CAPTURE_REQUEST_BODY=bool

Additional context.

@Oberon00
Copy link
Member

Oberon00 commented Oct 6, 2020

Related open-telemetry/opentelemetry-specification#1061 (comment)

EDIT: But here it may make sense to keep in one issue to define if we even want to support this at all. It seems problematic, both from a performance and also privacy perspective.

@pavolloffay
Copy link
Member Author

EDIT: But here it may make sense to keep in one issue to define if we even want to support this at all. It seems problematic, both from a performance and also privacy perspective.

This is an opt-in feature not enabled by default only users who need this would enable it with all the implications of course.

@jbadeau
Copy link

jbadeau commented Apr 22, 2021

This would be very useful for certain scenarios

@alfkonee
Copy link

Hello please has there been any progress made here. This specification would be really helpful in many enterprise Environments for also in many app dev scenarios for distributed trace of app data flow validations

@edtro
Copy link

edtro commented Jul 27, 2022

upvoted this request :-) would be really usefull

@iquirino
Copy link

+1

@gmreads
Copy link

gmreads commented Jan 30, 2023

@pavolloffay Is this still a priority for you ? Were there any discussions within the Otel team ?
I've been working on shipping custom sdks which capture request bodies.

How can we take this forward ? I would really like to make this happen. @Oberon00

@pavolloffay
Copy link
Member Author

I haven't looked into this ticket since October 2020

@trask
Copy link
Member

trask commented Jan 31, 2023

@gmreads check out open-telemetry/oteps#219

@nirga
Copy link
Contributor

nirga commented Jun 4, 2023

I've been looking into this, and read through open-telemetry/oteps#219 .
The short background is that we need a way to get the body of requests and responses through OpenTelemetry since this is how we generate and run tests at Traceloop. I don't think this is out of the ordinary for an observability protocol. Looking at tools like Sentry and others, you can regularly observe request/response bodies (with some obfuscation ofc).

Today, we patch the instrumentation for our clients so we can get the request body but the main problem is that there's no standard for this. I propose we standardize the semantic attribute name of http.request.body so that potential extensions to the standard SDK can tell which attribute name to use if they want to export this information.

Would love to start working on this.

@jack-berg
Copy link
Member

Some context: open-telemetry/opentelemetry-specification#2888 proposed extending attributes to support complex types like maps and heterogenous arrays. If added, this would be a natural place to put http request / response bodies. Unfortunately, we didn't achieve consensus and the PR is currently closed. That effort could be restarted, but I think it would take new use cases and / or new arguments to move the needle.

@nirga
Copy link
Contributor

nirga commented Jun 19, 2023

Open a new PR for that open-telemetry/oteps#234

@axlrate
Copy link

axlrate commented Mar 12, 2024

(I asked this on oteps 234 as well)

Should the payload be a span event instead of a span attribute? Most backends have higher size limits for span events.

@trask trask transferred this issue from open-telemetry/opentelemetry-specification Mar 29, 2024
@julealgon
Copy link

Should the payload be a span event instead of a span attribute? Most backends have higher size limits for span events.

That doesn't make sense from a semantics point of view though... a piece of request/response data is not "an event" IMHO. And the HTTP span itself already represents a request or response, so there is no room to define a "send event" inside of that in my mind.

@chameleon82
Copy link

I would prefer to have body and other request/response attributes as a structured log. Moreover, to have the ability to separate them into different pipelines from other logs (different security and rotation policies). It is make sense to me to be able to select them by certain parameters and groups and able to process via anomaly detection tools.
Currently, I do it this way already, but I had to write extra instruments and believe spec here will be helpful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests