Define the shape of a event by encapsulating the payload in event.data.

[MSNev]

Changes

This PR extends the event specification to define the "shape" of an event by encapsulating the payload (content) of the event in the event.data attribute.

This defines how events with a payload should be defined, this enables support for downstream systems to handle both known known domain-specific events (identified by event.domain and event.name) and any other arbitrary new or vender specific events.

Downstream systems can safely assume that when a Log record contains the event.domain, event.name and event.data that the Log record represents an event and can then perform any validation or storage (routing) changes that the vender may require.

This provides support not only for events being defined by the RUM Sig, but also includes support for the mapping of CloudEvents, which can be defined by a later PR.
Possible CloudEvents mapping Example:

• event.domain: "cloudevents"
• event.name: "xxxx"
• event.data: (The cloudEvents data)
  With Cloud Events Context Attributes (Required and optional) being mapped to top level OpenTelemetry Attributes of the same name

There was a discussion around using the "Body" of a log message as the transport for the payload instead of defining an event.data, however, while the Logs is intended to be the primary transport mechanism events the definition of an event is not limited to Logs only. This allows for an event to be transmitted as a span event (which doesn't have a "Body") and also supports direct logical mapping for CloudEvents.

[ronyis]

I think that log events should also support representing the payload of application requests. For example, in order for a user to collect the response body of an outgoing HTTP request, saving it as part of a span is problematic since the body could be read after the span representing the request is closed (many times it's also read asynchronously).

Using a map<string, any> is not generic enough for this example, using any is preferable.

I'm also in favor of using the log Body instead of an attribute. For the case of using a Span Event, I think we should be able to support an any type attribute as well (similar to as proposed in #2888).

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[scheler]

not stale

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.

[jack-berg]

The problems that this solves are too abstract for me support at this time. In particular, it's not clear how downstream validation of event.data would work and why including the event payload as top level attributes prevents this. There's a chance of collision, but is this a theoretical problem or one we've seen in practice? From a validation standpoint, a receiver of events in possession of a schema should be able to validate the known fields according to the schema, ignoring unrecognized fields. It's free to drop or accept unrecognized fields needed. This appears to be true whether or not the payload is encapsulated in a single field.

Based off discussions in the 2/22/23 log sig, this change seems important to the client instrumentation working group. If this inclusion is in fact blocking progress, can a demo be given which shows concrete situations where including event data as top level attributes fails?

[MSNev]

In particular, it's not clear how downstream validation of event.data would work

I have attempted to not restrict or define how (or if) downstream validation would occur as some vendors will and some wont. And others may choose to provide validation as a development (non-production) client side validation tool.

and why including the event payload as top level attributes prevents this.

Simplistically, It doesn't. If someone wants to construct a "Custom" event which only uses top-level attributes they could, as part of this specification definition I'm saying they SHOULDN'T because of the chance of collisions and to simplify how back-end systems can identify, process and store "Events". (Which can also be viewed as a specific Structured Log records). There are additional reasons why specifying that event detail is passed in a single attribute (Which I'll include below -- I explicitly left out of the spec detail). This is also the one of the primary reasons for choosing to send "events" via Logs as it IS the only signal that supports nested attributes. Otherwise, we could have just continued to hack zero-length Span's with span Event (which has additional issues)

There's a chance of collision, but is this a theoretical problem or one we've seen in practice?

Because many of the RUM events are defined by scraping and passing data based on existing W3C standards (at least for browser events), these names are fixed and as they evolve additional names WILL (and have) be added. The only way to avoid the collisions would be to prefix EVERY field which has performance, memory and payload considerations (all of which are major issues for client runtimes, whether a browser, mobile device, IOT or embedded into "SMART Tv's". (Yes, I've supported this)

Why use a single attribute

Without encapsulating the event payload into a single nested attribute, the following (or combination) would need to occur, whether to avoid collisions or not.

• prefix every "value" with a common prefix (eg event.<name / data>.xxxx) this will cause an explosion with the payload size, especially when the event data is large like W3C resource timing fields (these are used for page load as well Ajax requests and will include the "resources" (scripts, css, images) loaded as part of page load)
• Each "new" attribute name should be "added" to semantic conventions to avoid using the same name for different things / semantics (or be prefixed), this will slow down the event definition even further than it already is and would be unnecessary when the fields are only required for a single event. We have a specific Goal is to AVOID defining individual attribute names, we explicitly want to define the semantic conventions (collection of value / names) at the "Event" level only.
• Generally RUM systems can have MANY (10s, 100s) of specific events, and it's not just a "Span" / "Log", each event has a specific meaning for monitor real user interactions with a system. And depending on the application (or observability vendor) there may have many/many/many custom events (beyond what we will define as part of the standard OpenTelemetry Events) and will need to be supported.

I've mentioned payload size before, but let me restate here, during the "Page/Application unload" (user navigates to another page, closes the browser, switches tabs and on mobile devices turns the device (screen) off - forcing the app to hibernation (and potentially never wake up)), so at this point any SDK should attempt to send all observability content (as it may not have any persistence capability or even a chance to re-hydrate and send (eg. classic examples of this are crashes).

During this "unload" state modern browsers have a restricted set of API's to allow the "delivery" of any network requests, both of these have a MAXIMUM payload of 64kb (of total outstanding (unsent) requests -- you can't just send multiple 64kb payloads you can send 1 total or several which add up to 64kb), and while there is currently an experimental compression API (it's not supported on Firefox) and it appears that it can't be supported by the primary sending API (because we can't include headers informing the receiver that the HTTP request is compressed (gzipped)), likewise on unsupported browsers including the additional client side code (which increases the page load time) also doesn't help, because you still can't add the additional header telling the receiver that the payload is compressed....

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[MSNev]

Not stale

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.

[MSNev]

Waiting for outcome of #3406