Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AttributePropagatingSpanProcessor component to AWS X-Ray #856

Merged
merged 4 commits into from
May 9, 2023

Conversation

thpierce
Copy link
Contributor

@thpierce thpierce commented Apr 27, 2023

Context

AttributePropagatingSpanProcessor is designed to propagate attributes from parent to child spans, which will later be used to generate metrics under the background of #789. The processor also propagates span names from local root server/consumer spans into a specifiable attribute

Description

AttributePropagatingSpanProcessorBuilder constructs a AttributePropagatingSpanProcessor in a configurable manner, but the default behaviour creates it with aws.remote.service and aws.remote.operation as propagatable attributes and aws.local.operation as the span name propagation attribute. The aws.remote.service and aws.remote.operation attributes are designed to be set by users into local client spans through manual instrumentation. However, since client spans are mostly generated by auto-instrumentation libraries which make them unaccessible to service developers. SpanMetricsProcessor from #789 requires aws.local.operation (the initial entry point of a execution) to be injected into the local client spans to generate metrics that can later be used for other correlations.

AttributePropagatingSpanProcessor accomplishes two things:

1. Propagating attributes.

OTEP-207 seems to be a general solution that could help address this requirement, but it hasn't been finalized yet. We took the idea from this comment which suggests to let SpanProcessor to do the propagation work.

AttributePropagatingSpanProcessor is used as an alternative solution that propagate desired attributes from parent span to the child spans if present.

2. Setting and propagating span name attributes.

AwsAttributePropagatingSpanProcessor sets a configurable attribute with the span name of the local root span if the request is from remote, and passes it to child spans.

Testing

JUnit test is created for this new component in the package.

Documentation

It's an independent component that would require manual installation in the TracerProvider. We'll create a doc to describe the usage.

Outstanding items

NONE.

@thpierce thpierce changed the title Aws attributes processor Add AwsAttributeKeys to awsxray Apr 27, 2023
@thpierce thpierce changed the title Add AwsAttributeKeys to awsxray Add AwsAttributePropagatingSpanProcessor component to AWS X-Ray Apr 27, 2023
@thpierce thpierce marked this pull request as ready for review April 27, 2023 20:53
@thpierce thpierce requested a review from a team April 27, 2023 20:53
@srprash
Copy link

srprash commented Apr 27, 2023

Please update aws.remote.application to aws.remote.service in the PR description as well.

@thpierce thpierce force-pushed the aws_attributes_processor branch 2 times, most recently from 437c032 to 656bda2 Compare April 28, 2023 17:01
@thpierce thpierce requested a review from srprash April 28, 2023 17:05
@thpierce
Copy link
Contributor Author

@trask/other maintainers - review seems blocking on @willarmiros, but as per #820, the AWS component owner has changed to @srprash, who has approved, so I think we are able to move forward to maintainer review.

@trask
Copy link
Member

trask commented Apr 28, 2023

hey @thpierce @srprash!

it's not clear to me whether this is specific to AWS X-Ray backend or specific to AWS cloud?

we generally don't host monitoring vendor components in OpenTelemetry repositories (e.g. Splunk, Lightstep, Honeycomb all host their vendor-specific components in their own repositories).

we do host cloud specific things in OpenTelemetry repositories though since those are useful for users of all monitoring vendors

@thpierce
Copy link
Contributor Author

thpierce commented May 1, 2023

@trask thanks for taking a look, we will evaluate this feedback internally and come back with a reply soon, potentially with a revised PR. Concerns are understood. For now, the following PRs will be on pause, hope to "unpause" by mid-week:

@trask
Copy link
Member

trask commented May 2, 2023

hey all, sorry, I was mistaken, it's ok for contrib repositories to have vendor-specific components. let me know if you are ready and I will merge this, thx!

@thpierce thpierce marked this pull request as draft May 2, 2023 21:21
In this commit, we are updating AttributePropagatingSpanProcessorBuilder to be somewhat more flexible in use. We still default to the behaviour originally implemented, but we expose the ability to specify span attributes that are used for propagation. This will make the component more reusable, while still achieving the goals required by AWS.
Also, we are renaming Application terminology to Service terminology.
@thpierce thpierce changed the title Add AwsAttributePropagatingSpanProcessor component to AWS X-Ray Add AttributePropagatingSpanProcessor component to AWS X-Ray May 5, 2023
@thpierce
Copy link
Contributor Author

thpierce commented May 5, 2023

Updated PR to make it somewhat more configurable. The default behaviour is to propagate desired attributes for AWS, but others can make use of this processor by specifying different attribute keys. Should be no functional regression from the previous PR status.

@thpierce thpierce requested a review from srprash May 5, 2023 21:39
@thpierce thpierce marked this pull request as ready for review May 5, 2023 21:40
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this processor is now somewhat generic, would it make sense to extract it outside of contrib/awsxray to some shared package?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We slightly prefer to keep it in xray as the default behaviour is specific to our use-case. We made it more generic so others could use if desired, but this was still ultimately built for our usecase. If @trask/other maintainers feel this would be better placed outside this package/thinks that this is a fairly generic solution, we are open to moving it to another location (and honestly, removing the aws/xray-specific default would not be terribly difficult either). I'm happy to do what the community thinks is best.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree better to keep this in awsxray component for now, especially since the span name logic is fairly specific to this use case

@thpierce
Copy link
Contributor Author

thpierce commented May 5, 2023

@trask I think this is ready for your review - please note the comment here: #856 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree better to keep this in awsxray component for now, especially since the span name logic is fairly specific to this use case

@trask trask merged commit e24705d into open-telemetry:main May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants