Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for tracing runnable scenarios #8831

Merged

Conversation

Gaganjuneja
Copy link
Contributor

@Gaganjuneja Gaganjuneja commented Jul 23, 2023

Description

Issue -

Addresses the issue with code change #7543
For an instance where multiple tasks are being submitted from a loop the parent span is getting wrongly set.

Example :-
Parent - A

for(int i=0; i<array["B", "C", "D", "E"].length; i++){
       performPhaseOnShard(array[i]);
}

performPhaseOnShard executes the phase on the node where the shard is located, it could be locally or external data node. If we start a span from inside the performPhaseOnShard method, it creates the shard and sets this newly created span as a parent for the future spans. E.x here

performPhaseOnShard for shard B will have parent span as A.
performPhaseOnShard for shard C will have parent span as B. Which is wrong and should ideally be A
Similarly, performPhaseOnShard for shard C will have a parent span as C.

Ideally we need to reset the context in this scenario, so Tracer should provide a new Releasable TracerContext which can be reset to the original one of releasing the current context.

Related Issues

Resolves #8428

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Jul 23, 2023

Codecov Report

Merging #8831 (cbb38f6) into main (bb78930) will increase coverage by 0.09%.
Report is 14 commits behind head on main.
The diff coverage is 76.41%.

@@             Coverage Diff              @@
##               main    #8831      +/-   ##
============================================
+ Coverage     71.06%   71.15%   +0.09%     
- Complexity    57314    57359      +45     
============================================
  Files          4766     4767       +1     
  Lines        270453   270503      +50     
  Branches      39555    39555              
============================================
+ Hits         192197   192480     +283     
+ Misses        62045    61844     -201     
+ Partials      16211    16179      -32     
Files Changed Coverage Δ
.../opensearch/client/indices/CreateIndexRequest.java 74.44% <0.00%> (-2.23%) ⬇️
...opensearch/client/slm/SnapshotLifecyclePolicy.java 0.00% <ø> (ø)
...ch/client/slm/SnapshotLifecyclePolicyMetadata.java 0.00% <ø> (ø)
.../opensearch/client/slm/SnapshotLifecycleStats.java 0.00% <ø> (ø)
...rch/client/slm/SnapshotRetentionConfiguration.java 0.00% <ø> (ø)
...g/opensearch/core/common/util/CollectionUtils.java 77.06% <ø> (ø)
...h/script/mustache/MultiSearchTemplateResponse.java 59.01% <ø> (ø)
...rg/opensearch/index/rankeval/RankEvalResponse.java 98.57% <ø> (+8.57%) ⬆️
...va/org/opensearch/index/rankeval/RankEvalSpec.java 97.36% <ø> (ø)
...a/org/opensearch/index/rankeval/RatedDocument.java 81.03% <0.00%> (ø)
... and 140 more

... and 405 files with indirect coverage changes

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue
      1 org.opensearch.aliases.IndexAliasesIT.testSameAlias

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.cluster.allocation.ClusterRerouteIT.testDelayWithALargeAmountOfShards

@github-actions
Copy link
Contributor

github-actions bot commented Aug 7, 2023

Gradle Check (Jenkins) Run Completed with:

@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 28m 24s

Signed-off-by: Gagan Juneja <gagandeepjuneja@gmail.com>
@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 23m 25s

@github-actions
Copy link
Contributor

github-actions bot commented Aug 7, 2023

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 27m 29s

@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 26m 5s

@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.index.translog.RemoteFSTranslogTests.testConcurrentWriteViewsAndSnapshot

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

Gradle Check (Jenkins) Run Completed with:

@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:

Checking compatibility for: https://github.com/opensearch-project/neural-search.git with ref: main
Checking compatibility for: https://github.com/opensearch-project/security-analytics.git with ref: main
Checking compatibility for: https://github.com/opensearch-project/opensearch-oci-object-storage.git with ref: main
Incompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 23m 25s

@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

Gradle Check (Jenkins) Run Completed with:

@Gaganjuneja
Copy link
Contributor Author

Flakey shard allocation failure tests.

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

Gradle Check (Jenkins) Run Completed with:

@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 27m 56s

@reta reta merged commit 7655c32 into opensearch-project:main Aug 8, 2023
11 checks passed
@reta reta added the backport 2.x Backport to 2.x branch label Aug 8, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-8831-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7655c32a441facaf627c2d703f6b39489987e542
# Push it to GitHub
git push --set-upstream origin backport/backport-8831-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-8831-to-2.x.

@reta
Copy link
Collaborator

reta commented Aug 8, 2023

@Gaganjuneja please send a manual backport to 2.x, thank you

kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this pull request Sep 12, 2023
* Add support for tracer to create new TracerContextStorage

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Updated CHANGELOG

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Refactoring

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Updating the CHANGELOG

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Update Changelog

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Remove empty lines

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

---------

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Gagan Juneja <gagandeepjuneja@gmail.com>
Co-authored-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023
* Add support for tracer to create new TracerContextStorage

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Updated CHANGELOG

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Refactoring

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Updating the CHANGELOG

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Update Changelog

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Remove empty lines

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

---------

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Gagan Juneja <gagandeepjuneja@gmail.com>
Co-authored-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
* Add support for tracer to create new TracerContextStorage

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Updated CHANGELOG

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Refactoring

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Updating the CHANGELOG

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Adds feature to expose current span and start span with parent

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Update Changelog

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Remove empty lines

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

* Empty-Commit

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>

---------

Signed-off-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Gagan Juneja <gagandeepjuneja@gmail.com>
Co-authored-by: Gagan Juneja <gjjuneja@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Tracing Framework] Fix parent span setting in case multiple tasks are created from a single thread
3 participants