Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix multiple consecutive Exchange returning empty response #11885

Merged
merged 1 commit into from
Oct 27, 2023

Conversation

Jackie-Jiang
Copy link
Contributor

@Jackie-Jiang Jackie-Jiang commented Oct 26, 2023

Fix #11880

@Jackie-Jiang Jackie-Jiang added bugfix multi-stage Related to the multi-stage query engine labels Oct 26, 2023
@codecov-commenter
Copy link

codecov-commenter commented Oct 26, 2023

Codecov Report

Merging #11885 (1178bc7) into master (5d58102) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master   #11885      +/-   ##
============================================
- Coverage     61.45%   61.40%   -0.05%     
  Complexity     1147     1147              
============================================
  Files          2375     2375              
  Lines        128500   128501       +1     
  Branches      19846    19846              
============================================
- Hits          78974    78912      -62     
- Misses        43815    43892      +77     
+ Partials       5711     5697      -14     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (ø)
integration <0.01% <0.00%> (ø)
integration1 <0.01% <0.00%> (ø)
integration2 0.00% <0.00%> (ø)
java-11 61.37% <100.00%> (-0.03%) ⬇️
java-21 61.28% <100.00%> (-0.03%) ⬇️
skip-bytebuffers-false 61.40% <100.00%> (-0.02%) ⬇️
skip-bytebuffers-true 61.26% <100.00%> (-0.03%) ⬇️
temurin 61.40% <100.00%> (-0.05%) ⬇️
unittests 61.40% <100.00%> (-0.05%) ⬇️
unittests1 46.63% <100.00%> (-0.02%) ⬇️
unittests2 27.57% <0.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...he/pinot/query/planner/logical/PlanFragmenter.java 89.36% <100.00%> (+0.23%) ⬆️

... and 13 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@Jackie-Jiang Jackie-Jiang merged commit 6853991 into apache:master Oct 27, 2023
19 checks passed
@Jackie-Jiang Jackie-Jiang deleted the fix_multiple_exchanges branch October 27, 2023 15:54
Comment on lines +131 to +132
// Set previous PlanFragment ID in the context to be the next PlanFragment ID to be used by the child node.
context._previousPlanFragmentId = nextPlanFragmentId;
Copy link
Contributor

@walterddr walterddr Oct 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the logic of previous/current plan fragment id looks very convoluted

in normal node, _previousPlanFragmentId is set to the "node itself before it visit its children, thus guaranteed at the time child is being visit, current is always what the child is suppose to be on, and the previous is what the parent suppose to be on.

in exchange _previousPlanFragmentId setting to the next plan fragment id seems weird.

thus "previousPlanFragmentId" really should be named "plan fragment id for the parent PlanNode": the actual problem is

  • once exchange visits the child, the plan break into fragments
  • b/c mailbox send doesn't have a parent, the "plan fragment id for the parent" definition is moot, and thus will just use the current plan fragment ID :-/

^which still seems weird, i wonder keeping this as a global context might not be the best way

Copy link
Contributor

@walterddr walterddr Oct 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be more clear:

Suggested change
// Set previous PlanFragment ID in the context to be the next PlanFragment ID to be used by the child node.
context._previousPlanFragmentId = nextPlanFragmentId;
// when visiting the child of the exchange, the parent is no longer the exchange node, but the mailboxSendNode
// created by this exchange, thus the "previousPlanFragmentId" when visiting the child should be nextPlanFragmentId
context._previousPlanFragmentId = nextPlanFragmentId;

please let me know if my understanding is correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix multi-stage Related to the multi-stage query engine
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[multistage] mailbox receiving stage not correctly created
4 participants