Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the misuse of star-tree when all predicates are always false under OR #12003

Merged

Conversation

Jackie-Jiang
Copy link
Contributor

Fix #11718

Fixes 2 scenarios where star-tree shouldn't be used:

  • When the filter is empty (no matching doc). There is a bug when all predicates under OR are always false, it is mis-identified as always true
  • When null handling is enabled for GROUP BY queries

@codecov-commenter
Copy link

codecov-commenter commented Nov 15, 2023

Codecov Report

Attention: 17 lines in your changes are missing coverage. Please review.

Comparison is base (62abf1f) 61.65% compared to head (3a37769) 61.62%.

Files Patch % Lines
...rg/apache/pinot/core/plan/AggregationPlanNode.java 26.31% 13 Missing and 1 partial ⚠️
...va/org/apache/pinot/core/plan/GroupByPlanNode.java 0.00% 1 Missing and 1 partial ⚠️
.../org/apache/pinot/core/startree/StarTreeUtils.java 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12003      +/-   ##
============================================
- Coverage     61.65%   61.62%   -0.04%     
  Complexity     1150     1150              
============================================
  Files          2385     2385              
  Lines        129250   129251       +1     
  Branches      20007    20009       +2     
============================================
- Hits          79690    79645      -45     
- Misses        43748    43809      +61     
+ Partials       5812     5797      -15     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (ø)
integration <0.01% <0.00%> (ø)
integration1 <0.01% <0.00%> (ø)
integration2 0.00% <0.00%> (ø)
java-11 35.01% <29.16%> (-26.59%) ⬇️
java-21 61.49% <29.16%> (-0.01%) ⬇️
skip-bytebuffers-false 61.60% <29.16%> (-0.04%) ⬇️
skip-bytebuffers-true 34.91% <29.16%> (-26.56%) ⬇️
temurin 61.62% <29.16%> (-0.04%) ⬇️
unittests 61.61% <29.16%> (-0.04%) ⬇️
unittests1 46.96% <29.16%> (+0.02%) ⬆️
unittests2 27.57% <0.00%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang force-pushed the fix_star_tree_predicate_is_false branch from f12cefd to 3a37769 Compare November 15, 2023 08:21
@@ -78,7 +78,8 @@ private GroupByOperator buildNonFilteredGroupByPlan() {

// Use star-tree to solve the query if possible
List<StarTreeV2> starTrees = _indexSegment.getStarTrees();
if (starTrees != null && !_queryContext.isSkipStarTree()) {
if (!_queryContext.isNullHandlingEnabled() && !filterOperator.isResultEmpty() && starTrees != null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we were missing null handling enable flag before as well here? :-P

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately yes..

@Jackie-Jiang Jackie-Jiang merged commit 6d1b060 into apache:master Nov 15, 2023
19 checks passed
@Jackie-Jiang Jackie-Jiang deleted the fix_star_tree_predicate_is_false branch November 15, 2023 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Flaky-test] StarTreeClusterIntegrationTest.testGeneratedQueries
3 participants