Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the alias handling in single-stage engine #11610

Merged
merged 2 commits into from
Sep 19, 2023

Conversation

Jackie-Jiang
Copy link
Contributor

@Jackie-Jiang Jackie-Jiang commented Sep 18, 2023

Fix 3 issues:

Incompatible

Alias can no longer be applied to filter to comply with standard SQL

@codecov-commenter
Copy link

codecov-commenter commented Sep 18, 2023

Codecov Report

Merging #11610 (72397d7) into master (d9c0790) will decrease coverage by 52.79%.
The diff coverage is 23.07%.

@@              Coverage Diff              @@
##             master   #11610       +/-   ##
=============================================
- Coverage     67.28%   14.49%   -52.79%     
+ Complexity      905      201      -704     
=============================================
  Files          1731     2325      +594     
  Lines         90155   124913    +34758     
  Branches      14440    19146     +4706     
=============================================
- Hits          60661    18109    -42552     
- Misses        25406   105276    +79870     
+ Partials       4088     1528     -2560     
Flag Coverage Δ
integration 0.00% <0.00%> (?)
integration2 0.00% <0.00%> (?)
java-11 14.48% <23.07%> (-52.79%) ⬇️
java-17 14.49% <23.07%> (-52.60%) ⬇️
java-20 14.48% <23.07%> (-52.61%) ⬇️
temurin 14.49% <23.07%> (-52.79%) ⬇️
unittests 14.49% <23.07%> (-52.79%) ⬇️
unittests1 ?
unittests2 14.49% <23.07%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...requesthandler/MultiStageBrokerRequestHandler.java 19.25% <0.00%> (ø)
...pache/pinot/sql/parsers/rewriter/AliasApplier.java 0.00% <0.00%> (-96.30%) ⬇️
...not/sql/parsers/rewriter/QueryRewriterFactory.java 0.00% <0.00%> (-100.00%) ⬇️
.../core/realtime/PinotLLCRealtimeSegmentManager.java 60.57% <0.00%> (ø)
...roker/requesthandler/BaseBrokerRequestHandler.java 45.99% <75.00%> (ø)

... and 2109 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@Jackie-Jiang Jackie-Jiang added the incompatible Indicate PR that introduces backward incompatibility label Sep 18, 2023
Copy link
Contributor

@xiangfu0 xiangfu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall lgtm

aliasMap.put(identifierExpr.getIdentifier(), functionCall.getOperands().get(0));
String alias = function.getOperands().get(1).getIdentifier().getName();
if (aliasMap.put(alias, function.getOperands().get(0)) != null) {
throw new SqlCompilationException("Find duplicate alias: " + alias);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

public static final List<String> DEFAULT_QUERY_REWRITERS_CLASS_NAMES =
ImmutableList.of(CompileTimeFunctionsInvoker.class.getName(), SelectionsRewriter.class.getName(),
PredicateComparisonRewriter.class.getName(), OrdinalsUpdater.class.getName(),
AliasApplier.class.getName(), NonAggregationGroupByToDistinctQueryRewriter.class.getName());
PredicateComparisonRewriter.class.getName(), AliasApplier.class.getName(), OrdinalsUpdater.class.getName(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

}
}
return aliasMap;
}

private static void applyAlias(Map<Identifier, Expression> aliasMap, PinotQuery pinotQuery) {
Expression filterExpression = pinotQuery.getFilterExpression();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern here is that this will silently return empty results once users upgrade.

Shall we instead throw exception if detected the alias in filter and explicitly mention that alias shouldn't be used, so users can at least find out the root cause and either roll back or fix queries before upgrade?

@xiangfu0 xiangfu0 force-pushed the fix_alias branch 4 times, most recently from e8314ee to ac7e28f Compare September 19, 2023 02:37
@wirybeaver
Copy link
Contributor

Just for curiosity, what regression do you observe when the #10815 is introduced?

@xiangfu0
Copy link
Contributor

Just for curiosity, what regression do you observe when the #10815 is introduced?

master branch build failure:
https://github.com/apache/pinot/actions/runs/6230277100/job/16910046883

@xiangfu0 xiangfu0 merged commit e123642 into apache:master Sep 19, 2023
20 of 21 checks passed
@Jackie-Jiang Jackie-Jiang deleted the fix_alias branch September 19, 2023 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix incompatible Indicate PR that introduces backward incompatibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Aliasing a column with a function to the same name causes a StackOverflowException
4 participants