Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'WEEK' transform function in multi-stage query engine #13483

Merged
merged 1 commit into from
Jun 26, 2024

Conversation

yashmayya
Copy link
Collaborator

  • In the v2 multi-stage query engine, numerous date time functions are rewritten to a call to EXTRACT by Calcite.
  • Of all the functions that get re-written to an EXTRACT call, the only one that is not supported by Pinot's extract transformation function is WEEK. This leads to simple queries like SELECT WEEK(ts) FROM airlineStats failing with:
Received error query execution result block: {200=QueryExecutionError:
org.apache.pinot.spi.exception.BadQueryRequestException: Caught exception while initializing transform function: extract
	at org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:332)
	at org.apache.pinot.core.operator.transform.TransformOperator.<init>(TransformOperator.java:56)
	at org.apache.pinot.core.plan.ProjectPlanNode.run(ProjectPlanNode.java:85)
	at org.apache.pinot.core.plan.StreamingSelectionPlanNode.run(StreamingSelectionPlanNode.java:54)
...
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.pinot.core.operator.transform.function.ExtractTransformFunction.Field.WEEK
	at java.base/java.lang.Enum.valueOf(Enum.java:273)
	at org.apache.pinot.core.operator.transform.function.ExtractTransformFunction$Field.valueOf(ExtractTransformFunction.java:38)
	at org.apache.pinot.core.operator.transform.function.ExtractTransformFunction.init(ExtractTransformFunction.java:54)
	at org.apache.pinot.core.operator.transform.function.BaseTransformFunction.init(BaseTransformFunction.java:120)}
org.apache.pinot.query.service.dispatch.QueryDispatcher.runReducer(QueryDispatcher.java:306)
org.apache.pinot.query.service.dispatch.QueryDispatcher.submitAndReduce(QueryDispatcher.java:96)
org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:217)
org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:133)

@yashmayya yashmayya added bugfix multi-stage Related to the multi-stage query engine labels Jun 26, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jun 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.00%. Comparing base (59551e4) to head (6ddb781).
Report is 679 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13483      +/-   ##
============================================
+ Coverage     61.75%   62.00%   +0.25%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2560     +124     
  Lines        133233   141361    +8128     
  Branches      20636    21929    +1293     
============================================
+ Hits          82274    87654    +5380     
- Misses        44911    47051    +2140     
- Partials       6048     6656     +608     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 35.37% <100.00%> (-26.34%) ⬇️
java-21 61.88% <100.00%> (+0.25%) ⬆️
skip-bytebuffers-false 61.98% <100.00%> (+0.23%) ⬆️
skip-bytebuffers-true 61.87% <100.00%> (+34.14%) ⬆️
temurin 62.00% <100.00%> (+0.25%) ⬆️
unittests 62.00% <100.00%> (+0.25%) ⬆️
unittests1 46.65% <100.00%> (-0.24%) ⬇️
unittests2 27.53% <0.00%> (-0.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit 465a290 into apache:master Jun 26, 2024
19 of 20 checks passed
suyashpatel98 pushed a commit to suyashpatel98/pinot that referenced this pull request Jul 6, 2024
@npawar npawar added the v1v2 label Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix multi-stage Related to the multi-stage query engine v1v2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants