-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[multistage] Initial (phase 1) Query runtime for window functions with ORDER BY within the OVER() clause #10449
[multistage] Initial (phase 1) Query runtime for window functions with ORDER BY within the OVER() clause #10449
Conversation
Codecov Report
@@ Coverage Diff @@
## master #10449 +/- ##
=============================================
- Coverage 63.23% 24.16% -39.08%
+ Complexity 5905 58 -5847
=============================================
Files 2036 2065 +29
Lines 110973 111457 +484
Branches 16892 16990 +98
=============================================
- Hits 70177 26930 -43247
- Misses 35606 81757 +46151
+ Partials 5190 2770 -2420
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1347 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
...y-runtime/src/main/java/org/apache/pinot/query/runtime/operator/WindowAggregateOperator.java
Outdated
Show resolved
Hide resolved
...y-runtime/src/main/java/org/apache/pinot/query/runtime/operator/WindowAggregateOperator.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good mostly.
this PR assumes the mailbox receive will generate ordered block appropriately, correct? if so could we please state so in the javadoc of window aggregate operator?
Thanks for the review! Yes, this PR assumes that mailbox receive will generate ordered blocks. Added a note in the javadocs about this. I've also addressed all other comments. PTAL when you get a chance. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
This PR introduces the multi-stage runtime changes to support Phase 1 of Window Functions, specifically targeting queries of the type
OVER(ORDER BY)
andOVER(PARTITION BY key1 ORDER BY key2)
. Runtime support has been added for the following classes of window queries:OVER(ORDER BY)
- single and multipleOVER(ORDER BY)
in the same queryOVER(PARTITION BY key1 ORDER BY key2)
- single and multipleOVER(PARTITION BY key1 ORDER BY key2)
using the samePARTITION BY
key1 andORDER BY
key2 across allOVER()s
in the same queryThe window functions supported as part of Phase 1 are: SUM, AVG, MIN, MAX, COUNT, BOOL_OR, and BOOL_AND.
OVER()
andOVER(PARTITION BY)
: [multistage] Initial (phase 1) Query runtime for window functions - empty OVER() and OVER(PARTITION BY) #10286This PR does not include support for:
The above will be part of future changes for window function support
Note: One issue was found while working on this PR related to Apache Calcite parsing. Details can be found in this section of the design document. Look for heading "Calcite Parser Exception on ORDER BY with > 1 column using default frame"
cc @siddharthteotia @walterddr @vvivekiyer @ankitsultana