Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "SQL: Plan non-equijoin conditions as cross join followed by f… #15029

Merged
merged 1 commit into from
Sep 25, 2023

Conversation

somu-imply
Copy link
Contributor

@somu-imply somu-imply commented Sep 22, 2023

…ilter. (#14978)"

This reverts commit 4f498e6.

The commit was creating an issue with the rule CoreRules.JOIN_EXTRACT_FILTER, The Introduction of this rule caused cases with non-equijoin to get into a situation where Calcite was not able to decipher rule dependencies correctly as the outputs might be slightly differing from one another. This would cause planning to throw an OOM at the broker. We are reverting this for now to unblock the master. An example case is

EXPLAIN PLAN FOR
select t1.s_int, t1.c 
from 
  (select * from test_unnest_test_sql_array CROSS JOIN unnest(a_int) as u(c)) t1 
inner join 
  (select * from test_unnest_test_sql_array CROSS JOIN unnest(a_int) as u(c)) t2
on t1.c=cast(t2.s_int as char) 
where t1.s_int in (0, 1, 2, 999) and t2.s_int in (0, 1, 2, 999)

Will add unit test cases to catch issues after adding the Join_extract_filter_rule in a followup

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@github-actions github-actions bot added Area - Documentation Area - Batch Ingestion Area - Querying Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Sep 22, 2023
@soumyava soumyava added the WIP label Sep 22, 2023
.build()
),
"j0.",
equalsCondition(makeColumnExpression("dim1"), makeColumnExpression("j0.d0")),

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note test

Invoking
CalciteTestBase.makeColumnExpression
should be avoided because it has been deprecated.
.build()
),
"j0.",
equalsCondition(makeColumnExpression("dim1"), makeColumnExpression("j0.d0")),

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note test

Invoking
CalciteTestBase.makeColumnExpression
should be avoided because it has been deprecated.
@somu-imply somu-imply force-pushed the revert_join_extract_filter_rule branch from f0da120 to bfca67c Compare September 22, 2023 20:58
@somu-imply somu-imply marked this pull request as ready for review September 22, 2023 22:00
@soumyava soumyava removed Area - Batch Ingestion WIP Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Sep 22, 2023
@soumyava soumyava merged commit 75af741 into apache:master Sep 25, 2023
74 checks passed
@LakshSingla LakshSingla added this to the 28.0 milestone Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants