[multistage][bugfix] eliminate multiple exchanges

[walterddr]

this fixes #11881

this rule will be merged into a more sophisticated rule that will be created in #11831

[codecov-commenter]

Codecov Report

Merging #11882 (e224211) into master (6853991) will decrease coverage by 0.03%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #11882      +/-   ##
============================================
- Coverage     61.42%   61.40%   -0.03%     
+ Complexity     1147     1146       -1     
============================================
  Files          2375     2375              
  Lines        128501   128501              
  Branches      19846    19846              
============================================
- Hits          78936    78907      -29     
- Misses        43859    43887      +28     
- Partials       5706     5707       +1     

Flag	Coverage Δ
custom-integration1	<0.01% <ø> (ø)
integration	<0.01% <ø> (ø)
integration1	<0.01% <ø> (ø)
integration2	0.00% <ø> (ø)
java-11	34.80% <ø> (-26.59%)	⬇️
java-21	61.28% <ø> (+0.01%)	⬆️
skip-bytebuffers-false	34.84% <ø> (-26.58%)	⬇️
skip-bytebuffers-true	61.26% <ø> (+<0.01%)	⬆️
temurin	61.40% <ø> (-0.03%)	⬇️
unittests	61.40% <ø> (-0.03%)	⬇️
unittests1	46.63% <ø> (ø)
unittests2	27.56% <ø> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 11 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[xiangfu0]

This is interesting for two filters, I think visitFilter in ServerPlanRequestVisitor will override each other, so one predicate will be skipped.

[walterddr]

this is not unexpected. (we can potentially make it more smart). the problem is that

WITH tmp1
     AS (SELECT *
         FROM   a
         WHERE  col2 NOT IN ( 'foo', 'bar' )),
     tmp2
     AS (SELECT *
         FROM   b
         WHERE  col1 IN (SELECT col1
                         FROM   tmp1)
                AND col3 < 100)
SELECT *
FROM   tmp2
WHERE  col1 IN (SELECT col1
                FROM   tmp1  -- this is the tmp1 modification into tmp1'
                WHERE  col3 > 10) 

at this stage

• Calcite's doesn't know if you want to do a table spool of tmp1 and make tmp1' derived from tmp1 --> thus col3 > 10 is not pushed down.
• only when we decided that table spool is not possible and then we decided to copy tmp1 and tmp1' into 2 separate sub queries, at that time i think the hep optimizer already get passed the filter merging phase.

rewriting the query in this way

WITH tmp1
     AS (SELECT *
         FROM   a
         WHERE  col2 NOT IN ( 'foo', 'bar' )),
     tmp2
     AS (SELECT *
         FROM   b
         WHERE  col1 IN (SELECT col1
                         FROM   tmp1)
                AND col3 < 100),
     tmp3 -- here we explicitly tell that tmp3 and tmp1 are not related
     AS (SELECT *
         FROM   a
         WHERE  col2 NOT IN ( 'foo', 'bar' ) AND col3 > 10),
SELECT *
FROM   tmp2
WHERE  col1 IN (SELECT col1
                FROM   tmp3) 

produces no multi-filter plan

[xiangfu0]

If we cannot avoid this, then it means the ServerPlanRequestVisitor should handle the leaf query generation properly.

[walterddr]

this is a un-plannable sql before or after this bug fix PR. I would suggest differ the fix into a different PR (i will change the test to make sure it doesn't happen)