Add support for IS NULL and NOT IS NULL in transform functions

[KKcorps]

This PR addresses the issue #8251 . Currently, Pinot supports IS NULL syntax in filter expressions such as WHERE clause. But it doesn't support as part of SELECT expressions such as CASE WHEN col IS NULL THEN a ELSE b

The PR adds support for such expressions by utilising the NullValueVectorReader to get the docIds which are null.

The implementation has two requirements/limitations -

• nullHandlingEnabled should be set to true since native NULL values are not supported in Pinot and requires a null value bitmap index.

• Only column names can be provided as the first argument for expressions. The support for functions is still not there.
  e.g. colA IS NULL is supported but LOWER(colA) IS NULL is not supported.

Currently Pending -

• Documentation

[codecov-commenter]

Codecov Report

Merging #8264 (f0e7618) into master (25cd6e3) will decrease coverage by 5.72%.
The diff coverage is 64.70%.

@@             Coverage Diff              @@
##             master    #8264      +/-   ##
============================================
- Coverage     69.72%   64.00%   -5.73%     
- Complexity     4264     4272       +8     
============================================
  Files          1639     1600      -39     
  Lines         85920    84243    -1677     
  Branches      12922    12737     -185     
============================================
- Hits          59906    53918    -5988     
- Misses        21842    26431    +4589     
+ Partials       4172     3894     -278     

Flag	Coverage Δ
integration1	?
unittests1	66.92% <64.70%> (-0.03%)	⬇️
unittests2	14.18% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...r/transform/function/TransformFunctionFactory.java	80.90% <60.00%> (-2.73%)	⬇️
...transform/function/IsNotNullTransformFunction.java	63.33% <63.33%> (ø)
...or/transform/function/IsNullTransformFunction.java	63.33% <63.33%> (ø)
...e/pinot/common/function/TransformFunctionType.java	100.00% <100.00%> (ø)
...ransform/function/IdentifierTransformFunction.java	72.72% <100.00%> (+0.63%)	⬆️
...va/org/apache/pinot/common/config/NettyConfig.java	0.00% <0.00%> (-100.00%)	⬇️
...a/org/apache/pinot/common/metrics/MinionMeter.java	0.00% <0.00%> (-100.00%)	⬇️
...g/apache/pinot/common/metrics/ControllerMeter.java	0.00% <0.00%> (-100.00%)	⬇️
.../apache/pinot/common/metrics/BrokerQueryPhase.java	0.00% <0.00%> (-100.00%)	⬇️
.../apache/pinot/common/metrics/MinionQueryPhase.java	0.00% <0.00%> (-100.00%)	⬇️
... and 353 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 25cd6e3...f0e7618. Read the comment docs.

[richardstartin]

👍🏻

[Jackie-Jiang]

Suggest also add some tests in NullHandlingIntegrationTest

[KKcorps]

One more side note - If these functions are used with a table where Null handling is disabled, they assume everything is non-null. So IS NULL will always return false and IS NOT NULL will always return true.

Instead, If you want it to throw exception in this case, I can do that as well.

[Jackie-Jiang]

One more side note - If these functions are used with a table where Null handling is disabled, they assume everything is non-null. So IS NULL will always return false and IS NOT NULL will always return true.

This is the correct behavior which aligns with the filter semantic