Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Boolean assertion transform functions. #11547

Merged
merged 8 commits into from
Sep 10, 2023
Merged

Conversation

shenyu0127
Copy link
Contributor

This PR adds the following transform functions:

  • IS TRUE
  • IS NOT TRUE
  • IS FALSE
  • IS NOT FALSE

@codecov-commenter
Copy link

codecov-commenter commented Sep 8, 2023

Codecov Report

Merging #11547 (0e54d37) into master (6208e7c) will increase coverage by 0.02%.
Report is 1 commits behind head on master.
The diff coverage is 91.07%.

@@             Coverage Diff              @@
##             master   #11547      +/-   ##
============================================
+ Coverage     63.05%   63.08%   +0.02%     
  Complexity     1109     1109              
============================================
  Files          2320     2325       +5     
  Lines        124667   124723      +56     
  Branches      19033    19041       +8     
============================================
+ Hits          78614    78682      +68     
+ Misses        40458    40450       -8     
+ Partials       5595     5591       -4     
Flag Coverage Δ
integration <0.01% <0.00%> (ø)
integration1 <0.01% <0.00%> (ø)
integration2 0.00% <0.00%> (ø)
java-11 63.03% <91.07%> (+0.02%) ⬆️
java-17 62.93% <91.07%> (+0.02%) ⬆️
java-20 62.91% <91.07%> (+0.01%) ⬆️
temurin 63.08% <91.07%> (+0.02%) ⬆️
unittests 63.08% <91.07%> (+0.02%) ⬆️
unittests1 67.51% <91.07%> (+0.03%) ⬆️
unittests2 14.49% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...unction/BaseBooleanAssertionTransformFunction.java 79.16% <79.16%> (ø)
...e/pinot/common/function/TransformFunctionType.java 89.70% <100.00%> (+0.20%) ⬆️
...r/transform/function/IsFalseTransformFunction.java 100.00% <100.00%> (ø)
...ransform/function/IsNotFalseTransformFunction.java 100.00% <100.00%> (ø)
...transform/function/IsNotTrueTransformFunction.java 100.00% <100.00%> (ø)
...or/transform/function/IsTrueTransformFunction.java 100.00% <100.00%> (ø)
...r/transform/function/TransformFunctionFactory.java 89.53% <100.00%> (+0.24%) ⬆️

... and 8 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

import org.apache.pinot.core.operator.ColumnContext;


public class IsFalseTransformFunction extends IsTrueTransformFunction {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is a good idea to extend IsTrueTransformFunction in order to completely change the semantic.

Instead we should have some kind of AbstractIsBooleanTransformFunction that IsTrueTransformFunction, IsFalseTransformFunction, IsNotFalseTransformFunction, etc extend

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+100 to the abstract class. Four implementations should just implement the Boolean operations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@Override
public void init(List<TransformFunction> arguments, Map<String, ColumnContext> columnContextMap) {
Preconditions.checkArgument(arguments.size() == 1, "Exact 1 argument is required for IS_NULL");
_transformFunction = arguments.get(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more a question to understand our code than something to change in this PR. I guess all these functions require to have an argument whose type is boolean. Where do we enforce that type constraint?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In V1 transform functions we use integer to represent boolean and use BaseTransformFunction to do casting. So we don't need to enforce the type constraint.

if (returnsTrueWhenValueIsNull()) {
_intValuesSV[docId] = 1;
}
} else if (intValuesSV[docId] != 0) {
Copy link
Contributor

@xiangfu0 xiangfu0 Sep 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we can combine returnsTrueWhenValueIsTrue and returnsTrueWhenValueIsFalse into one method like

if (evalBooleanAssertionFunc()) {
  _intValuesSV[docId] = 1;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Good suggestion!

@xiangfu0 xiangfu0 merged commit 1ce2bf7 into apache:master Sep 10, 2023
21 checks passed
@shenyu0127 shenyu0127 deleted the is-true branch September 10, 2023 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants