-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BoundFilter optimizations, and related interface changes. #2727
Conversation
benchmarks:
|
8207545
to
a7e4b1b
Compare
@gianm should we change default to use roaring as it seems to be much better in most cases |
} | ||
} | ||
); | ||
} | ||
|
||
private boolean doesMatch(String input) | ||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this logic just copypasta of what was there before?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, no, it is not
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mostly yes, except for the null handling.
👍 |
a7e4b1b
to
77fed15
Compare
Some of the new tests depend on #2737 to work. |
BoundFilter: - For lexicographic bounds, use bitmapIndex.getIndex to find the start and end points, then union all bitmaps between those points. - For alphanumeric bounds, iterate through dimValues, and union all bitmaps for values matching the predicate. - Change behavior for nulls: it used to be that the BoundFilter would never match nulls, now it matches nulls if "" is allowed by the lower limit and not excluded by the upper limit. Interface changes: - BitmapIndex: add `int getIndex(value)` to make it possible to get the index for a value without retrieving the bitmap. - BitmapIndex: remove `ImmutableBitmap getBitmap(value)`, change callers to `getBitmap(getIndex(value))`. - BitmapIndexSelector: allow retrieving the underlying BitmapIndex through getBitmapIndex. - Clarified contract of indexOf in Indexed, GenericIndexed. Also added tests for SelectorFilter, NotFilter, and BoundFilter.
77fed15
to
2970b49
Compare
@@ -111,6 +112,17 @@ public BitmapFactory getBitmapFactory() | |||
} | |||
|
|||
@Override | |||
public BitmapIndex getBitmapIndex(String dimension) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should public BitmapIndex getBitmapIndex(String dimension)
and public ImmutableBitmap getBitmapIndex(String dimension, String value)
be named differently? Maybe slightly confusing to have functions with a different return type with the same name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I didn't want to touch too many files in this patch (all of the other filters), but I also didn't want to use a different name for this method, since it's the one that really should be called getBitmapIndex
.
I am indifferent though & happy to go with whatever people think is best
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, I'm fine with this PR as-is, that rename if desired could be handled in later patch
👍 after travis |
BoundFilter:
then union all bitmaps between those points.
matching the predicate.
now it matches nulls if "" is allowed by the lower limit and not excluded by the
upper limit.
Interface changes:
int getIndex(value)
to make it possible to get the index for avalue without retrieving the bitmap.
ImmutableBitmap getBitmap(value)
, change callers togetBitmap(getIndex(value))
.Also added tests for SelectorFilter, NotFilter, and BoundFilter.