Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

List Auxiliary Buffer NullMask Fix #3156

Merged
merged 2 commits into from
Mar 27, 2024
Merged

List Auxiliary Buffer NullMask Fix #3156

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9f8d811
initial commit
mxwli Mar 27, 2024
8dd8c1c
apply clang format fix
mxwli Mar 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/common/vector/auxiliary_buffer.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ void ListAuxiliaryBuffer::resizeDataVector(ValueVector* dataVector) {
auto buffer = std::make_unique<uint8_t[]>(capacity * dataVector->getNumBytesPerValue());
memcpy(buffer.get(), dataVector->valueBuffer.get(), size * dataVector->getNumBytesPerValue());
dataVector->valueBuffer = std::move(buffer);
dataVector->nullMask->resize(capacity); // note: allocating 64 times what is needed
dataVector->nullMask->resize(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u check if we are making the same mistake in other places when resizing nullMask? If so, this logic should be hidden inside NullMask class it self.

Copy link
Collaborator Author

@mxwli mxwli Mar 27, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran "Find All References" in VSCode and the only result that came up was this instance. The function is probably only used here.
More generally, the main problem is that NullMask refers to capacity in the int64 sense, rather than in the bitmap sense. For example, the capacity passed to the constructor specifies its size in int64 chunks, so it would make more sense to change everything to bitmap.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think int64 chunks or vector matters on this problem. We should encapsulate the details inside NullMask, so the best way is to modify resize function accordingly.

(capacity + NullMask::NUM_BITS_PER_NULL_ENTRY - 1)
>> NullMask::NUM_BITS_PER_NULL_ENTRY_LOG2);
// If the dataVector is a struct vector, we need to resize its field vectors.
if (dataVector->dataType.getPhysicalType() == PhysicalTypeID::STRUCT) {
resizeStructDataVector(dataVector);
Expand Down