Pandas Pyarrow Backend Bugfix and Tests #3152
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3152 +/- ##
==========================================
+ Coverage 91.91% 92.08% +0.16%
==========================================
Files 1169 1168 -1
Lines 43736 44065 +329
==========================================
+ Hits 40202 40576 +374
+ Misses 3534 3489 -45 ☔ View full report in Codecov by Sentry. |
| @@ -52,7 +52,7 @@ void ListAuxiliaryBuffer::resizeDataVector(ValueVector* dataVector) { | |||
| auto buffer = std::make_unique<uint8_t[]>(capacity * dataVector->getNumBytesPerValue()); | |||
| memcpy(buffer.get(), dataVector->valueBuffer.get(), size * dataVector->getNumBytesPerValue()); | |||
| dataVector->valueBuffer = std::move(buffer); | |||
| dataVector->nullMask->resize(capacity); | |||
| dataVector->nullMask->resize(capacity); // note: allocating 64 times what is needed | |||
There was a problem hiding this comment.
i am a little bit confused about the comment? What do you mean by 64times?
There was a problem hiding this comment.
capacity refers to the number of values inside our vector. However, to nullMask, capacity refers to the number of uint64_ts it should allocate to the buffer. NullMask is a bitmap, so directly resizing to capacity will allocate 64 times the number of bits necessary.
Update: I applied the change locally and ran tests. They passed.
Changes:
MAP scanning will be added in the next PR.