-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove DataTypeInfo and use LogicalType and column names #2539
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2539 +/- ##
==========================================
+ Coverage 92.28% 93.31% +1.03%
==========================================
Files 1161 1036 -125
Lines 44150 38819 -5331
==========================================
- Hits 40744 36225 -4519
+ Misses 3406 2594 -812 ☔ View full report in Codecov by Sentry. |
src/common/arrow/arrow_row_batch.cpp
Outdated
ArrowVector* vector, const main::DataTypeInfo& typeInfo, std::int64_t capacity) { | ||
initializeStructVector(vector, typeInfo, capacity); | ||
ArrowVector* vector, const LogicalType& type, std::int64_t capacity) { | ||
initializeStructVector(vector, type, capacity); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add test to cover all uncovered lines.
|
||
ArrowArray toArray(); | ||
|
||
private: | ||
std::vector<std::unique_ptr<main::DataTypeInfo>> typesInfo; | ||
std::vector<LogicalType> types; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use common::logical_types_t
9cd44eb
to
5915cb6
Compare
After attempting to add more Arrow tests, I've discovered a problem with this removal of DataTypeInfo that I do not know how to resolve. Details: I tried to return a node, rel, or internal ID in the Arrow tests, and this led to segmentation faults. After further investigation, I believe this is because the following logic in the old implementation using DataTypeInfo is not preserved: I tried to modify arrow_row_batch.cpp (including templateInitializeVector and templateCopyNonNullValue for internal IDs, nodes, and rels) to replicate this logic, but I didn't really know what I was doing and I ended up with a segmentation fault occurring during std::memcpy that I didn't know what to do about. My approach was probably incorrect and messy I won't push the code here. |
5915cb6
to
84700b3
Compare
84700b3
to
46dbb69
Compare
Should close #3224 as well |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make sure we add corresponding test cases for coverage.
fixes #2423