Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement union-loader #1907

Merged
merged 1 commit into from
Aug 8, 2023
Merged

Implement union-loader #1907

merged 1 commit into from
Aug 8, 2023

Conversation

acquamarin
Copy link
Collaborator

@acquamarin acquamarin commented Aug 8, 2023

This PR implements the very basic loader for union dataType.
DDL to define a union column: UNION(FIELD1 TYPE1, FIELD2 TYPE2)
Rules for union column in csv:
User simply needs to write the field value in the corresponding union column. The csv loader will set the value to the union field where a casting is possible. If the casting from the value to multiple struct fields are defined, the csv loader will choose the field with smallest struct_field_idx.
Currently, the csv reader can only cast primitive types to union (BOOL, INT64, INT32, INT16, DOUBLE, FLOAT, DATE, TIMESTAMP).

src/include/common/type_utils.h Outdated Show resolved Hide resolved
src/storage/copier/table_copy_utils.cpp Outdated Show resolved Hide resolved
src/storage/store/string_node_column.cpp Show resolved Hide resolved
src/include/common/types/timestamp_t.h Show resolved Hide resolved
src/include/common/type_utils.h Outdated Show resolved Hide resolved
@acquamarin acquamarin force-pushed the union-loader branch 2 times, most recently from 7d132ec to 75e011e Compare August 8, 2023 17:40
@codecov
Copy link

codecov bot commented Aug 8, 2023

Codecov Report

Patch coverage: 80.67% and project coverage change: +0.04% 🎉

Comparison is base (4a12985) 89.73% compared to head (f906657) 89.78%.
Report is 4 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1907      +/-   ##
==========================================
+ Coverage   89.73%   89.78%   +0.04%     
==========================================
  Files         869      869              
  Lines       31012    31103      +91     
==========================================
+ Hits        27828    27925      +97     
+ Misses       3184     3178       -6     
Files Changed Coverage Δ
src/include/common/types/date_t.h 100.00% <ø> (ø)
src/include/common/types/timestamp_t.h 100.00% <ø> (ø)
src/include/storage/copier/struct_column_chunk.h 100.00% <ø> (ø)
src/include/storage/copier/table_copy_utils.h 100.00% <ø> (ø)
...age/in_mem_storage_structure/in_mem_column_chunk.h 47.36% <0.00%> (ø)
...e/in_mem_storage_structure/in_mem_column_chunk.cpp 32.93% <0.00%> (ø)
src/storage/store/node_column.cpp 79.77% <ø> (+0.92%) ⬆️
src/storage/copier/struct_column_chunk.cpp 69.59% <42.85%> (+3.94%) ⬆️
src/storage/copier/table_copy_utils.cpp 70.46% <48.78%> (-2.12%) ⬇️
src/common/type_utils.cpp 62.31% <80.00%> (-5.94%) ⬇️
... and 24 more

... and 19 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@acquamarin acquamarin merged commit 25dfe2c into master Aug 8, 2023
10 checks passed
@acquamarin acquamarin deleted the union-loader branch August 8, 2023 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants