Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add loader and storage support for struct #1490

Merged
merged 1 commit into from
Apr 26, 2023
Merged

Conversation

acquamarin
Copy link
Collaborator

@acquamarin acquamarin commented Apr 25, 2023

This PR adds basic loader and storage support for struct dataType.
Unresolved struct dataType issues:

  1. Null struct value and null struct field value.
  2. Improves the loader for struct so that we can support more field types.
  3. Solves Match (m:movies) return m.description.rating exception bug.
  4. Query processing for unflat/flat struct field valuevectors.
  5. Implements lookup/write API for struct column.

@codecov
Copy link

codecov bot commented Apr 25, 2023

Codecov Report

Patch coverage: 88.74% and project coverage change: +0.13 🎉

Comparison is base (14e1277) 92.07% compared to head (14bf9c3) 92.20%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1490      +/-   ##
==========================================
+ Coverage   92.07%   92.20%   +0.13%     
==========================================
  Files         671      671              
  Lines       23894    24069     +175     
==========================================
+ Hits        22001    22194     +193     
+ Misses       1893     1875      -18     
Impacted Files Coverage Δ
src/include/common/ser_deser.h 100.00% <ø> (ø)
src/include/storage/copier/table_copy_executor.h 100.00% <ø> (ø)
...e/storage/in_mem_storage_structure/in_mem_column.h 95.65% <ø> (ø)
...ude/storage/storage_structure/disk_overflow_file.h 100.00% <ø> (ø)
src/include/storage/storage_utils.h 100.00% <ø> (ø)
src/processor/processor_task.cpp 100.00% <ø> (ø)
src/storage/copier/table_copy_executor.cpp 84.74% <52.94%> (-2.16%) ⬇️
...age/in_mem_storage_structure/in_mem_column_chunk.h 93.93% <60.00%> (-6.07%) ⬇️
...e/in_mem_storage_structure/in_mem_column_chunk.cpp 84.67% <80.51%> (-5.33%) ⬇️
src/storage/storage_structure/column.cpp 93.12% <96.66%> (+0.71%) ⬆️
... and 26 more

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ray6080 ray6080 requested review from ray6080 and removed request for andyfengHKU April 25, 2023 21:25
Copy link
Contributor

@ray6080 ray6080 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add more tests to cover more data types? See codecov report in InMemStructColumnChunk::copyValueToStructColumnField.

src/common/types/types.cpp Show resolved Hide resolved
src/common/types/types.cpp Outdated Show resolved Hide resolved
src/common/types/types.cpp Show resolved Hide resolved
src/common/types/types.cpp Outdated Show resolved Hide resolved
src/common/types/types.cpp Show resolved Hide resolved
template<>
void InMemColumnChunk::setValueFromString<bool>(
const char* value, uint64_t length, common::page_idx_t pageIdx, uint64_t posInPage) {
std::istringstream boolStream{std::string(value)};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. Can we not use istringstream here?

@acquamarin acquamarin merged commit 247cf1b into master Apr 26, 2023
@acquamarin acquamarin deleted the struct-storage branch April 26, 2023 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants