Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix COPY; rework Reader op #1963

Merged
merged 1 commit into from
Aug 27, 2023
Merged

Fix COPY; rework Reader op #1963

merged 1 commit into from
Aug 27, 2023

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Aug 26, 2023

No description provided.

@codecov
Copy link

codecov bot commented Aug 26, 2023

Codecov Report

❗ No coverage uploaded for pull request base (master@9526e5e). Click here to learn what that means.
Patch coverage: 96.02% of modified lines in pull request are covered.

❗ Current head f253c4c differs from pull request most recent head 7be20dc. Consider uploading reports for the commit 7be20dc to get more accurate results

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #1963   +/-   ##
=========================================
  Coverage          ?   89.74%           
=========================================
  Files             ?      868           
  Lines             ?    32027           
  Branches          ?        0           
=========================================
  Hits              ?    28743           
  Misses            ?     3284           
  Partials          ?        0           
Files Changed Coverage Δ
src/include/common/exception.h 80.95% <ø> (ø)
src/include/processor/operator/persistent/copy.h 42.85% <ø> (ø)
...c/include/processor/operator/persistent/copy_rel.h 57.14% <ø> (ø)
...rc/include/processor/operator/persistent/copy_to.h 100.00% <ø> (ø)
...de/processor/operator/persistent/csv_file_writer.h 100.00% <ø> (ø)
src/include/processor/operator/physical_operator.h 100.00% <ø> (ø)
src/include/storage/store/node_column.h 100.00% <ø> (ø)
src/processor/operator/persistent/copy.cpp 100.00% <ø> (ø)
src/processor/operator/persistent/copy_rel.cpp 100.00% <ø> (ø)
src/processor/operator/persistent/copy_to.cpp 100.00% <ø> (ø)
... and 25 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ColumnChunkMetadata() = default;
ColumnChunkMetadata(common::page_idx_t pageIdx, common::page_idx_t numPages)
struct BaseColumnChunkMetadata {
BaseColumnChunkMetadata() : pageIdx{common::INVALID_PAGE_IDX}, numPages{0} {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we always put constructor and deconstructor below the fields for structs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm ok. Will make the change.
Was this style coming from some standard? I wasn't aware of that. why do we have to make struct so different from class?

src/include/storage/copier/rel_copier.h Outdated Show resolved Hide resolved
return false;
}
for (auto i = 0u; i < readerInfo.dataColumnPoses.size(); i++) {
ArrowColumnVector::setArrowColumn(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better to store the readerVectors in the Reader operators, so you don't need to dynamically call getValueVector() during reading.

resultSet->getValueVector(readerInfo.dataColumnPoses[i]).get()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm changing this as well. Let me do this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I will leave this to @andyfengHKU .
But i think it's better that we just store either column pos or value vector, as these two are redundant. I would prefer storing column pos, and dynamically fetch the value vector. We can discuss a bit more.

src/storage/copier/reader_state.cpp Outdated Show resolved Hide resolved
src/storage/copier/reader_state.cpp Outdated Show resolved Hide resolved
src/storage/copier/reader_state.cpp Outdated Show resolved Hide resolved
src/storage/copier/reader_state.cpp Outdated Show resolved Hide resolved
@ray6080 ray6080 force-pushed the morsel branch 2 times, most recently from 42bf4ec to c222939 Compare August 27, 2023 09:10
@ray6080 ray6080 merged commit ee029b9 into master Aug 27, 2023
10 checks passed
@ray6080 ray6080 deleted the morsel branch August 27, 2023 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants