Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip first pass read of parquet data #1480

Merged
merged 1 commit into from
Apr 22, 2023
Merged

Skip first pass read of parquet data #1480

merged 1 commit into from
Apr 22, 2023

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Apr 21, 2023

Current implementation unnecessarily read parquet data twice. The first pass is just to figure out num of rows per row group, for this, simply read the metadata is enough.

@codecov
Copy link

codecov bot commented Apr 21, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (a0a51d5) 92.13% compared to head (9b50e11) 92.14%.

❗ Current head 9b50e11 differs from pull request most recent head 3a41372. Consider uploading reports for the commit 3a41372 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1480   +/-   ##
=======================================
  Coverage   92.13%   92.14%           
=======================================
  Files         670      670           
  Lines       23806    23806           
=======================================
+ Hits        21934    21936    +2     
+ Misses       1872     1870    -2     
Impacted Files Coverage Δ
src/storage/copier/node_copier.cpp 93.52% <ø> (ø)
src/storage/copier/table_copy_executor.cpp 86.89% <100.00%> (ø)

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ray6080 ray6080 merged commit 3e04497 into master Apr 22, 2023
@ray6080 ray6080 deleted the copy-rework branch April 22, 2023 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants