Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to copy from multiple parquet files #1377

Merged
merged 1 commit into from
Mar 15, 2023
Merged

Conversation

acquamarin
Copy link
Collaborator

@acquamarin acquamarin commented Mar 14, 2023

  1. Add support to copy from multiple parquet files.
  2. Remove support to load arrow files.

Copy link
Contributor

@andyfengHKU andyfengHKU left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a separate question. When using glob, can we try to ensure glob("a*.csv") will be expanded in alphabetical order, i.e. a0.csv, a1.csv ...

@codecov
Copy link

codecov bot commented Mar 15, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.23 🎉

Comparison is base (0fb8cd9) 92.52% compared to head (d81da19) 92.76%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1377      +/-   ##
==========================================
+ Coverage   92.52%   92.76%   +0.23%     
==========================================
  Files         641      641              
  Lines       22134    22077      -57     
==========================================
  Hits        20480    20480              
+ Misses       1654     1597      -57     
Impacted Files Coverage Δ
src/binder/bind/bind_copy.cpp 87.50% <ø> (-0.44%) ⬇️
src/common/copier_config/copier_config.cpp 100.00% <ø> (ø)
src/include/common/copier_config/copier_config.h 100.00% <ø> (ø)
src/include/storage/copy_arrow/copy_node_arrow.h 100.00% <ø> (ø)
src/include/storage/copy_arrow/copy_rel_arrow.h 100.00% <ø> (ø)
...include/storage/copy_arrow/copy_structures_arrow.h 100.00% <ø> (ø)
src/common/file_utils.cpp 71.83% <100.00%> (+0.40%) ⬆️
src/storage/copy_arrow/copy_node_arrow.cpp 95.03% <100.00%> (+11.05%) ⬆️
src/storage/copy_arrow/copy_rel_arrow.cpp 95.61% <100.00%> (+3.80%) ⬆️
src/storage/copy_arrow/copy_structures_arrow.cpp 82.05% <100.00%> (+5.95%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@acquamarin acquamarin merged commit e3961e6 into master Mar 15, 2023
@acquamarin acquamarin deleted the parquet-loader branch March 15, 2023 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants