Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support more native parquet types #1600

Merged
merged 1 commit into from
Jun 1, 2023
Merged

Support more native parquet types #1600

merged 1 commit into from
Jun 1, 2023

Conversation

acquamarin
Copy link
Collaborator

@acquamarin acquamarin commented May 31, 2023

This PR adds native support to PARQUET struct/date32/list/fixedSizeList.
Unresolved issues:

  1. Add native support to parquet timestamp/interval type: Add native support to parquet timestamp/interval #1601
  2. When the storage for lists/overflow has been refactored, we should consider do direct memcpy from arrow list array to our inMemColumnChunk.

@acquamarin acquamarin requested a review from ray6080 May 31, 2023 18:30
@codecov
Copy link

codecov bot commented May 31, 2023

Codecov Report

Patch coverage: 80.73% and project coverage change: -0.02 ⚠️

Comparison is base (da3b552) 91.93% compared to head (77c25ee) 91.91%.

❗ Current head 77c25ee differs from pull request most recent head 285b2df. Consider uploading reports for the commit 285b2df to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1600      +/-   ##
==========================================
- Coverage   91.93%   91.91%   -0.02%     
==========================================
  Files         704      704              
  Lines       25567    25666      +99     
==========================================
+ Hits        23504    23590      +86     
- Misses       2063     2076      +13     
Impacted Files Coverage Δ
...age/in_mem_storage_structure/in_mem_column_chunk.h 100.00% <ø> (ø)
...rc/include/storage/storage_structure/in_mem_file.h 100.00% <ø> (ø)
...e/in_mem_storage_structure/in_mem_column_chunk.cpp 80.33% <67.79%> (-0.95%) ⬇️
src/storage/storage_structure/in_mem_file.cpp 82.80% <96.00%> (+3.85%) ⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@acquamarin acquamarin merged commit 9b8e0ee into master Jun 1, 2023
6 checks passed
@acquamarin acquamarin deleted the arrow-types branch June 1, 2023 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants