Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reader: remove counting of blocks #2166

Merged
merged 1 commit into from
Oct 11, 2023
Merged

reader: remove counting of blocks #2166

merged 1 commit into from
Oct 11, 2023

Conversation

Riolku
Copy link
Contributor

@Riolku Riolku commented Oct 7, 2023

This allows us to move toward removing all upfront counting. None of the readers actually depend on upfront counting of the number of blocks.

Depends on #2156 , to avoid a merge conflict.

@codecov
Copy link

codecov bot commented Oct 7, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (abca5e6) 89.92% compared to head (67cbaca) 90.04%.
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2166      +/-   ##
==========================================
+ Coverage   89.92%   90.04%   +0.11%     
==========================================
  Files         989      989              
  Lines       35637    35452     -185     
==========================================
- Hits        32048    31921     -127     
+ Misses       3589     3531      -58     
Files Coverage Δ
src/common/arrow/arrow_converter.cpp 97.24% <ø> (+27.24%) ⬆️
src/common/types/date_t.cpp 89.62% <ø> (ø)
src/common/types/dtime_t.cpp 74.68% <ø> (ø)
src/common/types/types.cpp 92.25% <100.00%> (-0.02%) ⬇️
src/function/base_lower_upper_operation.cpp 100.00% <100.00%> (ø)
src/include/common/types/cast_helpers.h 87.14% <ø> (ø)
...include/function/arithmetic/arithmetic_functions.h 93.22% <100.00%> (+0.11%) ⬆️
src/include/function/boolean/boolean_functions.h 85.71% <100.00%> (-3.18%) ⬇️
...r/operator/persistent/reader/csv/base_csv_reader.h 100.00% <ø> (ø)
...e/processor/operator/persistent/reader_functions.h 100.00% <100.00%> (ø)
... and 20 more

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@ray6080 ray6080 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add isEOF() to serial csv reader too? I wonder if that can simplify the control logic by eventually remove doneAfterEmptyBlock later.

tools/shell/shell_runner.cpp Outdated Show resolved Hide resolved
@Riolku
Copy link
Contributor Author

Riolku commented Oct 10, 2023

By removing doneAfterEmptyBlock, you mean change this to a universal isEOF method?

@Riolku Riolku marked this pull request as ready for review October 10, 2023 15:15
@Riolku Riolku force-pushed the remove-reader-blocks branch 2 times, most recently from b298c18 to b0e9df4 Compare October 10, 2023 15:15
@Riolku Riolku marked this pull request as draft October 10, 2023 15:16
@Riolku Riolku force-pushed the remove-reader-blocks branch 5 times, most recently from b45df3c to 6ae22b8 Compare October 10, 2023 19:10
This allows us to move toward removing all upfront counting. None of the
readers actually depend on upfront counting of the number of blocks.
@Riolku Riolku marked this pull request as ready for review October 10, 2023 21:39
@Riolku Riolku merged commit 5aab076 into master Oct 11, 2023
11 checks passed
@Riolku Riolku deleted the remove-reader-blocks branch October 11, 2023 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants