Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement virtualFileSystem #2586

Merged
merged 1 commit into from
Dec 19, 2023
Merged

Implement virtualFileSystem #2586

merged 1 commit into from
Dec 19, 2023

Conversation

acquamarin
Copy link
Collaborator

@acquamarin acquamarin commented Dec 16, 2023

This is the first set of PRs to support HTTPFS/S3 filesystem extension.
This PR wraps all file operations under the FileSystem class.
Classes introduced:

  1. FileSystem: base class, defines all file operations supported in kuzu.
  2. VirtualFileSystem: dispatch the file operation to the corresponding file system.
  3. LocalFileSystem: default filesystem in kuzu, operates local files only.

@acquamarin acquamarin changed the title Filesystem Implement filesystem Dec 16, 2023
@acquamarin acquamarin changed the title Implement filesystem Implement virtualFileSystem Dec 16, 2023
Copy link

codecov bot commented Dec 16, 2023

Codecov Report

Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (1a33e48) 93.15% compared to head (54d3328) 93.19%.
Report is 7 commits behind head on master.

Files Patch % Lines
src/common/file_system/virtual_file_system.cpp 72.41% 8 Missing ⚠️
src/common/file_system/local_file_system.cpp 93.61% 3 Missing ⚠️
.../include/function/table_functions/scan_functions.h 66.66% 1 Missing ⚠️
...operator/persistent/reader/csv/serial_csv_reader.h 66.66% 1 Missing ⚠️
.../operator/persistent/reader/parquet/thrift_tools.h 66.66% 1 Missing ⚠️
src/planner/operator/logical_plan_util.cpp 80.00% 1 Missing ⚠️
...ator/persistent/reader/csv/parallel_csv_reader.cpp 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2586      +/-   ##
==========================================
+ Coverage   93.15%   93.19%   +0.03%     
==========================================
  Files        1027     1030       +3     
  Lines       38365    38664     +299     
==========================================
+ Hits        35740    36032     +292     
- Misses       2625     2632       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/include/storage/index/hash_index_builder.h Outdated Show resolved Hide resolved
src/include/storage/storage_structure/in_mem_file.h Outdated Show resolved Hide resolved
src/common/file_system/virtual_file_system.cpp Outdated Show resolved Hide resolved
src/common/file_system/virtual_file_system.cpp Outdated Show resolved Hide resolved
src/include/binder/binder.h Outdated Show resolved Hide resolved
src/include/catalog/catalog_content.h Outdated Show resolved Hide resolved
src/include/common/file_system/file_system.h Outdated Show resolved Hide resolved
src/include/main/database.h Outdated Show resolved Hide resolved
src/include/storage/stats/table_statistics_collection.h Outdated Show resolved Hide resolved
@acquamarin acquamarin merged commit 76af5c3 into master Dec 19, 2023
14 checks passed
@acquamarin acquamarin deleted the filesystem branch December 19, 2023 02:56
@ray6080 ray6080 mentioned this pull request Dec 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants