Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework copy transaction to not rely on file renaming #1649

Merged
merged 1 commit into from
Jun 10, 2023
Merged

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Jun 8, 2023

No description provided.

@ray6080 ray6080 requested a review from acquamarin June 8, 2023 16:53
@codecov
Copy link

codecov bot commented Jun 8, 2023

Codecov Report

Patch coverage: 76.28% and project coverage change: -0.01 ⚠️

Comparison is base (976d035) 91.45% compared to head (4d7df5f) 91.44%.

❗ Current head 4d7df5f differs from pull request most recent head 8f124a0. Consider uploading reports for the commit 8f124a0 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1649      +/-   ##
==========================================
- Coverage   91.45%   91.44%   -0.01%     
==========================================
  Files         726      727       +1     
  Lines       26411    26396      -15     
==========================================
- Hits        24153    24138      -15     
  Misses       2258     2258              
Impacted Files Coverage Δ
src/include/processor/operator/copy/copy_node.h 100.00% <ø> (ø)
src/include/storage/storage_structure/column.h 81.53% <0.00%> (-14.83%) ⬇️
...lude/storage/storage_structure/storage_structure.h 100.00% <ø> (ø)
src/include/storage/store/node_table.h 72.22% <0.00%> (-27.78%) ⬇️
src/include/storage/wal_replayer_utils.h 100.00% <ø> (ø)
...rc/storage/storage_structure/storage_structure.cpp 98.61% <ø> (+9.86%) ⬆️
src/storage/store/node_table.cpp 100.00% <ø> (ø)
src/storage/wal_replayer_utils.cpp 98.46% <ø> (-0.08%) ⬇️
src/storage/wal_replayer.cpp 92.81% <46.66%> (-2.21%) ⬇️
...e/processor/operator/scan/scan_rel_table_columns.h 100.00% <100.00%> (ø)
... and 13 more

... and 16 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

src/include/storage/storage_structure/column.h Outdated Show resolved Hide resolved
@@ -40,8 +40,8 @@ void CopyNodeSharedState::initializeColumns(
// Skip SERIAL, as it is not physically stored.
continue;
}
auto fPath = storage::StorageUtils::getNodePropertyColumnFName(directory,
nodeTableSchema->tableID, property.propertyID, common::DBFileType::WAL_VERSION);
auto fPath = storage::StorageUtils::getNodePropertyColumnFName(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove the dbfile type parameter from the getNodePropertyColumnFName, since we are always getting the original version of the node file

@@ -36,15 +36,15 @@ std::unique_ptr<DirectedInMemRelData> RelCopyExecutor::initializeDirectedInMemRe
auto relColumns = std::make_unique<DirectedInMemRelColumns>();
relColumns->adjColumn = std::make_unique<InMemColumn>(
StorageUtils::getAdjColumnFName(
outputDirectory, tableSchema->tableID, direction, DBFileType::WAL_VERSION),
outputDirectory, tableSchema->tableID, direction, DBFileType::ORIGINAL),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto, we can remove the dbfiletype parameter

src/include/storage/store/rel_table.h Outdated Show resolved Hide resolved
src/processor/operator/copy/copy_node.cpp Outdated Show resolved Hide resolved
src/processor/operator/copy/copy_rel.cpp Outdated Show resolved Hide resolved
@ray6080 ray6080 force-pushed the transaction branch 2 times, most recently from 14098f5 to 5f1baf4 Compare June 9, 2023 17:12
@ray6080 ray6080 merged commit b2c85b2 into master Jun 10, 2023
8 checks passed
@ray6080 ray6080 deleted the transaction branch June 10, 2023 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants