Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rdf test fix #2629

Merged
merged 1 commit into from
Jan 4, 2024
Merged

Rdf test fix #2629

merged 1 commit into from
Jan 4, 2024

Conversation

andyfengHKU
Copy link
Contributor

  • Fix rdf test
  • Differentiate local global index builder.
    • Global index builder: created during initGlobalState and stored in shared state. Responsible for
      • creating IndexBuilderSharedState
      • inserting and flushing last node group during finalize
    • Local index builder: created during initLocalState and stored in operator. Responsible for
      • local inserting and flushing
  • Move copy node finalize logic to shared state.

Copy link
Contributor

@Riolku Riolku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise LGTM, thanks for fixing these up.

@@ -20,7 +20,7 @@ using StringBuffer = common::StaticVector<std::pair<std::string, common::offset_

class IndexBuilderGlobalQueues {
public:
explicit IndexBuilderGlobalQueues(std::unique_ptr<storage::PrimaryKeyIndexBuilder> pkIndex);
explicit IndexBuilderGlobalQueues(storage::PrimaryKeyIndexBuilder* pkIndex);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, why can't the global queues have ownership of the index anymore? We have to be careful that accesses to the index in any other fashion are properly synchronized, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because RDF copy requires CopyNode and CopyRel in a single physical plan. And CopyRel needs to access PrimaryKeyIndexBuilder before it being flushed to disk. It's easier to access if the ownership is in CopyNode instead of GlobalQueue

Copy link

codecov bot commented Jan 3, 2024

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (92350a2) 91.11% compared to head (838c1a4) 93.21%.

Files Patch % Lines
src/processor/operator/index_lookup.cpp 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2629      +/-   ##
==========================================
+ Coverage   91.11%   93.21%   +2.09%     
==========================================
  Files        1058     1058              
  Lines       39235    39234       -1     
==========================================
+ Hits        35750    36571     +821     
+ Misses       3485     2663     -822     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@andyfengHKU andyfengHKU merged commit a8e55c5 into master Jan 4, 2024
14 checks passed
@andyfengHKU andyfengHKU deleted the rdf-test-fix branch January 4, 2024 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants