Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transaction to rel deletions #1126

Merged
merged 1 commit into from
Dec 21, 2022
Merged

Add transaction to rel deletions #1126

merged 1 commit into from
Dec 21, 2022

Conversation

acquamarin
Copy link
Collaborator

@acquamarin acquamarin commented Dec 20, 2022

  1. This PR adds the transaction logic for rel-deletion.
  2. Refactors the prepare-commiting logic for lists:
    Instead of commiting each list at a time, we commit each relTable at a time.

src/include/storage/storage_structure/lists/lists.h Outdated Show resolved Hide resolved
ListUpdates() : emptyListInPersistentStore{false} {}

inline bool hasUpdates() const {
return emptyListInPersistentStore || !insertedRelsTupleIdxInFT.empty() ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you check emptyListInPersistentStore? Whether the list was empty in persistent store seems irrelevant to whether v has updates or not.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of the hasUpdates() is to check whether we need to update the list/header/metadata for the current nodeOffset.
We want hasUpdates to return true if we want to update the list/header/metadata for a nodeOffset. When we insert a new node, we set the emptyListIPersistentStore to true in the listUpdates for that new node. During commiting, we need to fix the header for that newly inserted node. As a result, we need hasUpdates to return true in order to fix the header.

src/storage/storage_structure/lists/lists.cpp Show resolved Hide resolved
src/storage/storage_structure/lists/lists.cpp Outdated Show resolved Hide resolved
src/storage/storage_structure/lists/lists.cpp Outdated Show resolved Hide resolved
if (!listUpdates.hasUpdates()) {
continue;
}
if (listUpdates.emptyListInPersistentStore) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you want the else if as your first case because it is possible that there was a large list L, which became empty because all of its rels were deleted and we want to still call appendToLargeList on L. Then the current if can be the new else if.

Also this is a hard piece of code to read so comment on each case what the case is handling the way you comment on the current else if that it is the large list optimization etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we already have a comment for the large list optimization.
The if (listUpdates.emptyListInPersistentStore) is a very special branch which is designed for newly inserted nodes.

continue;
}
if (listUpdates.emptyListInPersistentStore) {
listsUpdateIterators[0]->updateList(nodeOffset,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a second comment I'm now debating if this first if clause is justified. Why don't you just have 2 cases:

  1. if the list is a large list and there are only updates;
  2. else

So you don't differentiate between whether the persistent list was empty or not and just call iterator->updateList(...), which will check whether to fill from the persistent store or not. I'm assuming updateList() should actually just work even if the persistent store is empty. It would simplify the code.

Given that you do this, I have a follow up question: Aside from here do we ever use emptyListInPersistentStore? If we remove this if case, it may be unused and if so, we should remove it too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed, let's keep the listsUpdates.emptyListInPersistentStore branch here since the logic is quite different from the newly inserted nodes and other nodes:

If the node is newly inserted, we should only read from the updateStore.
If the node is not newly inserted, we should both read from the persistent and update store.

btw: renamed the emptyListInPersistentStore -> newlyAddedNode

test/runner/e2e_delete_rel_test.cpp Outdated Show resolved Hide resolved
vector<string> expectedResult;
for (auto i = 0; i < NUM_RELS_FOR_ANIMAL_KNOWS_PERSON; i++) {
expectedResult.push_back(to_string(i));
void deleteMultipleRels(bool isCommit, TransactionTestType transactionTestType) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous tests above, e.g., deleteRelsFromSmall/LargList or deleteRelsFromUpdateStore also delete multiple rels? What exactly is this testing? If this test doesn't add much, let's remove it.

Copy link
Collaborator Author

@acquamarin acquamarin Dec 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. deleteRelsFromSmallList is to test the behaviour of deleting multiple rels from a small list. Specifically, i want to test whether the CSR structure has been fixed properly after each deletion.
  2. deleteRelsFromLargelist is to test the behaviour of deleting multiple rels from a large list. Specifically, i want to test whether the largelist has been fixed properly after each deletion.
  3. deleteRelsFromUpdateStore is quite different from deleting from persistent store. When we want to delete rels from updateStore, we can simply remove the rels from the insertedRelTupleIdxes of updateStore.
    BTW: i am not deleting the old tests, i am just extending them to test transaction behaviour


void deleteLargeNumRelsFromLargeList(bool isCommit, TransactionTestType transactionTestType) {
conn->beginWriteTransaction();
conn->query(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this deleting all rels or almost all rels? Maybe remove this test and add two tests that test the edge cases of deleting all rels from a small and large list.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is to test deleting large number of rels from a large list.
Addded two tests for deleting all rels from small/large list.

@acquamarin acquamarin merged commit 04efdad into master Dec 21, 2022
@acquamarin acquamarin deleted the rel-deletion-trx branch December 21, 2022 14:38
@ray6080 ray6080 changed the title add trx for rel-deletion Add transaction to rel deletion Jan 29, 2023
@ray6080 ray6080 changed the title Add transaction to rel deletion Add transaction to rel deletions Jan 29, 2023
@ray6080 ray6080 added the feature New features or missing components of existing features label Jan 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New features or missing components of existing features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants