Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC + Add bug #6252

Closed
Stebalien opened this issue Apr 25, 2019 · 3 comments
Closed

GC + Add bug #6252

Stebalien opened this issue Apr 25, 2019 · 3 comments
Assignees
Labels
kind/bug A bug in existing code (including security flaws) P0 Critical: Tackled by core team ASAP

Comments

@Stebalien
Copy link
Member

I believe there is a bug in the file adder that happens if we GC half-way through pinning a file.

When GCing while pinning, we pin a temporary root, unlock the pin lock, GC, then relock the temp lock. When we do this, we call:

https://github.com/ipfs/go-ipfs/blob/c1bc202f41ab07debdecd806ba97cec0a1241bf7/core/coreunix/add.go#L134-L138

This, unfortunately, caches the root. A subsequent attempt to pin the temporary root will pin the old temporary root, not a new one. The final pin attempt (https://github.com/ipfs/go-ipfs/blob/c1bc202f41ab07debdecd806ba97cec0a1241bf7/core/coreunix/add.go#L330) will pin the real root without checking if we still have all the blocks.

This looks related to #6207 however, I'm pretty sure this bug was introduced in 0.4.20. 0.4.19 had a slightly different bug because we wouldn't pin the correct root at the end, we'd re-pin the temporary root. While that bug could cause data loss, it wouldn't explain #6207 (which looks more like this bug).

@Stebalien Stebalien added the kind/bug A bug in existing code (including security flaws) label Apr 25, 2019
@BenLubar
Copy link
Contributor

If garbage collection worked via reference counting, I believe all garbage collection race conditions could be avoided assuming each individual object was discarded atomically with reference count changes

@Stebalien
Copy link
Member Author

@BenLubar I agree we should seriously consider using reference counting when pinning. However, that won't really help in this case. The issue here is that we simply aren't pinning.

@Stebalien Stebalien added interrupt P0 Critical: Tackled by core team ASAP labels Apr 25, 2019
@obo20
Copy link

obo20 commented Apr 26, 2019

Adding my voice in support of fixing garbage collection via reference counting. Being unable to add content to a node while a garbage collection is running is pretty problematic.

@eingenito eingenito self-assigned this Apr 30, 2019
eingenito pushed a commit that referenced this issue May 1, 2019
License: MIT
Signed-off-by: Erik Ingenito <erik.ingenito@protocol.ai>
eingenito pushed a commit that referenced this issue May 1, 2019
License: MIT
Signed-off-by: Erik Ingenito <erik.ingenito@protocol.ai>
eingenito pushed a commit that referenced this issue May 1, 2019
License: MIT
Signed-off-by: Erik Ingenito <erik.ingenito@protocol.ai>
eingenito pushed a commit that referenced this issue May 1, 2019
License: MIT
Signed-off-by: Erik Ingenito <erik.ingenito@protocol.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws) P0 Critical: Tackled by core team ASAP
Projects
None yet
Development

No branches or pull requests

4 participants