Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix mfs cp/mv bug #2690

Closed
wants to merge 3 commits into from
Closed

fix mfs cp/mv bug #2690

wants to merge 3 commits into from

Conversation

hackergrrl
Copy link
Contributor

Addresses #2654

@whyrusleeping whyrusleeping added the need/review Needs a review label May 13, 2016
ndir := NewDirectory(d.ctx, name, nd, d, d.dserv)
d.childDirs[name] = ndir
default:
panic("unrecognized! (NYI)")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets definitely not panic here. there are other types (symlinks, raw blocks) that might pop up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

@whyrusleeping whyrusleeping added need/author-input Needs input from the original author and removed need/review Needs a review labels May 13, 2016
@hackergrrl hackergrrl added need/review Needs a review and removed need/author-input Needs input from the original author labels May 14, 2016
@Kubuxu
Copy link
Member

Kubuxu commented May 14, 2016

@Kubuxu Kubuxu added need/author-input Needs input from the original author and removed need/review Needs a review labels May 14, 2016
@hackergrrl hackergrrl added need/review Needs a review and removed need/author-input Needs input from the original author labels May 16, 2016
@hackergrrl
Copy link
Contributor Author

Green!

@hackergrrl
Copy link
Contributor Author

@whyrusleeping anything else needed here?

case ft.TDirectory:
ndir := NewDirectory(d.ctx, name, nd, d, d.dserv)
d.childDirs[name] = ndir
case ft.TFile, ft.TMetadata, ft.TRaw, ft.TSymlink:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm pretty sure we don't want to treat these all the same

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is consistent with what the old code did -- what should happen instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no its not, take a look at the code in childNode

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome -- cacheNode is exactly what I wanted here.

@hackergrrl hackergrrl force-pushed the mfs_file_dirs branch 2 times, most recently from 1166e3f to c39545f Compare May 19, 2016 01:58
@@ -131,7 +131,7 @@ func (d *Directory) cacheNode(name string, nd *dag.Node) (FSNode, error) {
ndir := NewDirectory(d.ctx, name, nd, d, d.dserv)
d.childDirs[name] = ndir
return ndir, nil
case ufspb.Data_File, ufspb.Data_Raw:
case ufspb.Data_File, ufspb.Data_Raw, ufspb.Data_Symlink:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

symlink is not correct here. We don't currently handle them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What the old code in AddChild did was check if len(links) == 0, and if so, cached it as a file. it didn't check type (symlink, file, etc) at all like cacheNode does.

This is because ipfs add uses the mfs package to add files, which means in order to handle symlinks in directories (like we do today on ipfs add), AddChild needs to also support symlinks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

symlink is still not correct here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we pass a dag node that represents a symlink to 'new file' here, its not going to work properly if we try to use it (NewDagModifier will produce undefined behaviour when trying to modify a symlink).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason this currently works (since we use mfs for add) is that we never instantiate a NewFile as a symlink (or when we do, in the current incorrect code, it never gets used).

@whyrusleeping
Copy link
Member

the caching here is likely the cause of #2615

@hackergrrl
Copy link
Contributor Author

Looks like it was introduced in #2531. I think we had best remove the caching here and figure out how to fix #2531 separately; it's much less important than this being broken (and #2615).

@hackergrrl hackergrrl added need/author-input Needs input from the original author and removed need/review Needs a review labels May 19, 2016
@whyrusleeping
Copy link
Member

@noffle could I get an update here? This is causing issues for people.

@hackergrrl
Copy link
Contributor Author

hackergrrl commented May 31, 2016

Hey @whyrusleeping. The fix is simple enough: remove the caching piece introduced in #2531. However, this will also cause folders to appear out of order / not at all when using --only-hash. Fixing that piece is going to be more complicated. Either we can re-introduce the smaller regression (--only-hash output) now, or I can try and find some time this sprint to take a look.

@whyrusleeping
Copy link
Member

@noffle lets go ahead and temporarily disable the only hash flag when combined with the recursive flag, we can print a notice to the user referencing a tracking bug.

License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
@hackergrrl hackergrrl force-pushed the mfs_file_dirs branch 2 times, most recently from a438cca to 74517ea Compare June 1, 2016 21:31
License: MIT
Signed-off-by: Stephen Whitmore <noffle@ipfs.io>
@hackergrrl
Copy link
Contributor Author

@whyrusleeping good idea -- added

whyrusleeping added a commit that referenced this pull request Jun 2, 2016
License: MIT
Signed-off-by: Jeromy <why@ipfs.io>
@hackergrrl
Copy link
Contributor Author

hackergrrl commented Jun 2, 2016

Looks like this approach won't work either. Another approach -- here are why's notes:

10:59:42 whyrusleeping | the easy solution i see, is adding a way to set your own mfs root object on the fileadder
11:00:02 whyrusleeping | creating that with an in memory dagservice (so all of its writes are kept)
11:00:21 whyrusleeping | and giving the FileAdder itself the 'null' dagservice that throws away all the writes

I'll try and get to this next week.

@whyrusleeping
Copy link
Member

this has been fixed in #2795

@whyrusleeping whyrusleeping deleted the mfs_file_dirs branch June 9, 2016 05:09
@hackergrrl
Copy link
Contributor Author

Thank you for crushing this bug into a fine paste.

On 06/08 22:09, Jeromy Johnson wrote:

this has been fixed in #2795


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#2690 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/author-input Needs input from the original author
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants