Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean overridden dependencies #150

Closed
wants to merge 1 commit into from
Closed

Conversation

radu-matei
Copy link
Member

@radu-matei radu-matei commented Oct 19, 2019

depends on cnabio/cnab-to-oci#77
ref #121

When that PR is merged, we should update this PR, do a release of cnab-to-oci, then a release of cnab-go.

There are still some dependencies that are still overridden:

[[override]]
  name = "github.com/containerd/containerd"
  version = "v1.3.0"

[[override]]
  name = "github.com/docker/distribution"
  #version = "v2.7.1"
  revision = "0d3efadf0154c2b8a4e7b6621fff9809655cc580"

[[override]]
  name = "github.com/docker/docker-credential-helpers"
  version = "v0.6.3"

[[override]]
  name = "golang.org/x/sys"
  revision = "f49334f85ddcf0f08d7fb6dd7363e9e6d6b777eb"
  • without the x/sys override: github.com/docker/docker/pkg/system/filesys_windows.go:112:24: cannot use uintptr(unsafe.Pointer(&sd[0])) (type uintptr) as type *"github.com/deislabs/cnab-go/vendor/golang.org/x/sys/windows".SECURITY_DESCRIPTOR in assignment

  • go-containerregistry - I got this from duffle - I think it relates to image-relocation - @glyn might be able to help`

  • containerd, distribution, docker-credential-helpers - this helps pin compatible versions across cnab-to-oci, cli, client, docker/docker. Feel free to try turning these into constraints rather than overrides, I didn't have any luck so far.

Please test this on all operating systems and supported Go versions - I also added the osusergo build tag (see golang/go#23265) - without it, cross compilation on Darwin fails.

@radu-matei
Copy link
Member Author

Not sure what Azure Pipelines is doing, cancelled the build and reran it, it's green now - https://dev.azure.com/deislabs/cnab-go/_build/results?buildId=3919

@radu-matei
Copy link
Member Author

Also, we should probably consider adding the vendor/ directory to source control, like we do in the rest of the Go projects in CNAB.

Copy link
Member

@vdice vdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this @radu-matei ! Looks good for me on go version go1.12.9 darwin/amd64 after a rm -rf vendor/ && dep ensure && make build test cycle.

Had one proposal, but definitely understand if we wish it to be a follow-up.

go-tests = true
[[override]]
name = "github.com/google/go-containerregistry"
revision = "ef12d49c8daf6a6d72978966d5945e39bb898b4a"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we can override with a more recent commit for this repo? Though, perhaps this was pinned at the inherited commit on master?

google/go-containerregistry@0e086ae brings in a fix, courtesy @glyn, related to supporting pushing images (or indexes) with differing media types, which Porter hit with a use case mentioned in vmware-archive/image-relocation#34

Copy link
Member Author

@radu-matei radu-matei Oct 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to defer this change to someone familiar with the expected behavior of this library (cc @glyn).

opened #152 to track this

Add osusergo build tag for static compilation

Override x/sys because of Windows unsafe pointer bug

Override go-containerregistry

Signed-off-by: Radu M <root@radu.sh>
@radu-matei radu-matei changed the title [WIP] Clean overridden dependencies Clean overridden dependencies Oct 30, 2019

[[override]]
name = "golang.org/x/sys"
revision = "f49334f85ddcf0f08d7fb6dd7363e9e6d6b777eb"

[prune]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why were the prune options removed? If you are thinking about checking in vendor eventually, then we should be keeping the prune options.

[[constraint]]
name = "github.com/docker/cli"
revision = "f95ca8e1ba6c22c9abcdbf65e8dcc39c53958bba"

[[override]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overrides without constraints are not communicated to people who depend on your library. So other people who depend upon cnab-go won't get any help from dep on these dependencies, and won't know that containerd for example should be at version 1.3.0.

Each override needs to be paired with a matching constraint for downstream consumers of the library.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I forgot to mention the other part of the trick to let you constrain a transitive dependency with dep:

https://github.com/deislabs/porter/blob/master/pkg/dep/dep.go

It's a total hack, sorry you can blame google for killing the project before we got a chance to finish...

Anyhoo, you add a file like that in an out of the way package to get dep to consider it direct dependency. Now you can add constraints, and communicate your dependencies to your downstream consumers properly.

Copy link
Contributor

@carolynvs carolynvs Oct 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can get away with removing the overrides entirely with that this workaround. Give it a try.

Then in the downstream consumer's Gopkg.toml they can reference cnab-go without having to add a bunch of overrides and constraints because cnab-go did all the proper work up-front. That's what porter has been doing with its mixin repos.

[[override]]
name = "k8s.io/api"
revision = "kubernetes-1.11.2"
#version = "v2.7.1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using that revision instead of v2.7.1? That commented out line is a bit misleading. Whenever we do pin to a specific revision it really helps to explain why in the file, so that people can understand when it's safe to move back to a tag, what issue they should be watching and waiting for it to be merged, etc.

@carolynvs
Copy link
Contributor

I agree that a follow-up PR where we commit vendor would be appreciated. 👍

@jlegrone
Copy link
Member

jlegrone commented Dec 7, 2019

@radu-matei sorry for the churn, can you rebase this PR now that #161 is merged?

@carolynvs
Copy link
Contributor

Closing now that we use go mods.

@carolynvs carolynvs closed this Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants