Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix upload error when cookbook tarball uid/gid is very large #1810

Merged
merged 3 commits into from
May 15, 2019

Conversation

robbkidd
Copy link
Contributor

@robbkidd robbkidd commented May 14, 2019

Description

RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers.

This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent.

Related Issue

Fixes #1806

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

  • I have read the CONTRIBUTING document.
  • I have run the pre-merge tests locally and they pass.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • All commits have been signed-off for the Developer Certificate of Origin.

@robbkidd robbkidd requested a review from a team as a code owner May 14, 2019 16:20
@robbkidd robbkidd force-pushed the robb/tars-are-weird branch 4 times, most recently from 00d941a to e73e455 Compare May 14, 2019 17:46
@robbkidd
Copy link
Contributor Author

@teknofire Look past all the force pushing and this is where I would up after our pairing on Friday. Thoughts?

@robbkidd robbkidd requested a review from a team May 14, 2019 17:56
Copy link
Contributor

@tyler-ball tyler-ball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 question about the mime type but LGTM otherwise

@robbkidd robbkidd force-pushed the robb/tars-are-weird branch 2 times, most recently from 4507051 to 37fddd0 Compare May 14, 2019 18:52
Fixes #1806

RubyGems' Gem::Package::TarReader has some strict octal checking that is
incompatible with pax format tar files with large (greater than 8^8) uid
or gid entries (see rubygems/rubygems#2213). This can happen
particularly when publishing a cookbook on a Windows workstation that is
a member of a domain; the group IDs there can be very large numbers.

This change swaps out the handling of tar files from RubyGems' library
to Chef's own ffi-libarchive. The previous logic also assumed that
cookbook tarballs must be GZipped which is probably an assumption made
elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving
of the handling of multiple formats of tar files, the requirement for
GZipped tar files is preserved and implemented with FileMagic, already a
dependency of Supermarket via the Fieri engine subcomponent.

Signed-off-by: Robb Kidd <rkidd@chef.io>
Copy link

@teknofire teknofire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! :shipit:

@robbkidd
Copy link
Contributor Author

I missed adding libarchive to the omnibus build! Working on that and a test build locally of the package.

Signed-off-by: Robb Kidd <rkidd@chef.io>
Signed-off-by: Robb Kidd <rkidd@chef.io>
@robbkidd
Copy link
Contributor Author

OK. libarchive added to be included in omnibus build. I'm pretty sure the @chef/jex-team does not have any concerns with libarchive also being added to the Travis build. So ... shipping it.

@robbkidd robbkidd merged commit ac5f9ba into master May 15, 2019
@chef-ci chef-ci deleted the robb/tars-are-weird branch May 15, 2019 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: Critical Type: Bug Does not work as expected.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

supermarket upload errors when cookbook contents is owned by very large uid or gid
3 participants