-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
supermarket upload errors when cookbook contents is owned by very large uid or gid #1806
Comments
This comment chef-boneyard/chef-dk#2018 (comment) from the other issue in ChefDK found the issue to be with the release of supermarket 3.2.2 that RubyGems was updated and by reverting RubyGems version they were able to work around the issue. |
@Stromweld Thanks for the copy pasta to get this information here. Investigating. |
Interestingly, I am unable to replicate this in my test environment using the linked to One difference I notice between our environments is our platform versions; I'm using macOS 10.14 and you've got 10.12. I wonder if underlying libraries or utilities from the OS are producing a tarball with different headers. The error definitely appears on the Supermarket host while attempting to process the resultant tarball. I suspect one fix could probably be to teach Supermarket to work with tarballs via Supermarket
Chef Workstation
Platform
|
I’m have this same tarballs octal error if I try to upload to private chef or community chef supermarket. I have latest chef workstation and am running on macos 10.14. Our private supermarket is running on CentOS 7.6 and was built with chef-ingredient cookbook. Originally started on supermarket 3.1.96 and it auto upgrades with new release. |
On my Mac I installed chef-workstation via homebrew. And have upgraded via brew cask upgrades. After supermarket upgraded to 3.2.2 is when knife supermarket share command started throwing the error for me. I was also on chef workstation 0.2.43 at the time. Currently I’m on workstation 0.2.53 and supermarket 3.3.1 and issue is still persisting. |
Here's the output of those commands on my machine:
I also get this error trying to use stove as well:
|
Yep. Again, it's the processing of the tarball on the Supermarket server that causes the error. I have been able to replicate the error. I downloaded the latest version of the stig cookbook from Supermarket itself to tinker with. The problem is, indeed, Supermarket's processing of uploaded cookbook tarballs with I am reluctant to downgrade RubyGems in Supermarket, though, because of the plethora of CVEs addressed in versions since RubyGems v2.7.5. I'm currently investigating what other options we made have. |
More notes: can happen more often with cookbooks shared from workstations affiliated with an LDAP/ActiveDirectory because those environments often have very large
I think there are multiple solutions:
|
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>
Fixed and released in v3.3.3 and currently running at https://supermarket.chef.io. |
sweet thanks @robbkidd |
@Stromweld Thanks for reporting and with the information to suss out what was going on in there! |
Copied from chef-boneyard/chef-dk#2018 as I'm also seeing the same issue.
Description
When trying to upload cookbook to Chef Supermarket, I get the error
ERROR: Multipart POST part 'tarball' is corrupt: "\x80\x00\x00\x00\x0E:\xBFD" is not an octal string
Example:
I did try switching gem versions as mentioned here:
This did not help.
Possible related issues:
ChefDK Version
Using Chef Workstation.
Platform Version
MacOS:
Replication Case
What i do locally for a cloned tagged checkout of https://github.com/USGS-CIDA/stig:
My knife-chefio.rb file looks like:
Stacktrace
https://gist.github.com/isuftin/e8bb0804d5510a0258ead2ea65638fca
The text was updated successfully, but these errors were encountered: