supermarket upload errors when cookbook contents is owned by very large uid or gid #1806

Stromweld · 2019-04-24T13:50:39Z

Copied from chef-boneyard/chef-dk#2018 as I'm also seeing the same issue.

Description

When trying to upload cookbook to Chef Supermarket, I get the error ERROR: Multipart POST part 'tarball' is corrupt: "\x80\x00\x00\x00\x0E:\xBFD" is not an octal string

Example:

$ knife supermarket share stig -VV -o ../ -c ~/.chef/knife-chefio.rb
INFO: Using configuration from /Users/isuftin/.chef/knife-chefio.rb
INFO: Validating ruby files
INFO: Validating templates
INFO: Syntax OK
Generating metadata for stig from /var/folders/58/n9bfwg050sdg1dmn9688wrcm001rp_/T/chef-stig-build20190410-52953-ff735p/stig/metadata.rb
Making tarball stig.tgz
DEBUG: Signing: method: post, url: https://supermarket.chef.io/api/v1/cookbooks, file: #<File:0x00007ff745d18988>, User-id: isuftin-usgs, Timestamp: 2019-04-10T13:06:07Z
ERROR: Multipart POST part 'tarball' is corrupt: "\x80\x00\x00\x00\x0E:\xBFD" is not an octal string

I did try switching gem versions as mentioned here:

#1549 (comment)

This did not help.

Possible related issues:

ChefDK Version

Using Chef Workstation.

$ chef -v
Chef Workstation: 0.2.53
  chef-run: 0.2.8
  chef-client: 14.10.9
  delivery-cli: 0.0.52 (9d07501a3b347cc687c902319d23dc32dd5fa621)
  berks: 7.0.7
  test-kitchen: 1.24.0
  inspec: 3.6.6

$ chef gem -v
2.7.6

Platform Version

MacOS:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.12.6
BuildVersion:   16G1917

Replication Case

What i do locally for a cloned tagged checkout of https://github.com/USGS-CIDA/stig:

$ knife cookbook metadata stig -o ../
Generating metadata for stig from /Users/isuftin/Chef/cookbooks/stig/metadata.rb

$ knife supermarket share stig -VV -o ../ -c ~/.chef/knife-chefio.rb

INFO: Using configuration from /Users/isuftin/.chef/knife-chefio.rb
INFO: Validating ruby files
INFO: Validating templates
INFO: Syntax OK
Generating metadata for stig from /var/folders/58/n9bfwg050sdg1dmn9688wrcm001rp_/T/chef-stig-build20190410-60032-1e5qcb1/stig/metadata.rb
Making tarball stig.tgz
DEBUG: Signing: method: post, url: https://supermarket.chef.io/api/v1/cookbooks, file: #<File:0x00007fc414f2f398>, User-id: isuftin-usgs, Timestamp: 2019-04-10T13:14:14Z
ERROR: Multipart POST part 'tarball' is corrupt: "\x80\x00\x00\x00\x0E:\xBFD" is not an octal string

My knife-chefio.rb file looks like:

$ cat  ~/.chef/knife-chefio.rb
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "isuftin-usgs"
client_key               "#{current_dir}/isuftin-usgs-chefio.pem"
chef_server_url          "https://supermarket.chef.io/"

Stacktrace

https://gist.github.com/isuftin/e8bb0804d5510a0258ead2ea65638fca

The text was updated successfully, but these errors were encountered:

Stromweld · 2019-04-24T13:55:13Z

This comment chef-boneyard/chef-dk#2018 (comment) from the other issue in ChefDK found the issue to be with the release of supermarket 3.2.2 that RubyGems was updated and by reverting RubyGems version they were able to work around the issue.

robbkidd · 2019-05-08T20:15:00Z

@Stromweld Thanks for the copy pasta to get this information here. Investigating.

robbkidd · 2019-05-08T22:46:39Z

Interestingly, I am unable to replicate this in my test environment using the linked to stig cookbook.

One difference I notice between our environments is our platform versions; I'm using macOS 10.14 and you've got 10.12. I wonder if underlying libraries or utilities from the OS are producing a tarball with different headers.

The error definitely appears on the Supermarket host while attempting to process the resultant tarball. I suspect one fix could probably be to teach Supermarket to work with tarballs via mixlib-archive instead of Gem::Package::TarReader. Another option is to down-or-upgrade the RubyGems embedded within Supermarket (currently v3.0.3).

Supermarket

v3.3.1 running on an Ubuntu 16.04 host.

Chef Workstation

> chef -v
Chef Workstation: 0.2.53
  chef-run: 0.2.8
  chef-client: 14.10.9
  delivery-cli: 0.0.52 (9d07501a3b347cc687c902319d23dc32dd5fa621)
  berks: 7.0.7
  test-kitchen: 1.24.0
  inspec: 3.6.6

> chef gem -v
2.7.6

Platform

> sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14.4
BuildVersion:	18E226

Stromweld · 2019-05-09T00:32:38Z

I’m have this same tarballs octal error if I try to upload to private chef or community chef supermarket. I have latest chef workstation and am running on macos 10.14. Our private supermarket is running on CentOS 7.6 and was built with chef-ingredient cookbook. Originally started on supermarket 3.1.96 and it auto upgrades with new release.

Stromweld · 2019-05-09T00:37:50Z

On my Mac I installed chef-workstation via homebrew. And have upgraded via brew cask upgrades. After supermarket upgraded to 3.2.2 is when knife supermarket share command started throwing the error for me. I was also on chef workstation 0.2.43 at the time. Currently I’m on workstation 0.2.53 and supermarket 3.3.1 and issue is still persisting.

Stromweld · 2019-05-09T20:13:06Z

Here's the output of those commands on my machine:

$ chef -v
Chef Workstation: 0.2.53
  chef-run: 0.2.8
  chef-client: 14.10.9
  delivery-cli: 0.0.52 (9d07501a3b347cc687c902319d23dc32dd5fa621)
  berks: 7.0.7
  test-kitchen: 1.24.0
  inspec: 3.6.6

$ chef gem -v
2.7.6

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14.4
BuildVersion:	18E226

I also get this error trying to use stove as well:

$ stove --endpoint https://chef-supermarket.services.local --username chef-supermarket --key ~/github/netgain/.chef/chef-supermarket.pem --no-ssl-verify --no-git
W: Disabling SSL verification...
W: Neither ChefAPI nor the maintainers are responsible for damages incurred as a result of disabling SSL verification. Please use this with extreme caution, or consider specifying a custom certificate using `config.ssl_pem_file'.
E: Stove experienced an error!
E: ChefAPI::Error::HTTPServerUnavailable
Traceback (most recent call last):
	6: from /opt/chef-workstation/embedded/bin/stove:23:in `<main>'
	5: from /opt/chef-workstation/embedded/bin/stove:23:in `load'
	4: from /opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/stove-7.1.0/bin/stove:5:in `<top (required)>'
	3: from /opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/stove-7.1.0/lib/stove/cli.rb:13:in `execute!'
	2: from /opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/stove-7.1.0/lib/stove/cli.rb:67:in `rescue in execute!'
	1: from /opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-api-0.9.0/lib/chef-api/errors.rb:26:in `message'
/opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-api-0.9.0/lib/chef-api/errors.rb:26:in `read': No such file or directory @ rb_sysopen - /opt/chef-workstation/embedded/lib/ruby/gems/2.5.0/gems/chef-api-0.9.0/templates/errors/http_server_unavailable.erb (Errno::ENOENT)

robbkidd · 2019-05-09T20:28:24Z

Yep. Again, it's the processing of the tarball on the Supermarket server that causes the error.

I have been able to replicate the error. I downloaded the latest version of the stig cookbook from Supermarket itself to tinker with.

The problem is, indeed, Supermarket's processing of uploaded cookbook tarballs with Gem::Package::TarReader from the rubygems library and the strict_octal checking added in rubygems v2.7.6.

I am reluctant to downgrade RubyGems in Supermarket, though, because of the plethora of CVEs addressed in versions since RubyGems v2.7.5. I'm currently investigating what other options we made have.

robbkidd · 2019-05-13T20:55:53Z

More notes: can happen more often with cookbooks shared from workstations affiliated with an LDAP/ActiveDirectory because those environments often have very large uids and/or gids which bumps the data over into extended tar headers. And RubyGem's TarReader can't handle that at the moment.

uid and gid are getting stored in the tarballs created by knife supermarket share and stove because the tar creation does not override the default behavior to record user and group ownership.

I think there are multiple solutions:

(I'm working on this one.) Update Supermarket to deal with tar files with a different library (probably Chef's ffi-libarchive).
Update knife and stove to set a specific, smaller number default for uid and gid—0? 65534?—for cookbook tarballs. This would have the added benefit of removing extraneous (and maybe slightly sensitive) information from the sharable artifact. (But I see now that only gnutar supports that. Curses!)

Fixes #1806 RubyGems' Gem::Package::TarReader has some strict octal checking that is incompatible with pax format tar files with large (greater than 8^8) uid or gid entries (see rubygems/rubygems#2213). This can happen particularly when publishing a cookbook on a Windows workstation that is a member of a domain; the group IDs there can be very large numbers. This change swaps out the handling of tar files from RubyGems' library to Chef's own ffi-libarchive. The previous logic also assumed that cookbook tarballs must be GZipped which is probably an assumption made elsewhere in the Chef ecosystem. Because libarchive is pretty forgiving of the handling of multiple formats of tar files, the requirement for GZipped tar files is preserved and implemented with FileMagic, already a dependency of Supermarket via the Fieri engine subcomponent. Signed-off-by: Robb Kidd <rkidd@chef.io>

robbkidd · 2019-05-15T19:50:57Z

Fixed and released in v3.3.3 and currently running at https://supermarket.chef.io.

Stromweld · 2019-05-15T19:54:51Z

sweet thanks @robbkidd

robbkidd · 2019-05-15T19:56:17Z

@Stromweld Thanks for reporting and with the information to suss out what was going on in there!

robbkidd added Priority: Medium Status: Adopted An issue that is being worked on. Triage: Needs Information Indicates an issue needs more information in order to work on it. Type: Bug Does not work as expected. labels May 8, 2019

robbkidd added Triage: Confirmed Indicates and issue has been confirmed as described. and removed Triage: Needs Information Indicates an issue needs more information in order to work on it. labels May 13, 2019

robbkidd mentioned this issue May 14, 2019

fix upload error when cookbook tarball uid/gid is very large #1810

Merged

6 tasks

robbkidd changed the title ~~supermarket upload is broken~~ supermarket upload errors when cookbook contents is owned by very large uid or gid May 14, 2019

robbkidd closed this as completed in #1810 May 15, 2019

robbkidd mentioned this issue Jun 22, 2019

❗️omnibus build is broken #1817

Closed

robbkidd mentioned this issue Oct 17, 2019

[knife supermarket] set tarballs to uid/gid 0 to avoid large uid/gid errors on consumer machines chef/chef#9002

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

supermarket upload errors when cookbook contents is owned by very large uid or gid #1806

supermarket upload errors when cookbook contents is owned by very large uid or gid #1806

Stromweld commented Apr 24, 2019

Stromweld commented Apr 24, 2019

robbkidd commented May 8, 2019

robbkidd commented May 8, 2019

Stromweld commented May 9, 2019

Stromweld commented May 9, 2019

Stromweld commented May 9, 2019

robbkidd commented May 9, 2019 •

edited

Loading

robbkidd commented May 13, 2019 •

edited

Loading

robbkidd commented May 15, 2019 •

edited

Loading

Stromweld commented May 15, 2019

robbkidd commented May 15, 2019

supermarket upload errors when cookbook contents is owned by very large uid or gid #1806

supermarket upload errors when cookbook contents is owned by very large uid or gid #1806

Comments

Stromweld commented Apr 24, 2019

Description

ChefDK Version

Platform Version

Replication Case

Stacktrace

Stromweld commented Apr 24, 2019

robbkidd commented May 8, 2019

robbkidd commented May 8, 2019

Supermarket

Chef Workstation

Platform

Stromweld commented May 9, 2019

Stromweld commented May 9, 2019

Stromweld commented May 9, 2019

robbkidd commented May 9, 2019 • edited Loading

robbkidd commented May 13, 2019 • edited Loading

robbkidd commented May 15, 2019 • edited Loading

Stromweld commented May 15, 2019

robbkidd commented May 15, 2019

robbkidd commented May 9, 2019 •

edited

Loading

robbkidd commented May 13, 2019 •

edited

Loading

robbkidd commented May 15, 2019 •

edited

Loading