[19.03 backport] bump hcsshim to fix docker build failing on Windows 1903 #429

thaJeztah · 2019-12-03T15:23:50Z

includes:

update containerd/cgroups to c4b9ac5c7601384c965b9646fc515884e091ebb9 (taken from bump containerd and dependencies to v1.3.0 moby/moby#39713 bump containerd and dependencies to v1.3.0)
bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a moby/moby#40154 bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a
Rename "v1" to "statsV1" moby/moby#40167 Rename "v1" to "statsV1"
integration-cli: remove unnescessary conversions (unconvert) (taken from Replace gometalinter with golangci-lint moby/moby#39668 Replace gometalinter with golangci-lint)
bump Microsoft/hcsshim 2226e083fc390003ae5aa8325c3c92789afa0e7a (taken from Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion() moby/moby#39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion())
Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2 moby/moby#40250 Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2

hcsshim bump includes:

wclayer: Work around Windows bug when expanding sandbox size microsoft/hcsshim#718 wclayer: Work around Windows bug when expanding sandbox size
- fixes Windows Host Compute Service bug breaks docker (and other) sandboxes bigger than 20G on Windows 1903 microsoft/hcsshim#708 Windows Host Compute Service bug breaks docker (and other) sandboxes bigger than 20G on Windows 1903
- fixes https://github.com/microsoft/hcsshim/issues/624The hcsshim on Windows 10 1903 always fails to build Docker image
- fixes/addresses An error occurred while attempting to build Docker image docker/for-win#3884 An error occurred while attempting to build Docker image (especially this comment and the next comments after: An error occurred while attempting to build Docker image docker/for-win#3884 (comment))
- fixes/addresses Windows 1903 fails when storage-opt used docker/for-win#4100 Windows 1903 fails when storage-opt used
- fixes hcsshim::PrepareLayer failed in Win32: The parameter is incorrect moby/moby#36831 hcsshim::PrepareLayer failed in Win32: The parameter is incorrect ( hcsshim::PrepareLayer failed in Win32: The parameter is incorrect moby/moby#36831 (comment))
- fixes Docker won't build container. Stannieman/audacity-with-asio-builder#5 Docker won't build container
- fixes https://github.com/MicrosoftDocs/visualstudio-docs/issues/3523 Error when running build with storage-opts set
- fixes Docker build windows 19.03 --storage-opt size>20G moby/moby#39524 Docker build windows 19.03 --storage-opt size>20G

Note that this is a temporary workaround for a bug in the platform, and will be reverted once that is addressed:

Revert #718 when Windows 19H1 has expand sandbox fix microsoft/hcsshim#721 Revert 718 when Windows 19H1 has expand sandbox fix

# taken from https://github.com/moby/moby/pull/39713 bump containerd and dependencies to v1.3.0
git cherry-pick -s -S -x 0af1099a81861dd0269adad53bdfb387b5c78f39

# https://github.com/moby/moby/pull/40154 bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a
git cherry-pick -s -S -x 27552ceb15bca544820229e574427d4c1d6ef585

# https://github.com/moby/moby/pull/40167 Rename "v1" to "statsV1"
git cherry-pick -s -S -x 9a7e96b5b7e97e034ce7bb0f1e7788d1bd881c7f

# taken from https://github.com/moby/moby/pull/39668 Replace gometalinter with golangci-lint
git cherry-pick -s -S -x 7c40c0a9227089a7e3ee7c23c2bc0685ed133391

# taken from https://github.com/moby/moby/pull/39100 Use Microsoft/hcsshim constants and deprecate pkg/system.GetOsVersion(
git cherry-pick -s -S -x a5341aaf32d2ea1d7f527fa8f3bd08f4706e4872

# https://github.com/moby/moby/pull/40250 Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2
git cherry-pick -s -S -x c530c9cbb0da177337a90a6651305daa9eb0c42b \
    a7b6c3f0bf5d10c6227a29bac7dd46b9a7a779bc

Minor conflicts in 27552ce:

diff --cc libcontainerd/types/types_linux.go
index 0a2daf5777,8cc41052b3..0000000000
--- a/libcontainerd/types/types_linux.go
+++ b/libcontainerd/types/types_linux.go
@@@ -3,8 -3,8 +3,13 @@@ package types // import "github.com/doc
  import (
        "time"

++<<<<<<< HEAD
 +      "github.com/containerd/cgroups"
 +      "github.com/opencontainers/runtime-spec/specs-go"
++=======
+       v1 "github.com/containerd/cgroups/stats/v1"
+       specs "github.com/opencontainers/runtime-spec/specs-go"
++>>>>>>> 27552ceb15... bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a
  )

  // Summary is not used on linux
diff --cc vendor.conf
index dd18920ab2,0be5d884ca..0000000000
--- a/vendor.conf
+++ b/vendor.conf
@@@ -117,10 -117,10 +117,17 @@@ github.com/googleapis/gax-g
  google.golang.org/genproto                          694d95ba50e67b2e363f3483057db5d4910c18f9

  # containerd
++<<<<<<< HEAD
 +github.com/containerd/containerd                    7c1e88399ec0b0b077121d9d5ad97e647b11c870
 +github.com/containerd/fifo                          a9fb20d87448d386e6d50b1f2e1fa70dcf0de43c
 +github.com/containerd/continuity                    aaeac12a7ffcd198ae25440a9dff125c2e2703a7
 +github.com/containerd/cgroups                       c4b9ac5c7601384c965b9646fc515884e091ebb9
++=======
+ github.com/containerd/containerd                    36cf5b690dcc00ff0f34ff7799209050c3d0c59a # v1.3.0
+ github.com/containerd/fifo                          bda0ff6ed73c67bfb5e62bc9c697f146b7fd7f13
+ github.com/containerd/continuity                    f2a389ac0a02ce21c09edd7344677a601970f41c
+ github.com/containerd/cgroups                       5fbad35c2a7e855762d3c60f2e474ffcad0d470a
++>>>>>>> 27552ceb15... bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a
  github.com/containerd/console                       0650fd9eeb50bab4fc99dceb9f2e14cf58f36e7f
  github.com/containerd/go-runc                       e029b79d8cda8374981c64eba71f28ec38e5526f
  github.com/containerd/typeurl                       2a93cfde8c20b23de8eb84a5adbc234ddf7a9e8d

full diff: github.com/containerd/cgroups containerd/cgroups@4994991...c4b9ac5 changes included: - containerd/cgroups#81 Add network stats - addresses containerd/cgroups#80 Add network metrics - containerd/cgroups#85 Fix cgroup hugetlb size prefix for kB - addresses kubernetes/kubernetes#77169 Permission denied on hugetlb due to wrong filename - relates to opencontainers/runc#2065 Fix cgroup hugetlb size prefix for kB - containerd/cgroups#88 cgroups: fix MoveTo function fail problem - containerd/cgroups#92 fixed an issue with invalid soft memory limits - containerd/cgroups#93 avoid adding io_serviced and io_service_bytes duplicately - fixes containerd/containerd#3412 collected metric container_blkio_io_serviced_recursive_total: was collected before with the same name and label values Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 0af1099) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

full diff: containerd/cgroups@c4b9ac5...5fbad35 - containerd/cgroups#82 Add go module support - containerd/cgroups#96 Move metrics proto package to stats/v1 - containerd/cgroups#97 Allow overriding the default /proc folder in blkioController - containerd/cgroups#98 Allows ignoring memory modules - containerd/cgroups#99 Add Go 1.13 to Travis - containerd/cgroups#100 stats/v1: export per-cgroup stats Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 27552ce) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

follow-up to 27552ce, where this was left as a review comment, but the PR was already merged. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 9a7e96b) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 7c40c0a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Adds osversion.Build() utility Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit a5341aa) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Fixed failures in TestRunAttachFailedNoLeak caused by case mismatch Signed-off-by: vikrambirsingh <vikrambir.singh@docker.com> (cherry picked from commit c530c9c) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Among other things, this is required to pull in microsoft/hcsshim#718 Also fixes microsoft/hcsshim#737 which was caught by checks while attempting to bump up hcsshim version. Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com> (cherry picked from commit a7b6c3f) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

ebalders · 2019-12-17T14:26:47Z

Is there an ETA on 19.03.6?

full diff: https://2226e083fc390003ae5aa8325c3c92789afa0e7a...b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2 includes: - microsoft/hcsshim#718 wclayer: Work around Windows bug when expanding sandbox size - fixes microsoft/hcsshim#708 Windows Host Compute Service bug breaks docker (and other) sandboxes bigger than 20G on Windows 1903 - fixes microsoft/hcsshim#624The hcsshim on Windows 10 1903 always fails to build Docker image - fixes/addresses docker/for-win#3884 An error occurred while attempting to build Docker image (especially this comment and the next comments after: docker/for-win#3884 (comment)) - fixes/addresses docker/for-win#4100 Windows 1903 fails when storage-opt used - fixes moby/moby#36831 hcsshim::PrepareLayer failed in Win32: The parameter is incorrect (moby/moby#36831 (comment)) - fixes Stannieman/audacity-with-asio-builder#5 Docker won't build container - fixes MicrosoftDocs/visualstudio-docs#3523 Error when running build with storage-opts set - fixes moby/moby#39524 Docker build windows 19.03 --storage-opt size>20G Note that this is a temporary workaround for a bug in the platform, and will be reverted once that is addressed: - microsoft/hcsshim#721 Revert 718 when Windows 19H1 has expand sandbox fix Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

ViRb3 · 2020-01-08T00:43:58Z

Any update on releasing this? Quite a major blocker for Windows users.

cpuguy83

LGTM at least from the Linux side (since cgroups package was updated).

jazpearson · 2020-04-21T08:12:03Z

Do we have any idea when microsoft will publish an update for Windows Server 2019 users? We are still stuck on 19.03.5.

thaJeztah and others added 7 commits December 3, 2019 15:58

Rename "v1" to "statsV1"

4d190af

follow-up to 27552ce, where this was left as a review comment, but the PR was already merged. Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 9a7e96b) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

integration-cli: remove unnescessary conversions (unconvert)

3fca587

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit 7c40c0a) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

bump Microsoft/hcsshim 2226e083fc390003ae5aa8325c3c92789afa0e7a

1f18c73

Adds osversion.Build() utility Signed-off-by: Sebastiaan van Stijn <github@gone.nl> (cherry picked from commit a5341aa) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

TestRunAttachFailedNoLeak: Compare lowercase

5302429

Fixed failures in TestRunAttachFailedNoLeak caused by case mismatch Signed-off-by: vikrambirsingh <vikrambir.singh@docker.com> (cherry picked from commit c530c9c) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah added the impact/changelog label Dec 3, 2019

thaJeztah added this to the 19.03.6 milestone Dec 3, 2019

PatrickLang mentioned this pull request Dec 19, 2019

Enable configuration of C: drive size for pods kubernetes/kubernetes#83450

Closed

cpuguy83 approved these changes Jan 23, 2020

View reviewed changes

thaJeztah merged commit 6949793 into docker-archive:19.03 Jan 23, 2020

thaJeztah deleted the 19.03_backport_windows_1903_fixes branch January 23, 2020 19:48

This was referenced Feb 1, 2020

ue4-docker build 4.22.2 fails with 'out of space' with new-enough Docker EE adamrehn/ue4-docker#44

Closed

Windows 1903 fails when storage-opt used docker/for-win#4100

Closed

TBBle mentioned this pull request Apr 21, 2020

Docker fails to create Windows Containers with a file larger than 8gB moby/moby#40444

Closed

TBBle mentioned this pull request May 10, 2020

Unblacklist Windows 1903/1909 (with new-enough Docker Engine) adamrehn/ue4-docker#81

Closed

TBBle mentioned this pull request Sep 27, 2020

Error building 4.25.3 windows server 2019 "Not enough space on disk" adamrehn/ue4-docker#99

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[19.03 backport] bump hcsshim to fix docker build failing on Windows 1903 #429

[19.03 backport] bump hcsshim to fix docker build failing on Windows 1903 #429

thaJeztah commented Dec 3, 2019

ebalders commented Dec 17, 2019

ViRb3 commented Jan 8, 2020

cpuguy83 left a comment

jazpearson commented Apr 21, 2020

[19.03 backport] bump hcsshim to fix docker build failing on Windows 1903 #429

[19.03 backport] bump hcsshim to fix docker build failing on Windows 1903 #429

Conversation

thaJeztah commented Dec 3, 2019

ebalders commented Dec 17, 2019

ViRb3 commented Jan 8, 2020

cpuguy83 left a comment

Choose a reason for hiding this comment

jazpearson commented Apr 21, 2020