fix containerd ImageExists to look for image name and image id sha

[prezha]

pls see details in the comment: #17658 (comment)

this pr addresses the problem of (how we handle/expect) the ctr output - from logs:

I1126 22:25:11.544084   26459 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.28.4 registry.k8s.io/kube-controller-manager:v1.28.4 registry.k8s.io/kube-scheduler:v1.28.4 registry.k8s.io/kube-proxy:v1.28.4 registry.k8s.io/pause:3.9 registry.k8s.io/etcd:3.5.9-0 registry.k8s.io/coredns/coredns:v1.10.1 gcr.io/k8s-minikube/storage-provisioner:v5]
...
I1126 22:25:11.544147   26459 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.28.4
...
I1126 22:25:11.547965   26459 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.28.4: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.28.4
...
I1126 22:25:11.963349   26459 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/kube-proxy:v1.28.4"
...
I1126 22:25:12.937500   26459 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.28.4" needs transfer: "registry.k8s.io/kube-proxy:v1.28.4" does not exist at hash "83f6cc407eed88d214aad97f3539bde5c8e485ff14424cd021a3a2899304398e" in container runtime
...
I1126 22:25:12.937543   26459 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.28.4
...
I1126 22:25:13.509165   26459 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.28.4
...
I1126 22:25:14.276116   26459 cache_images.go:286] Loading image from: /home/prezha/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.28.4
...
W1126 22:25:14.361961   26459 out.go:239] ❌  Unable to load cached images: loading cached images: stat /home/prezha/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.28.4: no such file or directory
❌  Unable to load cached images: loading cached images: stat /home/prezha/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.28.4: no such file or directory
...

so, the above underlying error:

I1126 22:25:12.937500 26459 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.28.4" needs transfer: "registry.k8s.io/kube-proxy:v1.28.4" does not exist at hash "83f6cc407eed88d214aad97f3539bde5c8e485ff14424cd021a3a2899304398e" in container runtime

comes from this line, and that effectively come from this block

looking at the crt output, we have:

$ sudo ctr -n=k8s.io images list
REF                                                                                                             TYPE                                                      DIGEST                                                                  SIZE      PLATFORMS                                                                     LABELS
...
registry.k8s.io/kube-proxy:v1.28.4                                                                              application/vnd.docker.distribution.manifest.list.v2+json sha256:e63408a0f5068a7e9d4b34fd72b4a2b0e5512509b53cd2123a37fc991b0ef532 23.4 MiB  linux/amd64,linux/arm64,linux/ppc64le,linux/s390x                             io.cri-containerd.image=managed
...
registry.k8s.io/kube-proxy@sha256:e63408a0f5068a7e9d4b34fd72b4a2b0e5512509b53cd2123a37fc991b0ef532              application/vnd.docker.distribution.manifest.list.v2+json sha256:e63408a0f5068a7e9d4b34fd72b4a2b0e5512509b53cd2123a37fc991b0ef532 23.4 MiB  linux/amd64,linux/arm64,linux/ppc64le,linux/s390x                             io.cri-containerd.image=managed
...
sha256:83f6cc407eed88d214aad97f3539bde5c8e485ff14424cd021a3a2899304398e                                         application/vnd.docker.distribution.manifest.list.v2+json sha256:e63408a0f5068a7e9d4b34fd72b4a2b0e5512509b53cd2123a37fc991b0ef532 23.4 MiB  linux/amd64,linux/arm64,linux/ppc64le,linux/s390x                             io.cri-containerd.image=managed
...

which has image ref and image digest sha in each line, whereas we want image ref and image id sha on the same line, and hence that fails (as there's image id on a separate "line")

to prove the point, cricrl gives both image ref and id on the same line (from the same minikube cluster):

$ sudo crictl images --no-trunc --digests
IMAGE                                     TAG                  DIGEST                                                                    IMAGE ID                                                                  SIZE
...
registry.k8s.io/kube-proxy                v1.28.4              sha256:e63408a0f5068a7e9d4b34fd72b4a2b0e5512509b53cd2123a37fc991b0ef532   sha256:83f6cc407eed88d214aad97f3539bde5c8e485ff14424cd021a3a2899304398e   24.6MB
...

test:

$ time minikube start --driver kvm --container-runtime containerd --alsologtostderr -v=7

before:

real    0m58.370s
user    0m1.498s
sys     0m0.620s

after:

real    0m33.686s
user    0m1.356s
sys     0m0.404s

test output:

I1127 23:06:49.993578  115508 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.28.4 registry.k8s.io/kube-controller-manager:v1.28.4 registry.k8s.io/kube-scheduler:v1.28.4 registry.k8s.io/kube-proxy:v1.28.4 registry.k8s.io/pause:3.9 registry.k8s.io/etcd:3.5.9-0 registry.k8s.io/coredns/coredns:v1.10.1 gcr.io/k8s-minikube/storage-provisioner:v5]
...
I1127 23:06:49.993720  115508 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.28.4
...
I1127 23:06:49.994852  115508 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.28.4: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.28.4
...
I1127 23:06:50.401707  115508 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images check
...
I1127 23:06:50.906378  115508 cache_images.go:123] Successfully loaded all cached images
...

[prezha]

/ok-to-test

[minikube-pr-bot]

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 17671) |
+----------------+----------+---------------------+
| minikube start | 51.7s    | 51.6s               |
| enable ingress | 28.2s    | 28.4s               |
+----------------+----------+---------------------+

Times for minikube (PR 17671) start: 52.3s 50.5s 52.3s 52.3s 50.5s
Times for minikube start: 49.8s 52.2s 50.6s 52.4s 53.4s

Times for minikube ingress: 28.7s 27.8s 28.3s 28.4s 27.8s
Times for minikube (PR 17671) ingress: 27.8s 27.9s 27.8s 28.3s 30.3s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 17671) |
+----------------+----------+---------------------+
| minikube start | 24.5s    | 24.2s               |
| enable ingress | 20.5s    | 21.3s               |
+----------------+----------+---------------------+

Times for minikube ingress: 20.5s 20.4s 19.9s 20.9s 21.0s
Times for minikube (PR 17671) ingress: 18.9s 21.4s 20.4s 24.5s 21.5s

Times for minikube start: 25.6s 24.2s 23.8s 22.1s 26.7s
Times for minikube (PR 17671) start: 22.5s 25.3s 25.4s 24.5s 23.5s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 17671) |
+----------------+----------+---------------------+
| minikube start | 23.1s    | 22.7s               |
| enable ingress | 28.9s    | 31.6s               |
+----------------+----------+---------------------+

Times for minikube ingress: 18.3s 32.3s 31.4s 32.4s 30.4s
Times for minikube (PR 17671) ingress: 31.4s 31.4s 32.3s 31.4s 31.3s

Times for minikube start: 24.3s 21.3s 23.0s 24.7s 22.2s
Times for minikube (PR 17671) start: 24.1s 22.2s 21.3s 21.0s 24.7s

[minikube-pr-bot]

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
KVM_Linux	TestScheduledStopUnix (gopogh)	0.00 (chart)
KVM_Linux_crio	TestStartStop/group/newest-cni/serial/EnableAddonAfterStop (gopogh)	3.16 (chart)
KVM_Linux_crio	TestStartStop/group/newest-cni/serial/Stop (gopogh)	3.16 (chart)
Docker_Linux_crio_arm64	TestPause/serial/SecondStartNoReconfiguration (gopogh)	14.91 (chart)

To see the flake rates of all tests by environment, click here.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, prezha

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [medyagh,prezha]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment