Image refresh for rhel-7-9

[cockpituous]

Image refresh for rhel-7-9

• image-refresh rhel-7-9
• Fix npm CA: ansible: Move npm CA file back to env variable cockpituous#610

[cockpituous]

image-refresh rhel-7-9 done: https://github.com/cockpit-project/bots/commits/image-refresh-rhel-7-9-20240328-231545

[cockpituous]

Success. Log: https://cockpit-logs.us-east-1.linodeobjects.com/image-refresh-rhel-7-9-98723b79-20240328-223736/log.html

[martinpitt]

Repeated failure, needs work:

npm ERR! request to https://codeload.github.com/kubernetes-ui/object-describer/tar.gz/9a44712c75cacec0d328e40f9e1328a668e5c0ef failed, reason: unable to get local issuer certificate

[martinpitt]

When I run npm i on the rhel-7.9 branch on my laptop, it fails completely differently:

npm ERR! Could not resolve dependency:
npm ERR! @patternfly/react-console@"1.10.46" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: react@16.8.6
npm ERR! node_modules/react
npm ERR!   peer react@"^16.3.1" from @patternfly/react-console@1.10.46
npm ERR!   node_modules/@patternfly/react-console
npm ERR!     @patternfly/react-console@"1.10.46" from the root project

OTOH this doesn't really look like a problem in our infra or the RH mirror. Our other projects like c-ostree etc. work just fine.

Running this works both locally and on our infra:

curl -o /tmp/x.tar https://codeload.github.com/kubernetes-ui/object-describer/tar.gz/9a44712c75cacec0d328e40f9e1328a668e5c0ef

And running npm install on rhos-01-20 does work. I'll retry this test, I can't see what's wrong. Perhaps that outage got fixed since yesterday?

[martinpitt]

Oops, this isn't what actually happens, but:

npm prune --legacy-peer-deps
npm install --legacy-peer-deps # see package.json

... but this still succeedss right now on rhos-01-20 on an interactive terminal, and localy. But in the retry it failed again, so perhaps that's some subtle difference between job-runner and the persistent tasks containers?

I ran this on rhos-01-20:

./job-runner --debug run --sha fb1da1ecc9673698a929f8d9bbb270b47eb7504e cockpit-project/cockpit ./autogen.sh

which fails in the same way.

I see three options:

1. Get an interactive shell in an ephemeral job-runner container and debug what the heck is going on
2. Use the cache tarball from https://github.com/cockpit-project/cockpit/releases/tag/195.12 instead of npm install -- it's a known-good snapshot.
3. Rip out pkg/kubernetes from the branch, it has been unsupported for a long time, and commifb1da1ecc9673698a929f8d9bbb270b47eb7504e already dropped the tests.

[martinpitt]

For (1), this reproduces an interactive shell:

podman-remote --url=unix:///podman.sock run --init -it --rm --device=/dev/kvm --memory=24g --pids-limit=16384 --shm-size=1024m --tmpfs=/tmp:size=14g --env=TEST_OVERLAY_DIR=/tmp --env=COCKPIT_IMAGES_DATA_DIR=/cache/images --volume=/var/cache/cockpit-tasks/images:/cache/images:rw --env=COCKPIT_IMAGE_STORES_FILE=/config/image-stores --volume=/var/cache/cockpit-tasks/image-stores:/config/image-stores:ro --volume=/var/lib/cockpit-secrets/tasks/npm-registry.crt:/run/secrets/tasks/npm-registry.crt:ro --volume=/etc/npmrc:/etc/npmrc:ro  ghcr.io/cockpit-project/tasks:latest bash

in that, the direct curl command works fine, but trying to check out and build cockpit reproduces the issue (but takes like 15 minutes, due to hanging a lot 😢 ). To break this down, I created an empty directory and ran

npm install 'git+https://github.com/kubernetes-ui/object-describer.git#v1.0.4'

This unfortunately still takes > 5 minutes, presumably due to long/many retries. Reducing fetch-retries and friends in /etc/npmrc helps enormously. I have a feeling that it's the cafile= which somehow breaks fetching from github -- does it only use that CA then, instead of in addition to the usual system one!? But strace shows that it does read /etc/pki/tls/certs/ca-bundle.crt.

So this smells a lot like fallout from cockpit-project/cockpituous@dbd7643

[martinpitt]

Fix in cockpit-project/cockpituous#610