Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestStartStop: "Failed to setup certs" flake #7541

Closed
medyagh opened this issue Apr 9, 2020 · 4 comments
Closed

TestStartStop: "Failed to setup certs" flake #7541

medyagh opened this issue Apr 9, 2020 · 4 comments
Assignees
@tstromberg
Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone
v1.10.0

Comments

@medyagh
Copy link
Member

medyagh commented Apr 9, 2020

as seen here
https://storage.googleapis.com/minikube-builds/logs/7435/70adc95/Docker_Linux.html#fail_TestStartStop%2fgroup%2fold-docker

	W0408 20:09:24.196997    4909 exit.go:101] Failed to setup certs: certificate symlinks: create symlink for /usr/share/ca-certificates/4970.pem: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4970.pem && ln -fs /usr/share/ca-certificates/4970.pem /etc/ssl/certs/4970.pem": exit status 1
	stdout:
	
	stderr:
	* 
	X Failed to setup certs: certificate symlinks: create symlink for /usr/share/ca-certificates/4970.pem: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4970.pem && ln -fs /usr/share/ca-certificates/4970.pem /etc/ssl/certs/4970.pem": exit status 1
	stdout:
	
	stderr:
	
	* 
	* minikube is exiting due to an error. If the above message is not useful, open an issue:
@medyagh medyagh added this to the v1.10.0 milestone Apr 9, 2020
@medyagh
Copy link
Member Author

medyagh commented Apr 9, 2020

CC: @tstromberg

@tstromberg
Copy link
Contributor 

	I0408 20:09:15.512190    4909 certs.go:342] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-7435-2451-70adc95be27abfd1f53736cd23f13206bf501118/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-7435-2451-70adc95be27abfd1f53736cd23f13206bf501118/.minikube/certs/4970.pem (1338 bytes)
	W0408 20:09:15.512248    4909 certs.go:338] ignoring /home/jenkins/minikube-integration/linux-amd64-docker-7435-2451-70adc95be27abfd1f53736cd23f13206bf501118/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-7435-2451-70adc95be27abfd1f53736cd23f13206bf501118/.minikube/certs/4970_empty.pem, impossibly tiny 0 bytes
...
	I0408 20:09:22.527884    4909 certs.go:120] copying: /usr/share/ca-certificates/4970.pem
	I0408 20:09:23.815705    4909 kic_runner.go:91] Run: openssl version
	I0408 20:09:23.983664    4909 kic_runner.go:91] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4970.pem && ln -fs /usr/share/ca-certificates/4970.pem /etc/ssl/certs/4970.pem"
...
	I0408 20:09:23.983664    4909 kic_runner.go:91] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4970.pem && ln -fs /usr/share/ca-certificates/4970.pem /etc/ssl/certs/4970.pem"

	W0408 20:09:24.196997    4909 exit.go:101] Failed to setup certs: certificate symlinks: create symlink for /usr/share/ca-certificates/4970.pem: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4970.pem && ln -fs /usr/share/ca-certificates/4970.pem /etc/ssl/certs/4970.pem": exit status 1

@tstromberg tstromberg self-assigned this Apr 9, 2020
@tstromberg tstromberg mentioned this issue Apr 9, 2020
Merged
This was referenced Apr 10, 2020
Merged
Merged
@tstromberg
Copy link
Contributor

tstromberg commented Apr 10, 2020 •
edited
Loading

Great news! We now have a more interesting error message:

W0410 04:25:13.826351 26334 out.go:201] Failed to setup certs: Copy /home/jenkins/minikube-integration/linux-amd64-docker-7580-2643-10f924df4a5d1a7a5312424fb220970b28adf485/.minikube/certs/5144.pem: write: /tmp/tmpf-memory-asset852732570: expected to write 0 bytes, but wrote 1338 instead

Oddly though, we found it as a 1338 byte file:

I0410 04:25:05.989011 26334 certs.go:341] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-7580-2643-10f924df4a5d1a7a5312424fb220970b28adf485/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-7580-2643-10f924df4a5d1a7a5312424fb220970b28adf485/.minikube/certs/5144.pem (1338 bytes

https://storage.googleapis.com/minikube-builds/logs/7580/10f924d/Docker_Linux.html#fail_TestStartStop%2fgroup%2fembed-certs

This sounds super racey.

@tstromberg tstromberg changed the title TestStartStop: "Failed to setup certs: certificate symlinks: create symlink" TestStartStop: "Failed to setup certs" flake Apr 10, 2020
This was referenced Apr 10, 2020
Closed
Merged
@sharifelgamal sharifelgamal added kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. labels Apr 15, 2020
@priyawadhwa priyawadhwa mentioned this issue Apr 17, 2020
Merged
@priyawadhwa
Copy link

Can be closed via #7723

@furkatgofurov7 furkatgofurov7 mentioned this issue Jul 29, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tstromberg tstromberg

Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Milestone
v1.10.0
Development

No branches or pull requests
4 participants
@tstromberg @medyagh @priyawadhwa @sharifelgamal