Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry to SSH connectivity check #5848

Merged
merged 1 commit into from
Nov 14, 2019

Conversation

hypnoglow
Copy link
Contributor

Added retry to the SSH connectivity check on "start", starting from 1 second and up to a total of 10 seconds of attempts. Also added the 3-second timeout to dialer.

Fixes #5834


  • Is a 10-second timeout enough for the connectivity check?
  • Is a 3-second timeout suitable for dialer timeout?

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 6, 2019
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@k8s-ci-robot
Copy link
Contributor

Welcome @hypnoglow!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot
Copy link
Contributor

Hi @hypnoglow. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Nov 6, 2019
@hypnoglow
Copy link
Contributor Author

/assign @medyagh

@codecov-io
Copy link

codecov-io commented Nov 6, 2019

Codecov Report

Merging #5848 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5848      +/-   ##
==========================================
- Coverage   36.49%   36.47%   -0.03%     
==========================================
  Files         110      110              
  Lines        8124     8129       +5     
==========================================
  Hits         2965     2965              
- Misses       4770     4775       +5     
  Partials      389      389
Impacted Files Coverage Δ
cmd/minikube/cmd/start.go 19.62% <0%> (-0.15%) ⬇️

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this quick PR, just minor knits.

cmd/minikube/cmd/start.go Outdated Show resolved Hide resolved
cmd/minikube/cmd/start.go Outdated Show resolved Hide resolved
@medyagh
Copy link
Member

medyagh commented Nov 6, 2019

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 6, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hypnoglow
To complete the pull request process, please assign medyagh
You can assign the PR to them by writing /assign @medyagh in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 183.309209 176.651170 200.426672]
All Times Minikube (PR 5848): [ 190.213247 174.681006 201.200174]

Average minikube: 186.795684
Average Minikube (PR 5848): 188.698142

Averages Time Per Log

+------------------------+-----------+--------------------+
|          LOG           | MINIKUBE  | MINIKUBE (PR 5848) |
+------------------------+-----------+--------------------+
| minikube v             |  0.311055 |           0.316201 |
| Creating kvm2          | 41.608139 |          41.841112 |
| Preparing Kubernetes   | 55.090812 |          55.018305 |
| Pulling images         |  2.856562 |           3.027828 |
| Launching Kubernetes   | 20.349554 |          21.566405 |
| Waiting for: apiserver | 66.579562 |          66.928291 |
+------------------------+-----------+--------------------+

cvila84 pushed a commit to cvila84/minikube that referenced this pull request Nov 13, 2019
@minikube-bot
Copy link
Collaborator

All Times minikube: [ 124.147198 123.410258 124.486849]
All Times Minikube (PR 5848): [ 179.051754 178.325035 178.028663]

Average minikube: 124.014768
Average Minikube (PR 5848): 178.468484

Averages Time Per Log

+------------------------+-----------+--------------------+
|          LOG           | MINIKUBE  | MINIKUBE (PR 5848) |
+------------------------+-----------+--------------------+
| minikube v             |  0.300992 |           0.318872 |
| Creating kvm2          | 38.178453 |          37.870412 |
| Preparing Kubernetes   | 53.927328 |          52.674853 |
| Pulling images         |  2.979979 |           2.863649 |
| Launching Kubernetes   | 20.023461 |          19.829712 |
| Waiting for: apiserver |           |          64.910986 |
+------------------------+-----------+--------------------+

@medyagh
Copy link
Member

medyagh commented Nov 13, 2019

/retest this please

@medyagh
Copy link
Member

medyagh commented Nov 13, 2019

@hypnoglow this PR seems to have significally added to the start time of minikube ! check #5848 (comment)

@medyagh
Copy link
Member

medyagh commented Nov 13, 2019

@hypnoglow I am curious, can we investigate if the Retry Logic, waits First the initial delay time, Before even starting ? or does it do the Wait after first try ?

(admitibly I refactored that part of code but I haven't had chance to verify it's behaviour)

@medyagh medyagh changed the title Retry on SSH connectivity check wip; Retry on SSH connectivity check Nov 13, 2019
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 13, 2019
@medyagh medyagh changed the title wip; Retry on SSH connectivity check wip: Retry on SSH connectivity check Nov 13, 2019
@hypnoglow
Copy link
Contributor Author

hypnoglow commented Nov 13, 2019

@hypnoglow this PR seems to have significally added to the start time of minikube ! check #5848 (comment)

How can we be sure that this PR is related? Preparing Kubernetes log time did not change, and this happens after network validation occurs:

Am I understanding this correctly?

BTW why there is no log time for Waiting for: apiserver in the comment?

I am curious, can we investigate if the Retry Logic, waits First the initial delay time, Before even starting ? or does it do the Wait after first try ?

I think retry logic is fine since the initial delay is for the first retry, not the first call.

@tstromberg tstromberg removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 14, 2019
@tstromberg
Copy link
Contributor

/ok-to-test

@tstromberg tstromberg changed the title wip: Retry on SSH connectivity check Add retry to SSH connectivity check Nov 14, 2019
Copy link
Contributor

@tstromberg tstromberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thank you!

@tstromberg
Copy link
Contributor

/ok-to-test

@hypnoglow
Copy link
Contributor Author

/retest

@tstromberg tstromberg merged commit 1c46502 into kubernetes:master Nov 14, 2019
@tstromberg
Copy link
Contributor

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add retry logic to minikub SSH connectivity check
6 participants