Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude nodes from external load balancers in Kubernetes 1.19 #72

Conversation

sarahhodne
Copy link
Contributor

Starting in Kubernetes 1.19, unschedulable nodes were included in load balancers again (see kubernetes/kubernetes#90823), which causes a conflict between the cloud controller and lifecycle-manager, where lifecycle-manager attempts to remove the instance from the load balancer, but then the cloud controller attempts to re-add it.

Kubernetes 1.19 also moved the ServiceNodeExclusion feature gate to beta support, and defaults it to enabled, so you can set the node.kubernetes.io/exclude-from-external-load-balancers label on the node to have the node be excluded. An older version of this feature used the alpha.service-controller.kubernetes.io/exclude-balancer, but this label is removed in Kubernetes 1.19.

This commit sets the new node.kubernetes.io/exclude-from-external-load-balancers label in addition to the older alpha label so lifecycle-manager continues to work on both old and new clusters.

@sarahhodne sarahhodne requested a review from a team as a code owner June 23, 2021 13:57
@sarahhodne
Copy link
Contributor Author

CLA assistant was confusing, "Sign in" apparently means you sign the CLA immediately. I did not mean to sign it yet, since I believe I need to have it signed by my workplace, so please hold off on this for now.

@eytan-avisror
Copy link
Collaborator

Thanks @sarahhodne
We dont use CLA anymore actually, we need a DCO - so you just need to sign your commits accordingly
See here: https://github.com/keikoproj/keiko#news

I will remove the CLA assistant from this repo (I guess we left it)

@keikoproj keikoproj deleted a comment from CLAassistant Jun 25, 2021
@eytan-avisror
Copy link
Collaborator

Hey @sarahhodne looks like you may need to recreate this PR since it's out-of-date with base (not sure why it's not allowing a rebase on the PR).

We'd ideally like to merge this soon, let me know if you can use the DCO instead of CLA

@sarahhodne
Copy link
Contributor Author

I'm just checking with legal at $work regarding the DCO, I think that should be fine but I just want their sign-off first. Once I have that, I'll rebase or recreate this PR with the sign-off in the commit.

@eytan-avisror
Copy link
Collaborator

I'm just checking with legal at $work regarding the DCO, I think that should be fine but I just want their sign-off first. Once I have that, I'll rebase or recreate this PR with the sign-off in the commit.

OK, sounds good, thanks!

Starting in Kubernetes 1.19, unschedulable nodes were included in load
balancers again (see kubernetes/kubernetes#90823), which causes a
conflict between the cloud controller and lifecycle-manager, where
lifecycle-manager attempts to remove the instance from the load
balancer, but then the cloud controller attempts to re-add it.

Kubernetes 1.19 also moved the ServiceNodeExclusion feature gate to beta
support, and defaults it to enabled, so you can set the
node.kubernetes.io/exclude-from-external-load-balancers label on the
node to have the node be excluded. An older version of this feature used
the alpha.service-controller.kubernetes.io/exclude-balancer, but this
label is removed in Kubernetes 1.19.

This commit sets the new
node.kubernetes.io/exclude-from-external-load-balancers label in
addition to the older alpha label so lifecycle-manager continues to work
on both old and new clusters.

Signed-off-by: Sarah Hodne <sarah@circleci.com>
@sarahhodne sarahhodne force-pushed the sarah/service-node-exclusion-support branch from b82c3ef to a994368 Compare July 1, 2021 09:09
@sarahhodne
Copy link
Contributor Author

@eytan-avisror Added the DCO signoff to my commit and rebased against latest master so I think this should be ready for a review now.

Copy link
Collaborator

@eytan-avisror eytan-avisror left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

@codecov
Copy link

codecov bot commented Jul 1, 2021

Codecov Report

Merging #72 (a994368) into master (d2e9027) will decrease coverage by 0.11%.
The diff coverage is 33.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #72      +/-   ##
==========================================
- Coverage   69.81%   69.69%   -0.12%     
==========================================
  Files          12       12              
  Lines         911      914       +3     
==========================================
+ Hits          636      637       +1     
- Misses        215      216       +1     
- Partials       60       61       +1     
Impacted Files Coverage Δ
pkg/service/server.go 59.01% <33.33%> (-0.22%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2e9027...a994368. Read the comment docs.

@eytan-avisror eytan-avisror merged commit 9558000 into keikoproj:master Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants