-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cilium v1.14.3, kubernetes v1.28.2 Endpoint unreachability for only one node #28984
Comments
I'm experiencing something similar Kubernetes: Cilium pod log output
Connectivity test
|
@dalekurt , whats the output of cilium-health status from cilium agent pod ? |
@Rammurthy5 This is the output from a cilium pod
|
Not sure if this is 100% of your issue, but cilium/install/kubernetes/cilium/values.yaml.tmpl Line 1712 in 71908e7
|
Hey @Rammurthy5, it seems that you hit #25804. TL;DR; you need to cleanup stale AWS iptables chains on your nodes: cilium/Documentation/installation/k8s-install-helm.rst Lines 210 to 217 in b2ff077
@dalekurt According to the logs, you configured Cilium in direct routing mode using
This is not possible. The error is also not related to clustermesh (and the original report), as all nodes in the health status report belong to the same cluster. I'm closing this issue as #25804 has been fixed (#29448 is also following up on another related issue). |
Is there an existing issue for this?
What happened?
I have 2 EKS clusters setup in two different regions, and they are VPC peered.
Both clusters have 2 nodes each, running cilium v1.14.3, kubernetes v1.28.2.
CoreDNS pods are running, restarted them - did not help.
clustermesh status
andcilium status
on both clusters seem perfectly alright.On cluster1, all 4 endpoints are reachable. But on cluster2 only 3 endpoints are reachable. The unreachable endpoint is from one of the two nodes in other region (1 node endpoint from other region is reachable).
I had tried this with cilium v1.14.2 on kubernetes v1.28, v1.27.
cilium v1.14.3 on kubernetes v1.27, v1.28.
cluster2
cluster1
cilium-health status on cluster2 cilium agent:
Cilium Version
1.14.3
Kernel Version
aarch64 aarch64 aarch64 GNU/Linux
Kubernetes Version
1.28.2
Sysdump
cluster2 sysdump
cilium-sysdump-20231104-232847.zip
cluster1 sysdump
cilium-sysdump-20231104-233044.zip
Relevant log output
No response
Anything else?
Helm Command used to install cilium
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: