Skip to content

kube ovn join子网CIDR修改

Jump to bottom
oilbeater edited this page Jun 27, 2022 · 2 revisions

Wiki 下的中文文档将不在维护，请访问我们最新的中文文档网站，获取最新的文档更新。

背景

kube-ovn安装会默认创建两个子网，ovn-default和join子网，分别用于给Pod和Node分配IP地址。

默认子网信息如下

mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl get subnet
NAME          PROVIDER   VPC           PROTOCOL   CIDR            PRIVATE   NAT     DEFAULT   GATEWAYTYPE   V4USED   V4AVAILABLE   V6USED   V6AVAILABLE
join          ovn        ovn-cluster   IPv4       100.64.0.0/16   false     false   false     distributed   3        65530         0        0
ovn-default   ovn        ovn-cluster   IPv4       10.16.0.0/16    false     true    true      distributed   6        65527         0        0
mac@macdeMacBook-Pro-2 kube-ovn_1.0 %

如果安装后发现join子网CIDR与规划的子网CIDR冲突，需要变更join子网CIDR范围，可以按照以下步骤进行操作。

更改步骤

  1. 删除join子网

执行命令 kubectl edit subnet join, 编辑join子网信息，删除以下两行finalizers参数

  finalizers:
  - kube-ovn-controller

然后执行命令 kubectl delete subnet join，删除join子网。

mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl delete subnet join
subnet.kubeovn.io "join" deleted
mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl get subnet
kuNAME          PROVIDER   VPC           PROTOCOL   CIDR           PRIVATE   NAT    DEFAULT   GATEWAYTYPE   V4USED   V4AVAILABLE   V6USED   V6AVAILABLE
ovn-default   ovn        ovn-cluster   IPv4       10.16.0.0/16   false     true   true      distributed   6        65527         0        0
mac@macdeMacBook-Pro-2 kube-ovn_1.0 %
  1. join子网用于给Node分配IP，需要将所有Node标识为未申请IP状态。 执行命令kubectl annotate node ovn.kubernetes.io/allocated=false --all --overwrite
mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl annotate node ovn.kubernetes.io/allocated=false --all --overwrite
node/kube-ovn-control-plane annotated
node/kube-ovn-worker annotated
node/kube-ovn-worker2 annotated
mac@macdeMacBook-Pro-2 kube-ovn_1.0 %
  1. 执行命令 kubectl edit deployment -n kube-system kube-ovn-controller，编辑kube-ovn-controller deployment，指定node-switch-cidr参数的范围，这个参数对应join子网范围。
        args:
        - --node-switch-cidr=100.51.0.0/16

修改参数后，kube-ovn-controller pod会自动重启，pod重启后会进行子网重建。

mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl get subnet
NAME          PROVIDER   VPC           PROTOCOL   CIDR            PRIVATE   NAT     DEFAULT   GATEWAYTYPE   V4USED   V4AVAILABLE   V6USED   V6AVAILABLE
join          ovn        ovn-cluster   IPv4       100.51.0.0/16   false     false   false     distributed   3        65530         0        0
ovn-default   ovn        ovn-cluster   IPv4       10.16.0.0/16    false     true    true      distributed   6        65527         0        0
mac@macdeMacBook-Pro-2 kube-ovn_1.0 %
mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl get ips
NAME                                                         V4IP         V6IP   MAC                 NODE                     SUBNET
node-kube-ovn-control-plane                                  100.51.0.3          00:00:00:C3:A3:78   kube-ovn-control-plane   join
node-kube-ovn-worker                                         100.51.0.4          00:00:00:4A:62:3B   kube-ovn-worker          join
node-kube-ovn-worker2                                        100.51.0.2          00:00:00:F7:4A:BA   kube-ovn-worker2         join
mac@macdeMacBook-Pro-2 kube-ovn_1.0 %

Pod重建后，可以看到join子网的CIDR已经更新为修改后的取值范围。Node节点的IP地址，也变更为新子网的地址。

  1. 执行命令 kubectl delete pods -l app=kube-ovn-cni -n kube-system, 删除kube-ovn-cni的pod，pod重建过程会重新配置ovn0网卡的IP地址。
mac@macdeMacBook-Pro-2 kube-ovn_1.0 % kubectl delete pods -l app=kube-ovn-cni -n kube-system
pod "kube-ovn-cni-lphtv" deleted
pod "kube-ovn-cni-sh56h" deleted
pod "kube-ovn-cni-wnsjx" deleted
mac@macdeMacBook-Pro-2 kube-ovn_1.0 %
  1. 将以下代码放入脚本执行，重新添加Pod到节点的路由信息
for ns in $(kubectl get ns --no-headers -o  custom-columns=NAME:.metadata.name); do
  for pod in $(kubectl get pod --no-headers -n "$ns" --field-selector spec.restartPolicy=Always -o custom-columns=NAME:.metadata.name,HOST:spec.hostNetwork | awk '{if ($2!="true") print $1}'); do
    kubectl annotate pod "$pod" -n "$ns" ovn.kubernetes.io/routed=false --overwrite
  done
done
Clone this wiki locally