This repo serves as an example of how to setup and configure kubernetes for multicast support using the Intel Multus CNI plugin.
Ensure you have Vagrant installed, clone this repo and cd into the Vagrant directory. Run vagrant up
to bring up the VMs:
[ylb@spectric ~]$ cd k8s-mcast-example/Vagrant
[ylb@spectric Vagrant]$ vagrant up
You now have 2 nodes, k8s
and mcastsrc
. You can ssh to either via:
[ylb@spectric Vagrant]$ vagrant ssh <node name>
The k8s node is a single-node Kubernetes installation and the mcast host will act as our multicast source.
Multicast Outside of Kuberentes:
Verify multicast works outside of kubernetes first by using iperf
and the hosts network.
The following iperf command will subscribe to multicast packets for the IGMP group address
224.0.67.67
. We will do this on the k8s VM and send the multicast packets on the
mcastsrc VM
[ylb@spectric Vagrant]$ vagrant ssh k8s
[vagrant@k8s ~] docker run --net=host --rm bagoulla/iperf:2.0 -s -u -B 224.0.67.67%eth1 -i 1
...
From a new terminal, ssh into our mcastsrc VM and begin sending the mcast packets, you should see the successful reception in the other terminal.
[ylb@spectric Vagrant]$ vagrant ssh mcastsrc
[vagrant@mcastsrc ~]$ docker run --net=host --rm bagoulla/iperf:2.0 -c 224.0.67.67 -u --ttl 5 -t 60 -B 10.0.0.12
...
You can ctrl+c
to break out of iperf. We now want to mimic these results from within Kubernetes.
Multicast within Kuberentes:
From the k8s VM, apply the intel multus daemonset which will allow us to mount in specific host network interfaces:
[ylb@spectric Vagrant]$ vagrant ssh k8s
[vagrant@k8s ~]$ kubectl apply -f https://github.com/raw/intel/multus-cni/master/images/multus-daemonset.yml
We now must apply a NetworkAttachmentDefinition which tells multus which network interface we want to expose to our containers and how. We need to specify which interface on the parent to pass and an IP range to assign an addresses from. Many other optional parameters may be specified including a default gateway and custom routes; see the multus documentation for additional configuration options. Below is the NetworkAttachmentDefinition we will use to expose eth1, a copy has been placed in /vagrant
:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: eth1-multicast
spec:
config: '{
"cniVersion": "0.3.0",
"type": "macvlan",
"master": "eth1",
"mode": "bridge",
"ipam": {
"type": "host-local",
"subnet": "10.0.0.0/24",
"rangeStart": "10.0.0.13",
"rangeEnd": "10.0.0.254"
}
}'
Apply the configuration:
[vagrant@k8s ~]$ kubectl apply -f /vagrant/net-config.yml
Now simply reference that network configuration in our pod spec to pass the interface into our Pod. This can be done within a pod definition, deployment, etc. Apply the example pod shown below from /vagrant/example-mcast-pod.yml
to kick off our multicast consumer.
apiVersion: v1
kind: Pod
metadata:
name: multicast-example
annotations:
k8s.v1.cni.cncf.io/networks: eth1-multicast@eth1
spec:
containers:
- name: example-multicast-pod
command: ["iperf", "-s", "-u", "-B", "224.0.67.67%eth1", "-i", "1"]
image: bagoulla/iperf:2.0
Once it has successfully deployed tail the logs:
[vagrant@k8s ~]$ kubectl apply -f /vagrant/example-mcast-pod.yml
pod/multicast-example created
... # Use "kubectl describe pod multicast-example"
... # to track the launch
[vagrant@k8s ~]$ kubectl logs -f multicast-example
While tailing those logs, kick off the multicast sender from our mcastsrc VM and verify reception.
[vagrant@mcastsrc ~]$ docker run --net=host --rm bagoulla/iperf:2.0 -c 224.0.67.67 -u --ttl 5 -t 5 -B 10.0.0.12
...
There are two images used here, the first is a base image that has docker installed and the kubernetes yum repositories configured. The second has kubernetes installed. Our docker image is used as the mcastsrc while the kubernetes image is the k8s VM.
Both are built using packer. To rebuild both run:
[ylb@spectric ~]$ cd k8s-mcast-example/packer
[ylb@spectric packer]$ packer build docker_box.json
[ylb@spectric packer]$ vagrant box add bagoulla/docker-centos7 build/bagoulla/docker-centos7/package.box
[ylb@spectric packer]$ packer build k8s_box.json
[ylb@spectric packer]$ vagrant box add bagoulla/k8s-centos7 build/bagoulla/k8s-centos7/package.box
The configuration and installation of both the docker and Kubernetes image are found within packer/scripts/
.