Support for Other K8s Distros (K0s, K3s)

[Sharpz7]

I specifically would be interested in trying to get sysbox to run from k3s, but to the best of my knowledge this is not possible. I also didn;t see much in the docs about where support for these distros is right now, some clarification would be great :))

#606

[ctalledo]

Thanks for the question @Sharpz7; as discussed in #606, Sysbox does not yet (as of 01/2023) support installing itself on k3s or k0s.

Specifically, the sysbox-deploy-k8s daemonset that installs Sysbox on K8s nodes only works on main / upstream K8s, and relies on the ability to reconfigure the Kubelet to use CRI-O instead of containerd, as the former supports pods with user-namespace while the latter does not yet (though that's WIP).

I believe K3s and K0s use a customized kubelet which causes sysbox-deploy-k8s to get fail to install Sysbox.

The problem will likely go away once containerd adds support for pods with user-namespaces (coming very soon).

@rodnymolina may have more detailed info on this.

[Sharpz7]

Thanks for getting back to me :)

The problem will likely go away once containerd adds support for pods with user-namespaces (coming very soon).

Do you happen to know where I can keep track of the progress of this change? I potentially want to do a few talks/blog posts including sysbox.

[ctalledo]

@rodnymolina has been tracking this effort on the containerd side, he will respond soon.

[ctalledo]

Thanks for planning to spread the word on Sysbox !

[ctalledo]

Needless to say that our goal is for Sysbox to run in as many K8s distros as possible.

[Sharpz7]

Any update on this?

[gdraganic]

@Sharpz7

I've managed to install and run sysbox on K3s. In the end the biggest problem was that embedded flannel in K3s is not compatible with cri-o (or any external CRI?) so you have to install the CNI yourself.

k3s-io/k3s#107

@ctalledo I can contribute the kubelet-config-helper for K3s detection and reconfiguration but IMHO it is not worth to support it at the moment.

[ctalledo]

Hi @gdraganic, that's great, thanks for working on that and letting us know!

@ctalledo I can contribute the kubelet-config-helper for K3s detection and reconfiguration but IMHO it is not worth to support it at the moment.

If you have the cycles, that would be great; several folks have asked about this for a while now.

Otherwise if you could at least document the steps (in this discussion), particularly around the k3s flannel/CRI-O incompatibility, that would be really cool.

Thanks again and glad you got it working!