How to get metrics like "kubelet_volume_stats_used_bytes" and "kubelet_volume_stats_available_bytes" to get PVC usage in k3s cluster?

[dsolanki28]

We would like to have metrics "kubelet_volume_stats_used_bytes" and "kubelet_volume_stats_available_bytes" available in prometheus to monitor PVC usage in k3s cluster. To enable them, we tried to install k3s cluster with arguements "--kubelet-arg kubelet-preferred-address-types=InternalIP,Hostname,ExternalIP --kubelet-arg enable-kubelet-volume-stats" but error showing the flags are unknown to k3s.

Request you please help us to enable these metrics, or other supported metrics to get PVC usage in k3s cluster.

[brandond]

This is an issue with Kubernetes/cadvisor when used with containerd (or pretty much any other CRI backend now that dockershim has been removed). You can track it upstream at google/cadvisor#2785 (comment)