prevent CSI driver from being evicted

[pohly]

Without any resource specifications, the CSI driver pod is among the first that gets evicted when resources are running low on a node. I was filling up /var by pulling a large image on host-1 of the QEMU virtual cluster and the result was:

pmem-csi-node-testing-dml4m               0/1     Evicted   0          2m4s    <none>        host-1   <none>           <none>
pmem-csi-node-testing-fn86q               1/1     Running   0          74m     192.168.8.8   host-3   <none>           <none>
pmem-csi-node-testing-pz64d               1/1     Running   0          74m     192.168.8.6   host-2   <none>           <none>
pmem-csi-node-w56mn                       0/2     Evicted   0          4m15s   <none>        host-1   <none>           <none>
...

I did not catch the description, but some other evicted pod had this event:

 Warning  Evicted    6m28s  kubelet, host-1    The node was low on resource: ephemeral-storage. Container .... was using 52Ki, which exceeds its request of 0.

We need to ensure that we properly declare the resource usage of our containers, including all sidecars.

[pohly]

The other evicted pod was rfs-redisfailover-pmem-75d4b4d856-2qwzf, from the spotahome redis operator. That operator also needs to be fixed to create pods with resource settings.