helm-charts upgrade fails on ambassador

[OrBarda]

When trying to upgrade the helm charts of Seldon-core to release 0.2.7, it shows a problem in the deployment which is related to the ambassador:

Then when trying again to upgrade getting the following:

And eventually what happens is that a new cluster-manager is up but the old one stay as well and for upgrading the ambassador a delete of the chart is needed which causes a problem in availability.

[ryandawsonuk]

I think this is because we're now using the ambassador helm chart. helm/helm#1193 has been a helm issue for a while - an example of a similar failure with a chart resource (also a service account but can happen with other resources too) is cert-manager/cert-manager#677. The issue is reported as fixed but the fix is quite recent. Which helm version are you using?

[OrBarda]

we are using 2.12.3.

[ryandawsonuk]

I think the next step is to see if we can recreate this with the latest version of helm. We can install an old version of seldon-core and then do a helm upgrade to the new version. I'll aim to do this soon.

[ryandawsonuk]

Actually ambassador will soon be removed from the seldon-core main chart so anyone upgrading to that version will need to install ambassador separately or add it to their own wrapper chart. This issue will then no longer apply

[ukclivecox]

Fix by new separation of operator from ingress.