[Story] Operator should handle target namespace deletion

[ebaron]

I guess for this to work the Operator would also need to be watching for any namespaces to be created/deleted in the cluster, so that it can reconcile by deleting the original secrets if namespaces are deleted, or recreating/re-copying them if a matching namespace is recreated too?

I suppose other objects that are placed in target namespaces would suffer from the same issue (e.g. role bindings, CA secrets). This might be a bit tricky to handle. Since we can't have cross-namespace owner references, maybe we could do something like this:

1. Apply a label to all cross-namespace objects pointing to the namespace/name of the CR
2. Add a custom controller watch on cross-namepsace object types (e.g. secret, rolebinding) that checks for the label(s)
3. If the label exists, enqueue a reconcile request for that CR's namespace/name
4. Controller recreates any missing objects

As for deleting the certificate object from the install namespace for deleted namespaces, we could issue a get request for the namespace and, if it doesn't exist, delete the certificate and secret. I think the namespace deletion event should be captured with the above custom controller watch.

Originally posted by @ebaron in #938 (comment)

[ebaron]

@andrewazores I suppose with the above implementation, that CR would fail to reconcile until the namespace is recreated. The operator would keep trying to create the objects in a non-existent namespace. Perhaps that's okay, the CR does specify an invalid spec if the target namespace doesn't exist (currently).

[andrewazores]

Perhaps that's okay, the CR does specify an invalid spec if the target namespace doesn't exist (currently).

I think that makes enough sense - we just have to make sure we document this. But I agree that listing target namespaces that don't exist should be considered an invalid spec, and it's probably best that this results in some kind of a failure rather than a silent acceptance. In particular, if we are doing things like copying secrets into the target namespaces, then silently failing if the namespace doesn't exist, only to then perform the copy at a later date if it does get created, seems like it might be surprising behaviour and potentially a security issue - the user might have typo'd a namespace name by accident, not noticed because the deployment succeeded, and then their secret gets inadvertently created and shared into a different namespace later on which they did not intend.

We have talked about this kind of problem before, where the Operator should be proactive about checking RBAC for each of the target namespaces - is there a particular Issue we can link this to?

[ebaron]

We have talked about this kind of problem before, where the Operator should be proactive about checking RBAC for each of the target namespaces - is there a particular Issue we can link this to?

Do you mean this sort of multi-tenancy issue: #579?

[andrewazores]

Yes that looks like it's probably what I'm thinking of.