expose.tls.secretName try to volume mount to core pod

[Hokwang]

Hi,

I have local kubernetes

I want to use ingress with tls enabled.

I guess here's the problem.

When I use tls in ingress,

kubectl create namespace ingress-nginx
kubectl create secret tls ingress-tls --key key.pem --cert cert.pem -n ingress-nginx
helm install stable/nginx-ingress -n ingress-nginx-harbor --namespace ingress-nginx -f ingress-nginx-harbor-values.yaml

ingress-nginx-harbor-values.yaml is

controller:
  extraArgs:
    default-ssl-certificate: "ingress-nginx-prod/ingress-tls"
<snip>

and my values.yaml for harbor is

expose:
  type: ingress
  tls:
    enabled: true
    secretName: "ingress-tls"
  ingress:
    hosts:
      core: harbor.sec
<snip>

and then run
helm install harbor/harbor -n harbor --namespace harbor -f harbor-values.yaml

Here's an error.

# kubectl describe pod/harbor-harbor-core-69d55875f5-cd6gk -n harbor
Events:
  Type     Reason       Age                  From                  Message
  ----     ------       ----                 ----                  -------
  Warning  FailedMount  105s (x5 over 113s)  kubelet, host  MountVolume.SetUp failed for volume "ca-download" : secret "ingress-tls" not found
  Normal   Scheduled    15s                  default-scheduler     Successfully assigned harbor/harbor-harbor-core-69d55875f5-cd6gk to host

few minutes later,

Warning  FailedMount  3m27s                  kubelet, host  Unable to mount volumes for pod "harbor-harbor-core-69d55875f5-cd6gk_harbor(22ba4fd1-90f0-11e9-ad13-246e966fb6cc)": timeout expired waiting for volumes to attach or mount for pod "harbor"/"harbor-harbor-core-69d55875f5-cd6gk". list of unmounted volumes=[ca-download]. list of unattached volumes=[config secret-key token-service-private-key ca-download psc default-token-n6bhw]

This related with core-dpl.yaml

harbor-helm/templates/core/core-dpl.yaml

Lines 111 to 123 in 8420734

	{{- if .Values.expose.tls.enabled }}
	- name: ca-download
	secret:
	{{- if eq (include "harbor.autoGenCertForIngress" .) "true" }}
	secretName: "{{ template "harbor.ingress" . }}"
	{{- else if eq (include "harbor.autoGenCertForNginx" .) "true" }}
	secretName: {{ template "harbor.nginx" . }}
	{{- else }}
	secretName: {{ .Values.expose.tls.secretName }}
	{{- end }}
	items:
	- key: ca.crt
	path: ca.crt

I don't understand why core need ca cert and why use ingress's!
and the secret does not have ca.crt as you know.

Now work-around is
kubectl create secret generic ingress-tls --from-file=ca.crt=cacerts -n harbor
create(extract) cacerts file manually and
create same name secret forcely.

Thanks.

[ywk253100]

The certificate can be downloaded from Harbor portal, this is why the certificate is mounted into core. Your ingress controller should use a separate default certificate rather than using the same one with Harbor.

[Hokwang]

I am in the corporate environment, so we don't need download ca file from harbor
and if the reason why user should download ca file is this,
we do not need --ca-file --cert-file --key-file option when using helm command.

If you really need, anyway, I think you have to separate with tls.secretName and ca-download secretName first,
and then provide the chart option for this (user can choose use or not).

[ywk253100]

The certificate isn't used only for helm client, it is used for docker client or any other http clients that need to call Harbor API. The certificate downloaded from portal must be the same one used for Harbor ingress.

As I said, you should use a separate certificate for Harbor ingress rather than using the default one for ingress controller. If your certificate isn't signed by a CA, you can just copy the tls.crt to ca.crt and create a secret using the tls.key, tls.crt, ca.crt.

[Hokwang]

For clarification,

1. please make user can use different name for ca-download secretName
2. please make user choose use ca-download secret or not.

I don't understand basically, there's no options for ca.crt file in "kubectl create secret tls" command.

[pfyod]

Stumbled on this... I am using cert manager with Let's Encrypt, and it does not (and will not) provide "ca.crt" key in the automatically provisioned ACME certs/secrets: cert-manager/cert-manager#1571

Dirty wourkaround is to add "ca.crt" with intermediate certs to the cert manager generated secret:

kubectl patch secret \
  -n <namespace> <secret name> \
  -p="{\"data\":{\"ca.crt\": \"$(kubectl get secret \
  -n <namespace> <secret name> \
  -o json -o=jsonpath="{.data.tls\.crt}" \
  | base64 -d | awk 'f;/-----END CERTIFICATE-----/{f=1}' - | base64 -w 0)\"}}"

[qingchn]

I have a same problem. I use aliyun LB service。
this is my valuse.yaml

expose:
  # Set the way how to expose the service. Set the type as "ingress",
  # "clusterIP", "nodePort" or "loadBalancer" and fill the information
  # in the corresponding section
  type: loadBalancer
  tls:
    # Enable the tls or not. Note: if the type is "ingress" and the tls
    # is disabled, the port must be included in the command when pull/push
    # images. Refer to https://github.com/goharbor/harbor/issues/5291
    # for the detail.
    enabled: true
    # Fill the name of secret if you want to use your own TLS certificate.
    # The secret must contain keys named:
    # "tls.crt" - the certificate
    # "tls.key" - the private key
    # "ca.crt" - the certificate of CA
    # These files will be generated automatically if the "secretName" is not set
    secretName: "tls-string"
    # By default, the Notary service will use the same cert and key as
    # described above. Fill the name of secret if you want to use a
    # separated one. Only needed when the type is "ingress".
    notarySecretName: ""
    # The commmon name used to generate the certificate, it's necessary
    # when the type isn't "ingress" and "secretName" is null
    commonName: ""

I have modified the code
https://github.com/goharbor/harbor-helm/blob/master/templates/core/core-dpl.yaml

      - name: ca-download
        secret:
        {{- if eq (include "harbor.autoGenCertForIngress" .) "true" }}
          secretName: "{{ template "harbor.ingress" . }}"
        {{- else if eq (include "harbor.autoGenCertForNginx" .) "true" }}
          secretName: {{ template "harbor.nginx" . }}
        {{- else }}
          secretName: {{ .Values.expose.tls.secretName }}
        {{- end }}
        #  items:
        #    - key: ca.crt
        #      path: ca.crt
      {{- end }}```

so  now all is ok

[Sayrus]

Same problem.
Please let us pick a secret for the ingress and a different secret for harbor itself.