[nginx-ingress-controller] Refactoring of template handling

[aledbf]

This also watch for changes in the template

fixes #1457

---

This change is

[bprashanth]

I would vastly prefer if we did this in a compsable way. Perhaps we can have a sidecar that watches the config map, notices a change, and forces a reload on the nginx controller through a /reload http endpoint ? that would make for a nice reusable component.

$ watch-configmaps --reload-url="url-in-same-netns" --file="/path/to/file"

When we upgrade to docker1.12 we will get shared pid namespaces and can consider replacing the --reload-url arg with just pkill SIGHIUP nginx.

The most reusable way to do this right now would be to avoid even the http server, and do:

$ watch-configmaps --pod-name="foopod, defaults to self" \
    --container-name="nginx, defaults to first peer container in list of own pod" \
    --file="/path/to/file" \
    --script="/reload.sh"

That would kubectl exec in the give pod and exec the the signal to the given container/process. If you're interested you can download the kubectl client from:
wget https://storage.googleapis.com/kubernetes-release/release/${KUBERNETES_VERSION}/bin/linux/amd64/kubectl

Or exec through a REST client like we do in our e2es:
https://github.com/kubernetes/kubernetes/blob/4bb5fdc47f4c409f7d290b3af348fb135d0daf0f/test/e2e/framework/util.go#L3756

Quite a few people have asked for something like the last one. I would put this under ingress/ so it doesn't get lost when we split ingress out of contrib.

[aledbf]

I would vastly prefer if we did this in a composable way

Agree. I'll start working on this.

[aledbf]

@bprashanth the last commit contains a sidecar that watch changes in a file or a kubernetes resource (configmap and secrets for now).
Writing the example I realized that this requires the use of volumes (at least emptyDir, copying the template before the start of the controller) to be able to watch the file located in a different container.

Is this change acceptable just to be able to use a sidecar and not add this code in the controller?

[k8s-github-robot]

@aledbf PR needs rebase

[k8s-github-robot]

@aledbf PR needs rebase

[aledbf]

@bprashanth ping

[bprashanth]

don't you want to reload nginx when this happens?

[aledbf]

Right. The correct content should be curl http://localhost:10254/reload-template

[bprashanth]

Suggest simplifying the pr to just watch configmaps mounted into 2 containers, and send a signal from the first to the second through kubeClient. Then write a good example that uses a configmap, modifies it , and has an nginx reload. We can iterate on growing that out as required.

[aledbf]

Suggest simplifying the pr to just watch configmaps mounted into 2 containers, and send a signal from the first to the second through kubeClient.

Just in case if an user just upgrades the current version (without a volume) the controller behaves like now (no reload if the template changes)

Then write a good example that uses a configmap, modifies it , and has an nginx reload.

I think this https://gist.github.com/aledbf/e9e669657ab00ae2f8562e89e47c3cb1 is that example

[aledbf]

Suggest simplifying the pr to just watch

done

Here is the important part

Basically I watch for the template file for changes. Is not important from where the file comes (emptyDir, another volume, configmap, etc).

[aledbf]

This is an example of the change

kubectl exec nginx-ingress-controller-7g2rv -- echo "" >> /etc/nginx/template/nginx.tmpl

I0825 00:21:01.317244       1 main.go:94] new NGINX template loaded
I0825 00:21:04.832265       1 command.go:81] change in configuration detected. Reloading...

kubectl exec nginx-ingress-controller-7g2rv -- echo "{{ something }}" >> /etc/nginx/template/nginx.tmpl

W0825 00:21:59.913088       1 main.go:88] invalid NGINX template: template: nginx.tmpl:370: function "something" not defined

[aledbf]

Also contains update to godeps required by #1623

[bprashanth]

why emptyDir volume? isn't this just like the nginx example in main repo, except this sidecar is reusable with any existing pod that mounts a config map?

[bprashanth]

LGTM

[k8s-github-robot]

Automatic merge from submit-queue

[tjliupeng]

@aledbf , I change the content of /etc/nginx/template/nginx.tmpl in a running container, but no new nginx.conf is generated. What do I miss?