diff --git a/docs/gitbook/SUMMARY.md b/docs/gitbook/SUMMARY.md index ee7dce007..ef97dd2eb 100644 --- a/docs/gitbook/SUMMARY.md +++ b/docs/gitbook/SUMMARY.md @@ -14,5 +14,7 @@ * [Monitoring](usage/monitoring.md) * [Alerting](usage/alerting.md) +# Tutorials +* [Canary Deployments with Helm](tutorials/canary-helm-chart.md) diff --git a/docs/gitbook/tutorials/canary-helm-chart.md b/docs/gitbook/tutorials/canary-helm-chart.md new file mode 100644 index 000000000..324809ecf --- /dev/null +++ b/docs/gitbook/tutorials/canary-helm-chart.md @@ -0,0 +1,222 @@ +# Canary Deployments with Helm charts + +This guide shows you how to package a web app into a Helm chart and trigger a canary deployment on upgrade. + +### Packaging + +You'll be be using the [podinfo](https://github.com/stefanprodan/flagger/tree/master/charts/podinfo) chart. +This chart packages a web app made with Go, it's configuration, a horizontal pod autoscaler (HPA) +and the canary configuration file. + +``` +├── Chart.yaml +├── README.md +├── templates +│   ├── NOTES.txt +│   ├── _helpers.tpl +│   ├── canary.yaml +│   ├── configmap.yaml +│   ├── deployment.yaml +│   └── hpa.yaml +└── values.yaml +``` + +### Install + +Create a test namespace with Istio sidecar injection enabled: + +```bash +export REPO=https://github.com/raw/stefanprodan/flagger/master + +kubectl apply -f ${REPO}/artifacts/namespaces/test.yaml +``` + +Add Flagger Helm repository: + +```bash +helm repo add flagger https://flagger.app +``` + +Install podinfo with the release name `frontend` (replace `example.com` with your own domain): + +```bash +helm upgrade -i frontend flagger/podinfo \ +--namespace test \ +--set nameOverride=frontend \ +--set backend=http://backend.test:9898/echo \ +--set canary.enabled=true \ +--set canary.istioIngress.enabled=true \ +--set canary.istioIngress.gateway=public-gateway.istio-system.svc.cluster.local \ +--set canary.istioIngress.host=frontend.istio.example.com +``` + +After a couple of seconds Flagger will create the canary objects: + +```bash +# generated by Helm +configmap/frontend +deployment.apps/frontend +horizontalpodautoscaler.autoscaling/frontend +canary.flagger.app/frontend + +# generated by Flagger +configmap/frontend-primary +deployment.apps/frontend-primary +horizontalpodautoscaler.autoscaling/frontend-primary +service/frontend +service/frontend-canary +service/frontend-primary +virtualservice.networking.istio.io/frontend +``` + +After the `frontend-primary` deployment is available, Flagger will scale to zero the `frontend` deployment +and route all traffic to the primary pods. + +Open your browser and navigate to the frontend URL: + +![Frontend](https://github.com/raw/stefanprodan/flagger/master/docs/screens/demo-frontend.png) + +Now let's install the `backend` release without exposing it outside the mesh: + +```bash +helm upgrade -i backend flagger/podinfo \ +--namespace test \ +--set nameOverride=backend \ +--set canary.enabled=true \ +--set canary.istioIngress.enabled=false +``` + +Check if Flagger has successfully deployed the canaries: + +``` +kubectl -n test get canaries + +NAME STATUS WEIGHT LASTTRANSITIONTIME +backend Initialized 0 2019-02-12T18:53:18Z +frontend Initialized 0 2019-02-12T17:50:50Z +``` + +Click on the ping button in the `frontend` UI to trigger a HTTP POST request +that will reach the `backend` app: + +![Jaeger](https://github.com/raw/stefanprodan/flagger/master/docs/screens/demo-frontend-jaeger.png) + +### Upgrade + +First let's install a load testing service that will generate traffic during analysis: + +```bash +helm upgrade -i flagger-loadtester flagger/loadtester \ +--namepace=test +``` + +Let's enable the load tester and deploy a new `frontend` version: + +```bash +helm upgrade -i frontend flagger/podinfo/ \ +--namespace test \ +--reuse-values \ +--set canary.loadtest.enabled=true \ +--set image.tag=1.4.1 +``` + +Flagger detects that the deployment revision changed and starts the canary analysis along with the load test: + +``` +kubectl -n istio-system logs deployment/flagger -f | jq .msg + +New revision detected! Scaling up frontend.test +Halt advancement frontend.test waiting for rollout to finish: 0 of 2 updated replicas are available +Starting canary analysis for frontend.test +Advance frontend.test canary weight 5 +Advance frontend.test canary weight 10 +Advance frontend.test canary weight 15 +Advance frontend.test canary weight 20 +Advance frontend.test canary weight 25 +Advance frontend.test canary weight 30 +Advance frontend.test canary weight 35 +Advance frontend.test canary weight 40 +Advance frontend.test canary weight 45 +Advance frontend.test canary weight 50 +Copying frontend.test template spec to frontend-primary.test +Halt advancement frontend-primary.test waiting for rollout to finish: 1 old replicas are pending termination +Promotion completed! Scaling down frontend.test +``` + +You can monitor the canary deployment with Grafana, open the Flagger dashboard, +select `test` from the namespace dropdown, `frontend-primary` from the primary dropdown and `frontend` from the +canary dropdown. + +![Flagger Grafana Dashboard](https://github.com/raw/stefanprodan/flagger/master/docs/screens/demo-frontend-dashboard.png) + +Now let's trigger a canary deployment for the `backend` app, but this time you'll change a value in the configmap: + +```bash +helm upgrade -i backend flagger/podinfo/ \ +--namespace test \ +--reuse-values \ +--set canary.loadtest.enabled=true \ +--set httpServer.timeout=25s +``` + +Generate HTTP 500 errors: + +```bash +kubectl -n test exec -it flagger-loadtester-xxx-yyy sh + +watch curl http://backend-canary:9898/status/500 +``` + +Generate latency: + +```bash +kubectl -n test exec -it flagger-loadtester-xxx-yyy sh + +watch curl http://backend-canary:9898/delay/1 +``` + +Flagger detects the config map change and starts a canary analysis. Flagger will pause the advancement +when the HTTP success rate drops under 99% or when the average request duration in the last minute is over 500ms: + +``` +kubectl -n istio-system logs deployment/flagger -f | jq .msg + +ConfigMap backend has changed +New revision detected! Scaling up backend.test +Starting canary analysis for backend.test +Advance backend.test canary weight 5 +Advance backend.test canary weight 10 +Advance backend.test canary weight 15 +Advance backend.test canary weight 20 +Advance backend.test canary weight 25 +Advance backend.test canary weight 30 +Advance backend.test canary weight 35 +Halt backend.test advancement success rate 62.50% < 99% +Halt backend.test advancement success rate 88.24% < 99% +Advance backend.test canary weight 40 +Advance backend.test canary weight 45 +Halt backend.test advancement request duration 2.415s > 500ms +Halt backend.test advancement request duration 2.42s > 500ms +Advance backend.test canary weight 50 +Copying backend.test template spec to backend-primary.test +ConfigMap backend-primary synced +Promotion completed! Scaling down backend.test +``` + +![Flagger Grafana Dashboard](https://github.com/raw/stefanprodan/flagger/master/docs/screens/demo-backend-dashboard.png) + +If the number of failed checks reaches the canary analysis threshold, the traffic is routed back to the primary, +the canary is scaled to zero and the rollout is marked as failed. + +```bash +kubectl -n test get canary + +NAME STATUS WEIGHT LASTTRANSITIONTIME +backend Succeeded 0 2019-02-12T19:33:11Z +frontend Failed 0 2019-02-12T19:47:20Z +``` + +If you've enabled the Slack notifications, you'll receive an alert with the reason why the `backend` promotion failed. + + + diff --git a/docs/screens/demo-backend-dashboard.png b/docs/screens/demo-backend-dashboard.png new file mode 100644 index 000000000..d83034173 Binary files /dev/null and b/docs/screens/demo-backend-dashboard.png differ diff --git a/docs/screens/demo-frontend-dashboard.png b/docs/screens/demo-frontend-dashboard.png new file mode 100644 index 000000000..febf78cd6 Binary files /dev/null and b/docs/screens/demo-frontend-dashboard.png differ diff --git a/docs/screens/demo-frontend-jaeger.png b/docs/screens/demo-frontend-jaeger.png new file mode 100644 index 000000000..28da7ff0f Binary files /dev/null and b/docs/screens/demo-frontend-jaeger.png differ diff --git a/docs/screens/demo-frontend.png b/docs/screens/demo-frontend.png new file mode 100644 index 000000000..1dbf922cd Binary files /dev/null and b/docs/screens/demo-frontend.png differ