Skip to content

Latest commit

 

History

History
176 lines (155 loc) · 3.67 KB

challenges.istio.3.sm-remove-retry.md

File metadata and controls

176 lines (155 loc) · 3.67 KB
Raw

Remove Slow/Buggy Containers

Need help? All neccessary files are here 📘!

!! DUE TO A BUG/MISSING FEATURE CURRENTLY EXISTING ISTIO, THIS EXAMPLE WON'T FULLY WORK !!

Here is what you will learn

  • use Istio service mesh to automatically remove services from your application that throw errors

Deploy new JS Backend (with errors)

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jscalcerror-v3-error
  namespace: challengeistio
spec:
  replicas: 1
  minReadySeconds: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  template:
    metadata:
      labels:
        name: jscalcerror
        app: backend
        version: v3-error
    spec:
      containers:
      - name: jscalcerror
        image: csaocpger/jscalcerror:6.0
        ports:
          - containerPort: 80
            name: http
            protocol: TCP
        env:
          - name: "PORT"
            value: "80"

Deployment File 📘!

Add Destionation Rules

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: calcbackend-rule
  namespace: challengeistio
spec:
  host: calcbackendsvc
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3-error
    labels:
      version: v3-error

Deployment File 📘!

Include Error-Backend in Request Routing

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: backend-vs
  namespace: challengeistio
spec:
  hosts:
  - calcbackendsvc
  http:
  - match:
    - headers:
        user-agent:
          regex: .*Mobile.*
    route:
      - destination:
          host: calcbackendsvc
          subset: v2
  - route:
    - destination:
        host: calcbackendsvc
        subset: v1
      weight: 50
    - destination:
        host: calcbackendsvc
        subset: v3-error
      weight: 50

Deployment File 📘!

See how errors are thrown

Open browser an run calculation in loop and see how errors appear approximately 2 minutes after the deployment of the JS "error backend".

Add traffic policy to remove/suspend buggy pods

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: calcbackend-rule
  namespace: challengeistio
spec:
  host: calcbackendsvc
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3-error
    labels:
      version: v3-error
    trafficPolicy:
      connectionPool:
        http: {}
        tcp: {}
      outlierDetection:
        baseEjectionTime: 3m
        consecutiveErrors: 1
        interval: 10s
        maxEjectionPercent: 100

See how pods are removed after errors appear and be brought back after 3 minutes via kubectl get po -n challengeistio -w

Add retry-strategies

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: backend-vs
  namespace: challengeistio
spec:
  hosts:
  - calcbackendsvc
  http:
  - match:
    - headers:
        user-agent:
          regex: .*Mobile.*
    route:
      - destination:
          host: calcbackendsvc
          subset: v2
  - route:
    - destination:
        host: calcbackendsvc
        subset: v1
      weight: 50
    - destination:
        host: calcbackendsvc
        subset: v3-error
      weight: 50
    retries:
      attempts: 3
      perTryTimeout: 2s

See how errors disappear...