Skip to content

Commit

Permalink
Fixed white-space linting and did some re-ordering
Browse files Browse the repository at this point in the history
  • Loading branch information
Casey Kurosawa committed May 5, 2021
1 parent 5db0560 commit e146cc3
Showing 1 changed file with 16 additions and 2 deletions.
18 changes: 16 additions & 2 deletions doc/source/ingress/ambassador.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Ingress with Ambassador

Seldon Core works well with [Ambassador](https://www.getambassador.io/), allowing a single ingress to be used to expose Ambassador and [running machine learning deployments that can then be dynamically exposed](https://kubernetes.io/blog/2018/06/07/dynamic-ingress-in-kubernetes/) through Seldon-created Ambassador configurations. Ambassador is a Kubernetes-native API Gateway built on Envoy Proxy. Managed entirely via Kubernetes Custom Resource Definitions, Ambassador provides powerful capabilities for traffic management, authentication, and observability. Ambassador has native integrations for popular service meshes, including Consul, Istio, and Linkerd. In this doc we will discuss how your Seldon Deployments are exposed via Ambassador and how you can use both to do various production rollout strategies.

## Installing Ambassador

You have [two options](https://www.getambassador.io/editions/) when installing Ambassador:

### Option 1: Ambassador Edge Stack
The [Ambassador Edge Stack](https://www.getambassador.io/products/edge-stack/) is the easiest way to get started with Ambassador, providing easy, [Automatic TLS configuration](https://www.getambassador.io/docs/edge-stack/latest/topics/running/host-crd/#acme-support) in addition to other features like Rate Limiting and Authentication.

The [Ambassador Edge Stack](https://www.getambassador.io/products/edge-stack/) is the easiest way to get started with Ambassador, providing easy, [Automatic TLS configuration](https://www.getambassador.io/docs/edge-stack/latest/topics/running/host-crd/#acme-support) in addition to other features, like [Authentication](https://www.getambassador.io/docs/edge-stack/latest/topics/using/filters/), [Rate Limiting](https://www.getambassador.io/docs/edge-stack/latest/topics/using/rate-limits/), and advanced routing behaviors, like [custom prefixes](https://www.getambassador.io/docs/edge-stack/latest/topics/using/intro-mappings/), [virtual hosting](https://www.getambassador.io/docs/edge-stack/latest/topics/using/headers/host/), [method](https://www.getambassador.io/docs/edge-stack/latest/topics/using/method/)/[query-parameter](https://www.getambassador.io/docs/edge-stack/latest/topics/using/query_parameters/) based routing, and many others.
Follow the [AES installation instructions](https://www.getambassador.io/docs/edge-stack/latest/tutorials/getting-started/) to install it on your Kubernetes cluster.

Using `helm` the steps can be summarised as
Expand All @@ -20,6 +23,7 @@ helm install ambassador datawire/ambassador \
```

### Option 2: Ambassador API Gateway (now Emissary Ingress)

The [Ambassador API Gateway](https://www.getambassador.io/products/api-gateway/) (now Emissary Ingress) is the fully open source version of Ambassador Edge Stack and provides all the functionality of a traditional ingress controller.
Follow the [Ambassador OSS instructions](https://www.getambassador.io/docs/latest/topics/install/install-ambassador-oss/) to install it on your kubernetes cluster.

Expand All @@ -36,9 +40,11 @@ helm install ambassador datawire/ambassador \
```

### TLS

It is highly recommended to utilize TLS to encrypt traffic that is coming into Ambassador. The default Ambassador Edge Stack installation comes with a self-signed certificate that will be used absent any other TLS configuration. Follow the [instructions](https://www.getambassador.io/docs/edge-stack/latest/howtos/tls-termination/) for setting up TLS on your cluster.

## Ambassador REST

Assuming Ambassador is exposed at `<ambassadorEndpoint>` and with a Seldon deployment name `<deploymentName>` running in a namespace `namespace`:

Note, if you chose to install the Ambassador Edge Stack then you will need to use https. You can either [set up TLS](https://www.getambassador.io/docs/edge-stack/latest/howtos/tls-termination/) or pass the `-k` flag in `curl` to allow the self-signed certificate.
Expand All @@ -55,6 +61,7 @@ For Seldon Core running cluster wide, `singleNamespace=false`, the endpoints exp
## Example Curl

### Ambassador REST

If you installed the OSS Ambassador API Gateway, and assuming a Seldon Deployment `mymodel` with Ambassador exposed on `0.0.0.0:8003` you can send a curl request as follows:

```bash
Expand All @@ -74,6 +81,7 @@ curl -vk https://0.0.0.0/seldon/mymodel/api/v1.0/predictions -d '{"data":{"names
```

## Ambassador Configuration Annotations Reference

| Annotation | Description |
|------------|-------------|
|`seldon.io/ambassador-config:<configuration>`| Custom Ambassador Configuration |
Expand All @@ -94,6 +102,7 @@ All annotations should be placed in `spec.annotations`.
See below for details.

### Canary Deployments

Canary rollouts are available where you wish to push a certain percentage of traffic to a new model to test whether it works ok in production. To add a canary to your SeldonDeployment simply add a new predictor section and set the traffic levels for the main and canary to desired levels. For example:

```YAML
Expand Down Expand Up @@ -140,6 +149,7 @@ The above example has a "main" predictor with 75% of traffic and a "canary" with
A worked example for [canary deployments](../examples/ambassador_canary.html) is provided.

### Shadow Deployments

Shadow deployments allow you to send duplicate requests to a parallel deployment but throw away the response. This allows you to test machine learning models under load and compare the results to the live deployment.

Simply set the `shadow` boolean in your shadow predictor.
Expand All @@ -149,6 +159,7 @@ A worked example for [shadow deployments](../examples/ambassador_shadow.html) is
To understand more about the Ambassador configuration for this see [their docs on shadow deployments](https://www.getambassador.io/reference/shadowing/).

### Header based Routing

Header based routing allows you to route requests to particular Seldon Deployments based on headers in the incoming requests.

You simply need to add some annotations to your Seldon Deployment resource.
Expand All @@ -165,6 +176,7 @@ A worked example for [header based routing](../examples/ambassador_headers.html)
To understand more about the Ambassador configuration for this see [their docs on header based routing](https://www.getambassador.io/reference/headers).

### Circuit Breakers

By preventing additional connections or requests to an overloaded Seldon Deployment, circuit breakers help improve resilience of your system.

You simply need to add some annotations to your Seldon Deployment resource.
Expand All @@ -183,6 +195,7 @@ A worked example for [circuit breakers](../examples/ambassador_circuit_breakers.
To understand more about the Ambassador configuration for this see [their docs on circuit breakers](https://www.getambassador.io/docs/latest/topics/using/circuit-breakers/).

## Multiple Ambassadors in the same cluster

To avoid conflicts in a cluster with multiple ambassadors running, you can add the following annotation to your Seldon Deployment resource.

* `seldon.io/ambassador-id:<instance id>`: The instance id to be added to Ambassador `ambassador_id` configuration
Expand All @@ -205,7 +218,8 @@ Note that your Ambassador instance must be configured with matching `ambassador_
See [AMBASSADOR_ID](https://www.getambassador.io/docs/latest/topics/running/running/#ambassador_id) for details

### Custom Amabassador configuration
The above discussed configurations should cover most cases, however custom configuration is necessary to leverage the full feature-set of Ambassador Edge Stack, including things like [Authentication](https://www.getambassador.io/docs/edge-stack/latest/topics/using/filters/), [Rate Limiting](https://www.getambassador.io/docs/edge-stack/latest/topics/using/rate-limits/), and advanced routing behaviors like [custom prefixes](https://www.getambassador.io/docs/edge-stack/latest/topics/using/intro-mappings/), [virtual hosting](https://www.getambassador.io/docs/edge-stack/latest/topics/using/headers/host/), [method](https://www.getambassador.io/docs/edge-stack/latest/topics/using/method/)/[query-parameter](https://www.getambassador.io/docs/edge-stack/latest/topics/using/query_parameters/) based routing, and many others. There are two options for configuring Ambassador, based on how you want to manage the configuration: Custom Resource Definitions (CRD's) and Annotations.

The above discussed configurations should cover most cases, however custom configuration is necessary to leverage the full feature-set of Ambassador Edge Stack as a traffic management and authentication tool. There are two options for configuring Ambassador, based on how you want to manage the configuration: Custom Resource Definitions (CRD's) and Annotations.

Ambassador primarily utilizes [Custom Resource Definitions](https://www.getambassador.io/docs/edge-stack/latest/topics/concepts/gitops-continuous-delivery/#policies-declarative-configuration-and-custom-resource-definitions) for managing configuration. These are custom `kind` resources that the Kubernetes API can read, and are used to update Ambassador's config in a way that is observable to the cluster as a whole (e.g. you can `kubectl get` these resources). These CRD's can be managed independent of the Seldon Deploy itself.

Expand Down

0 comments on commit e146cc3

Please sign in to comment.