Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dependencies namespace propagation #109

Merged
merged 9 commits into from
Dec 8, 2022

Conversation

didierofrivia
Copy link
Collaborator

@didierofrivia didierofrivia commented Nov 22, 2022

This PR aims to solve the limitation introduced by #48. Noting that this will not be the final behaviour since a RFC is still being defined.

Closes #69
Addresses partially #56

A few considerations and limitations adopted:

  1. There's only one Kuadrant instance (CR applied) in the cluster.
  2. When the Kuadrant CR is applied, it will manage every single Gateway resource. (Will tag them)
  3. New Gateways are not managed by the existing Kuadrant.

Diagram of how everything will be setup in the verification steps:
custom-namespace

Namespaces
  • istio-system: This one is where istio and its gateway is installed. It also includes the Istio Authentication Policies.
  • kuadrant-system: This namespace is where the operators are installed, it could be setup with an env var.
  • kuadrant-test: An admin chosen namespace where the Kuadrant instance (Kuadrant CR and services) are gonna be installed. It also will host the Authorino AuthConfigs.
  • default: Or any other chosen namespace where the app developer will install its services, define Kuadrant policies, own networking objects, etc.

Verification Steps:

  1. Setup a local Kuadrant cluster
make local-setup
  1. Create a test namespace
kubectl create namespace kuadrant-test
  1. Apply Kuadrant object in order to deploy required dependencies (this will change in the future most probably)
kubectl -n kuadrant-test apply -f - <<EOF
---
apiVersion: kuadrant.io/v1beta1
kind: Kuadrant
metadata:
  name: kuadrant-sample
spec: {}
EOF
  1. Forward istio gateway port
kubectl port-forward -n istio-system service/istio-ingressgateway 9080:80 &
  1. Install toystore service:
 kubectl apply -f - <<EOF
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: toystore
  labels:
    app: toystore
spec:
  selector:
    matchLabels:
      app: toystore
  template:
    metadata:
      labels:
        app: toystore
    spec:
      containers:
        - name: toystore
          image: quay.io/3scale/authorino:echo-api
          env:
            - name: PORT
              value: "3000"
          ports:
            - containerPort: 3000
              name: http
  replicas: 1
---
apiVersion: v1
kind: Service
metadata:
  name: toystore
spec:
  selector:
    app: toystore
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 3000
EOF
  1. Add a HTTPRoute for the toystore service
kubectl apply -f - <<EOF
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
  name: toystore
  labels:
    app: toystore
spec:
  parentRefs:
    - name: istio-ingressgateway
      namespace: istio-system
  hostnames: ["*.toystore.com"]
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: "/toy"
          method: GET
      backendRefs:
        - name: toystore
          port: 80
EOF
  1. Check if the service is running. It might take up a couple of minutes for the deployment to be ready.
curl -v -H 'Host: api.toystore.com' http://localhost:9080/toy

It should return a 200

  1. Create API keys for user Bob and Alice
kubectl apply -f -<<EOF
---
apiVersion: v1
kind: Secret
metadata:
  annotations:
    secret.kuadrant.io/user-id: bob
  name: bob-key
  labels:
    authorino.kuadrant.io/managed-by: authorino
    app: toystore
stringData:
  api_key: IAMBOB
type: Opaque
---
apiVersion: v1
kind: Secret
metadata:
  annotations:
    secret.kuadrant.io/user-id: alice
  name: alice-key
  labels:
    authorino.kuadrant.io/managed-by: authorino
    app: toystore
stringData:
  api_key: IAMALICE
type: Opaque
EOF
  1. Create Kuadrant's AuthPolicy to configure API key based authentication
kubectl apply -f - <<EOF
---
apiVersion: kuadrant.io/v1beta1
kind: AuthPolicy
metadata:
  name: toystore
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: toystore
  rules:
  - hosts: ["*.toystore.com"]
    paths: ["/toy*"]
  authScheme:
    identity:
    - name: friends
      apiKey:
        allNamespaces: true
        selector:
          matchLabels:
            app: toystore
      credentials:
        in: authorization_header
        keySelector: APIKEY
    response:
    - json:
        properties:
          - name: userID
            value: null
            valueFrom:
              authJSON: auth.identity.metadata.annotations.secret\.kuadrant\.io/user-id
      name: rate-limit-apikey
      wrapper: envoyDynamicMetadata
      wrapperKey: ext_auth_data
EOF
  1. Now the endpoint should be protected
curl -v -H 'Host: api.toystore.com' http://localhost:9080/toy

It should return a 401 Unauthorized

You should be able to use either Alice or Bob API Key like:

curl -v -H 'Authorization: APIKEY IAMBOB' -H 'Host: api.toystore.com' http://localhost:9080/toy
  1. Create RateLimitPolicy to rate limit per API key basis
User Rate Limits
Bob 2 reqs / 10 secs (0.2 rps)
Alice 5 reqs / 10 secs (0.5 rps)
kubectl apply -f -<<EOF
---
apiVersion: kuadrant.io/v1beta1
kind: RateLimitPolicy
metadata:
  name: toystore
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: toystore
  rateLimits:
  - configurations:
      - actions:
          - metadata:
              descriptor_key: "userID"
              default_value: "no-user"
              metadata_key:
                key: "envoy.filters.http.ext_authz"
                path:
                  - segment:
                      key: "ext_auth_data"
                  - segment:
                      key: "userID"
    limits:
      - conditions:
          - "userID == bob"
        maxValue: 2
        seconds: 10
        variables: []
      - conditions:
          - "userID == alice"
        maxValue: 5
        seconds: 10
        variables: []
EOF
  1. Validating the rate limit policy

Only 2 requests every 10 allowed for Bob.

while :; do curl --write-out '%{http_code}' --silent --output /dev/null -H 'Authorization: APIKEY IAMBOB' -H 'Host: api.toystore.com'  -X GET http://localhost:9080/toy | egrep --color "\b(429)\b|$"; sleep 1; done

Only 5 requests every 10 allowed for Alice.

while :; do curl --write-out '%{http_code}' --silent --output /dev/null -H 'Authorization: APIKEY IAMALICE' -H 'Host: api.toystore.com'  -X GET http://localhost:9080/toy | egrep --color "\b(429)\b|$"; sleep 1; done
  1. Profit!

@didierofrivia didierofrivia force-pushed the fix-dependencies-namespace-propagation branch 11 times, most recently from 3cfbeb0 to 4d451fc Compare November 29, 2022 11:52
@didierofrivia didierofrivia force-pushed the fix-dependencies-namespace-propagation branch 5 times, most recently from 4757c9e to ecd20c2 Compare December 5, 2022 15:36
@didierofrivia didierofrivia force-pushed the fix-dependencies-namespace-propagation branch from ecd20c2 to 4f94832 Compare December 5, 2022 16:36
@didierofrivia didierofrivia changed the title [WIP] Fix dependencies namespace propagation Fix dependencies namespace propagation Dec 6, 2022
@didierofrivia didierofrivia self-assigned this Dec 6, 2022
@didierofrivia didierofrivia marked this pull request as ready for review December 6, 2022 13:21
@didierofrivia didierofrivia requested a review from a team as a code owner December 6, 2022 13:21
NamespaceSeparator = '/'
LimitadorName = "limitador"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to remove the ability to redefine this thru a environment variable?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I could rollback to env var with default too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just thought if there's only one limitador and authorino per kuadrant, there was no need to make a difference

@@ -142,6 +141,10 @@ func (r *KuadrantReconciler) Reconcile(eventCtx context.Context, req ctrl.Reques
}
}

if gwErr := r.reconcileClusterGateways(ctx, kObj); gwErr != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What drove this? This is new, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is basically the key step to communicate the Kuadrant instance namespace to the policies. It will annotate the Gateways with its namespace, thus the policies, who knows about the GW or at least the HTTPRoute pointing to the Gateway will have access to this annotation

@alexsnaps
Copy link
Member

Also, if I read this right, limitador now has to be in the kuadrant ns... this hadn't to be the case before, or was it?

@didierofrivia
Copy link
Collaborator Author

Previously it was the same, limitador and authorino were installed in the hardcoded kuadrant-system and previously that, to the ENV that kuadrant-controller was.

@didierofrivia
Copy link
Collaborator Author

I'll merge, then I could do a following PR addressing any future comments

@didierofrivia didierofrivia changed the base branch from main to OSSM December 8, 2022 14:31
@didierofrivia didierofrivia changed the base branch from OSSM to main December 8, 2022 14:32
@didierofrivia didierofrivia merged commit ab78b9a into main Dec 8, 2022
@didierofrivia didierofrivia deleted the fix-dependencies-namespace-propagation branch December 8, 2022 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Kuadrant instance only works in the kuadrant-system namespace
2 participants