Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AP Defaults #503

Merged
merged 10 commits into from
Apr 8, 2024
5 changes: 3 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -406,14 +406,15 @@ kind-load-bundle: ## Load image to local cluster
##@ Deployment

install: manifests kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config.
$(KUSTOMIZE) build config/crd | kubectl apply -f -
# Use server side apply, otherwise will hit into this issue https://medium.com/pareture/kubectl-install-crd-failed-annotations-too-long-2ebc91b40c7d
$(KUSTOMIZE) build config/crd | kubectl apply --server-side -f -
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure will this be an issue for OLM upgrading CRDs. I'm wishful that this will not be an issue for OLM, and this is just a local issue 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AuthPolicy CRD itself has reached ~600 Kb already with this change. IIRC, by default, the maximum request size accepted by etcd is 1.5 Mb – actual value size between 2 Mb (default) and 8 Mb (recommended maximum.)


uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config.
$(KUSTOMIZE) build config/crd | kubectl delete -f -

deploy: manifests dependencies-manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
$(KUSTOMIZE) build config/deploy | kubectl apply -f -
$(KUSTOMIZE) build config/deploy | kubectl apply --server-side -f -
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMAGE_TAG_BASE):latest

undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config.
Expand Down
91 changes: 65 additions & 26 deletions api/v1beta2/authpolicy_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ type CommonAuthRuleSpec struct {
// At least one selected HTTPRoute rule must match to trigger the auth rule.
// If no route selectors are specified, the auth rule will be evaluated at all requests to the protected routes.
// +optional
// +kubebuilder:validation:MaxItems=15
// +kubebuilder:validation:MaxItems=8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need any test to cover this being changed in the future, as it was reduced from 15 to 8?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, can add a test. Though, I'm expecting this value to decrease again when the overrides field is added in #464

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, and thinking out loud. If we need to increase that number in the future, is the workaround to create a webhook to do the validation and drop the use of the CEL?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added integration test for this.

I don't think just simply switching to using validation webhook would work, as I believe this more so relates to using a slice of RouteSelectors that the fields itself also have CEL validation for and how frequently we have this slice of RouteSelectors in the various structs used within the CRD. This will fail the CEL cost estimation when applying the CRD even if switch to using a validating webhook 🤔

type RouteSelector struct {
// Hostnames defines a set of hostname that should match against the HTTP Host header to select a HTTPRoute to process the request
// https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.HTTPRouteSpec
// +optional
Hostnames []gatewayapiv1.Hostname `json:"hostnames,omitempty"`
// Matches define conditions used for matching the rule against incoming HTTP requests.
// https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.HTTPRouteSpec
// +optional
// +kubebuilder:validation:MaxItems=8
Matches []gatewayapiv1.HTTPRouteMatch `json:"matches,omitempty"`
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because we're importing Gateway API types and, with those, their CEL validation as well.

I guess our best options to avoid this hard limit are:

  1. wait for Adding GEP-995 for support for named route rules kubernetes-sigs/gateway-api#2593 to be implemented and then simplify the route selectors
  2. redefine the HTTPRouteMatch types ourselves without the validation – we could do this because the validation that matters is the one enforced on the HTTPRoute, not on the policy; on the policy, the only consequence of an "invalid" route selector is that the selector will not select any route rule.

RouteSelectors []RouteSelector `json:"routeSelectors,omitempty"`
}

Expand Down Expand Up @@ -128,19 +128,42 @@ type CallbackSpec struct {
CommonAuthRuleSpec `json:""`
}

// RouteSelectors - implicit default validation
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.routeSelectors)",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules.authentication) || !self.rules.authentication.exists(x, has(self.rules.authentication[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules.metadata) || !self.rules.metadata.exists(x, has(self.rules.metadata[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules.authorization) || !self.rules.authorization.exists(x, has(self.rules.authorization[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules.response) || !has(self.rules.response.success) || !has(self.rules.response.success.headers) || !self.rules.response.success.headers.exists(x, has(self.rules.response.success.headers[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules.response) || !has(self.rules.response.success) || !has(self.rules.response.success.dynamicMetadata) || !self.rules.response.success.dynamicMetadata.exists(x, has(self.rules.response.success.dynamicMetadata[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules.callbacks) || !self.rules.callbacks.exists(x, has(self.rules.callbacks[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules) || !has(self.rules.authentication) || !self.rules.authentication.exists(x, has(self.rules.authentication[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules) || !has(self.rules.metadata) || !self.rules.metadata.exists(x, has(self.rules.metadata[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules) || !has(self.rules.authorization) || !self.rules.authorization.exists(x, has(self.rules.authorization[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules) || !has(self.rules.response) || !has(self.rules.response.success) || !has(self.rules.response.success.headers) || !self.rules.response.success.headers.exists(x, has(self.rules.response.success.headers[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules) || !has(self.rules.response) || !has(self.rules.response.success) || !has(self.rules.response.success.dynamicMetadata) || !self.rules.response.success.dynamicMetadata.exists(x, has(self.rules.response.success.dynamicMetadata[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.rules) || !has(self.rules.callbacks) || !self.rules.callbacks.exists(x, has(self.rules.callbacks[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// RouteSelectors - explicit default validation
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.routeSelectors)",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.rules) || !has(self.defaults.rules.authentication) || !self.defaults.rules.authentication.exists(x, has(self.defaults.rules.authentication[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.rules) || !has(self.defaults.rules.metadata) || !self.defaults.rules.metadata.exists(x, has(self.defaults.rules.metadata[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.rules) || !has(self.defaults.rules.authorization) || !self.defaults.rules.authorization.exists(x, has(self.defaults.rules.authorization[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.rules) || !has(self.defaults.rules.response) || !has(self.defaults.rules.response.success) || !has(self.defaults.rules.response.success.headers) || !self.defaults.rules.response.success.headers.exists(x, has(self.defaults.rules.response.success.headers[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.rules) || !has(self.defaults.rules.response) || !has(self.defaults.rules.response.success) || !has(self.defaults.rules.response.success.dynamicMetadata) || !self.defaults.rules.response.success.dynamicMetadata.exists(x, has(self.defaults.rules.response.success.dynamicMetadata[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// +kubebuilder:validation:XValidation:rule="self.targetRef.kind != 'Gateway' || !has(self.defaults) || !has(self.defaults.rules) || !has(self.defaults.rules.callbacks) || !self.defaults.rules.callbacks.exists(x, has(self.defaults.rules.callbacks[x].routeSelectors))",message="route selectors not supported when targeting a Gateway"
// Mutual Exclusivity Validation
// +kubebuilder:validation:XValidation:rule="!(has(self.defaults) && (has(self.routeSelectors) || has(self.patterns) || has(self.when) || has(self.rules)))",message="Implicit and explicit defaults are mutually exclusive"
type AuthPolicySpec struct {
// TargetRef identifies an API object to apply policy to.
// +kubebuilder:validation:XValidation:rule="self.group == 'gateway.networking.k8s.io'",message="Invalid targetRef.group. The only supported value is 'gateway.networking.k8s.io'"
// +kubebuilder:validation:XValidation:rule="self.kind == 'HTTPRoute' || self.kind == 'Gateway'",message="Invalid targetRef.kind. The only supported values are 'HTTPRoute' and 'Gateway'"
TargetRef gatewayapiv1alpha2.PolicyTargetReference `json:"targetRef"`

// Defaults define explicit default values for this policy and for policies inheriting this policy.
// Defaults are mutually exclusive with implicit defaults defined by AuthPolicyCommonSpec.
// +optional
Defaults *AuthPolicyCommonSpec `json:"defaults,omitempty"`

// AuthPolicyCommonSpec defines implicit default values for this policy and for policies inheriting this policy.
// AuthPolicyCommonSpec is mutually exclusive with explicit defaults defined by Defaults.
AuthPolicyCommonSpec `json:""`
}

// AuthPolicyCommonSpec contains common shared fields for defaults and overrides
type AuthPolicyCommonSpec struct {
// Top-level route selectors.
// If present, the elements will be used to select HTTPRoute rules that, when activated, trigger the external authorization service.
// At least one selected HTTPRoute rule must match to trigger the AuthPolicy.
Expand All @@ -161,13 +184,13 @@ type AuthPolicySpec struct {

// The auth rules of the policy.
// See Authorino's AuthConfig CRD for more details.
AuthScheme AuthSchemeSpec `json:"rules,omitempty"`
AuthScheme *AuthSchemeSpec `json:"rules,omitempty"`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to pointer so that this will not show on json spec as empty object {} when empty which would be a bit confusing to a user, especially if spec.defaults is used instead 🤔 Also makes the mutual exclusive CEL rule a bit easier by switching to pointer

}

// GetRouteSelectors returns the top-level route selectors of the auth scheme.
// impl: RouteSelectorsGetter
func (s AuthPolicySpec) GetRouteSelectors() []RouteSelector {
return s.RouteSelectors
func (c AuthPolicyCommonSpec) GetRouteSelectors() []RouteSelector {
return c.RouteSelectors
}

type AuthPolicyStatus struct {
Expand Down Expand Up @@ -262,28 +285,36 @@ func (ap *AuthPolicy) GetRulesHostnames() (ruleHosts []string) {
}
}

appendRuleHosts(ap.Spec)
for _, config := range ap.Spec.AuthScheme.Authentication {
appendRuleHosts(config)
}
for _, config := range ap.Spec.AuthScheme.Metadata {
appendRuleHosts(config)
}
for _, config := range ap.Spec.AuthScheme.Authorization {
appendRuleHosts(config)
}
if response := ap.Spec.AuthScheme.Response; response != nil {
for _, config := range response.Success.Headers {
appendCommonSpecRuleHosts := func(c *AuthPolicyCommonSpec) {
if c.AuthScheme == nil {
return
}

for _, config := range c.AuthScheme.Authentication {
appendRuleHosts(config)
}
for _, config := range response.Success.DynamicMetadata {
for _, config := range c.AuthScheme.Metadata {
appendRuleHosts(config)
}
for _, config := range c.AuthScheme.Authorization {
appendRuleHosts(config)
}
if response := c.AuthScheme.Response; response != nil {
for _, config := range response.Success.Headers {
appendRuleHosts(config)
}
for _, config := range response.Success.DynamicMetadata {
appendRuleHosts(config)
}
}
for _, config := range c.AuthScheme.Callbacks {
appendRuleHosts(config)
}
}
for _, config := range ap.Spec.AuthScheme.Callbacks {
appendRuleHosts(config)
}

appendRuleHosts(ap.Spec.CommonSpec())
appendCommonSpecRuleHosts(ap.Spec.CommonSpec())

return
}

Expand All @@ -299,6 +330,14 @@ func (ap *AuthPolicy) DirectReferenceAnnotationName() string {
return AuthPolicyDirectReferenceAnnotationName
}

func (ap *AuthPolicySpec) CommonSpec() *AuthPolicyCommonSpec {
if ap.Defaults != nil {
return ap.Defaults
}

return &ap.AuthPolicyCommonSpec
}

//+kubebuilder:object:root=true

// AuthPolicyList contains a list of AuthPolicy
Expand Down
33 changes: 18 additions & 15 deletions api/v1beta2/authpolicy_types_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ func TestAuthPolicyGetRulesHostnames(t *testing.T) {
t.Errorf("Expected hostname to be %s, got %s", expected, result[1])
}
// + 1 authentication route selector with 1 hostname
policy.Spec.AuthScheme = &AuthSchemeSpec{}
policy.Spec.AuthScheme.Authentication = map[string]AuthenticationSpec{
"my-authn": {
CommonAuthRuleSpec: CommonAuthRuleSpec{
Expand Down Expand Up @@ -261,22 +262,24 @@ func TestAuthPolicyValidate(t *testing.T) {
Name: "my-route",
Namespace: ptr.To(gatewayapiv1.Namespace("other-namespace")),
},
AuthScheme: AuthSchemeSpec{
Authentication: map[string]AuthenticationSpec{
"my-rule": {
AuthenticationSpec: authorinoapi.AuthenticationSpec{
AuthenticationMethodSpec: authorinoapi.AuthenticationMethodSpec{
AnonymousAccess: &authorinoapi.AnonymousAccessSpec{},
AuthPolicyCommonSpec: AuthPolicyCommonSpec{
AuthScheme: &AuthSchemeSpec{
Authentication: map[string]AuthenticationSpec{
"my-rule": {
AuthenticationSpec: authorinoapi.AuthenticationSpec{
AuthenticationMethodSpec: authorinoapi.AuthenticationMethodSpec{
AnonymousAccess: &authorinoapi.AnonymousAccessSpec{},
},
},
},
CommonAuthRuleSpec: CommonAuthRuleSpec{
RouteSelectors: []RouteSelector{
{
Hostnames: []gatewayapiv1.Hostname{"*.foo.io"},
Matches: []gatewayapiv1.HTTPRouteMatch{
{
Path: &gatewayapiv1.HTTPPathMatch{
Value: ptr.To("/foo"),
CommonAuthRuleSpec: CommonAuthRuleSpec{
RouteSelectors: []RouteSelector{
{
Hostnames: []gatewayapiv1.Hostname{"*.foo.io"},
Matches: []gatewayapiv1.HTTPRouteMatch{
{
Path: &gatewayapiv1.HTTPPathMatch{
Value: ptr.To("/foo"),
},
},
},
},
Expand Down
85 changes: 55 additions & 30 deletions api/v1beta2/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading