Skip to content

Commit

Permalink
Add networkpolices to protect metrics endpoint and allow communicatio…
Browse files Browse the repository at this point in the history
…n with webhooks
  • Loading branch information
camilamacedo86 committed Jul 21, 2024
1 parent b3eeace commit 675b8b6
Show file tree
Hide file tree
Showing 38 changed files with 790 additions and 21 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test-sample-go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
run: |
KUSTOMIZATION_FILE_PATH="testdata/project-v4/config/default/kustomization.yaml"
sed -i '25s/^#//' $KUSTOMIZATION_FILE_PATH
sed -i '46s/^#//' $KUSTOMIZATION_FILE_PATH
sed -i '51s/^#//' $KUSTOMIZATION_FILE_PATH
- name: Test
run: |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ resources:
- ../prometheus
# [METRICS] Expose the controller manager metrics service.
- metrics_service.yaml
# [NETWORK POLICY] Protect the /metrics endpoint and Webhook Server with NetworkPolicy.
# Only Pod(s) running a namespace labeled with 'metrics: enabled' will be able to gather the metrics.
# Only CR(s) which requires webhooks and are applied on namespaces labeled 'webhooks: enabled' will
# be able to communicate with the Webhook Server.
#- ../policy

# Uncomment the patches line if you enable Metrics, and/or are using webhooks and cert-manager
patches:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# This NetworkPolicy allows ingress and egress traffic
# with Pods running on namespaces labeled with 'metrics: enabled'. Only Pods on those
# namespaces are able to gathering data from the metrics endpoint.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
labels:
app.kubernetes.io/name: project
app.kubernetes.io/managed-by: kustomize
name: allow-metrics-traffic
namespace: system
spec:
podSelector:
matchLabels:
control-plane: controller-manager
policyTypes:
- Egress
- Ingress
egress:
- {} # This allows all egress traffic
ingress:
# This allows ingress traffic from any namespace with the label metrics: enabled
- from:
- namespaceSelector:
matchLabels:
metrics: enabled # Only from namespaces with this label
ports:
- port: 8443
protocol: TCP
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# This NetworkPolicy allows ingress traffic to your webhook server running
# as part of the controller-manager from specific namespaces and pods.
# It also allows egress traffic from the controller-manager pods to any namespace.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
labels:
app.kubernetes.io/name: project
app.kubernetes.io/managed-by: kustomize
name: allow-webhook-traffic
namespace: system
spec:
podSelector:
matchLabels:
control-plane: controller-manager
policyTypes:
- Egress
- Ingress
egress:
- {} # This allows all egress traffic
ingress:
# This allows ingress traffic from any namespace with the label webhook: enabled
- from:
- namespaceSelector:
matchLabels:
webhook: enabled # Only from namespaces with this label
ports:
- port: 443
protocol: TCP
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
resources:
- allow-webhook-traffic.yaml
- allow-metrics-traffic.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ resources:
#- ../prometheus
# [METRICS] Expose the controller manager metrics service.
- metrics_service.yaml
# [NETWORK POLICY] Protect the /metrics endpoint and Webhook Server with NetworkPolicy.
# Only Pod(s) running a namespace labeled with 'metrics: enabled' will be able to gather the metrics.
# Only CR(s) which requires webhooks and are applied on namespaces labeled 'webhooks: enabled' will
# be able to communicate with the Webhook Server.
#- ../policy

# Uncomment the patches line if you enable Metrics, and/or are using webhooks and cert-manager
patches:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# This NetworkPolicy allows ingress and egress traffic
# with Pods running on namespaces labeled with 'metrics: enabled'. Only Pods on those
# namespaces are able to gathering data from the metrics endpoint.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
labels:
app.kubernetes.io/name: project
app.kubernetes.io/managed-by: kustomize
name: allow-metrics-traffic
namespace: system
spec:
podSelector:
matchLabels:
control-plane: controller-manager
policyTypes:
- Egress
- Ingress
egress:
- {} # This allows all egress traffic
ingress:
# This allows ingress traffic from any namespace with the label metrics: enabled
- from:
- namespaceSelector:
matchLabels:
metrics: enabled # Only from namespaces with this label
ports:
- port: 8443
protocol: TCP
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
resources:
- allow-metrics-traffic.yaml
16 changes: 12 additions & 4 deletions docs/book/src/reference/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ operational metrics. This exposure can lead to security vulnerabilities
where an attacker could gain insights into the system's operation
and exploit weaknesses.

### By using authn/authz (Enabled by default)
### (Enabled by default) By using authn/authz

To mitigate these risks, Kubebuilder projects utilize authentication (authn) and authorization (authz) to protect the
metrics endpoint. This approach ensures that only authorized users and service accounts can access sensitive metrics
Expand Down Expand Up @@ -189,11 +189,19 @@ enhance the controller-runtime and address these considerations.
</aside>


### By using Network Policy
### (You can optionally enable) By using Network Policy

NetworkPolicy acts as a basic firewall for pods within a Kubernetes cluster, controlling traffic
flow at the IP address or port level. However, it doesn't handle authentication (authn), authorization (authz),
or encryption directly like [kube-rbac-proxy](https://github.com/brancz/kube-rbac-proxy) solution.
flow at the IP address or port level. However, it doesn't handle `authn/authz`.

Uncomment the following line in the `config/default/kustomization.yaml`:

```
# [NETWORK POLICY] Protect the /metrics endpoint and Webhook Server with NetworkPolicy.
# Only Pod(s) running a namespace labeled with 'metrics: enabled' will be able to gather the metrics.
# Only CR(s) which uses webhooks and applied on namespaces labeled 'webhooks: enabled' will be able to work properly.
#- ../policy
```

### By exposing the metrics endpoint using HTTPS and CertManager

Expand Down
3 changes: 3 additions & 0 deletions pkg/plugins/common/kustomize/v2/scaffolds/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package scaffolds

import (
log "github.com/sirupsen/logrus"
"sigs.k8s.io/kubebuilder/v4/pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/policy"

"sigs.k8s.io/kubebuilder/v4/pkg/config"
"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
Expand Down Expand Up @@ -79,6 +80,8 @@ func (s *initScaffolder) Scaffold() error {
&kdefault.ManagerMetricsPatch{},
&manager.Config{Image: imageName},
&kdefault.Kustomization{},
&policy.Kustomization{},
&policy.NetworkPolicyAllowMetrics{},
&prometheus.Kustomization{},
&prometheus.Monitor{},
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ resources:
#- ../prometheus
# [METRICS] Expose the controller manager metrics service.
- metrics_service.yaml
# [NETWORK POLICY] Protect the /metrics endpoint and Webhook Server with NetworkPolicy.
# Only Pod(s) running a namespace labeled with 'metrics: enabled' will be able to gather the metrics.
# Only CR(s) which requires webhooks and are applied on namespaces labeled 'webhooks: enabled' will
# be able to communicate with the Webhook Server.
#- ../policy
# Uncomment the patches line if you enable Metrics, and/or are using webhooks and cert-manager
patches:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/*
Copyright 2024 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package policy

import (
"path/filepath"

"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
)

var _ machinery.Template = &NetworkPolicyAllowMetrics{}

// NetworkPolicyAllowMetrics scaffolds a file that defines the NetworkPolicy
// to allow access to the metrics endpoint
type NetworkPolicyAllowMetrics struct {
machinery.TemplateMixin
machinery.ProjectNameMixin
}

// SetTemplateDefaults implements file.Template
func (f *NetworkPolicyAllowMetrics) SetTemplateDefaults() error {
if f.Path == "" {
f.Path = filepath.Join("config", "policy", "allow-metrics-traffic.yaml")
}

f.TemplateBody = metricsNetworkPolicyTemplate

return nil
}

const metricsNetworkPolicyTemplate = `# This NetworkPolicy allows ingress and egress traffic
# with Pods running on namespaces labeled with 'metrics: enabled'. Only Pods on those
# namespaces are able to gathering data from the metrics endpoint.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
labels:
app.kubernetes.io/name: {{ .ProjectName }}
app.kubernetes.io/managed-by: kustomize
name: allow-metrics-traffic
namespace: system
spec:
podSelector:
matchLabels:
control-plane: controller-manager
policyTypes:
- Egress
- Ingress
egress:
- {} # This allows all egress traffic
ingress:
# This allows ingress traffic from any namespace with the label metrics: enabled
- from:
- namespaceSelector:
matchLabels:
metrics: enabled # Only from namespaces with this label
ports:
- port: 8443
protocol: TCP
`
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/*
Copyright 2024 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package policy

import (
"path/filepath"

"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
)

var _ machinery.Template = &NetworkPolicyAllowWebhooks{}

// NetworkPolicyAllowWebhooks in scaffolds a file that defines the NetworkPolicy
// to allow the webhook server can communicate
type NetworkPolicyAllowWebhooks struct {
machinery.TemplateMixin
machinery.ProjectNameMixin
}

// SetTemplateDefaults implements file.Template
func (f *NetworkPolicyAllowWebhooks) SetTemplateDefaults() error {
if f.Path == "" {
f.Path = filepath.Join("config", "policy", "allow-webhook-traffic.yaml")
}

f.TemplateBody = webhooksNetworkPolicyTemplate

return nil
}

const webhooksNetworkPolicyTemplate = `# This NetworkPolicy allows ingress traffic to your webhook server running
# as part of the controller-manager from specific namespaces and pods.
# It also allows egress traffic from the controller-manager pods to any namespace.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
labels:
app.kubernetes.io/name: {{ .ProjectName }}
app.kubernetes.io/managed-by: kustomize
name: allow-webhook-traffic
namespace: system
spec:
podSelector:
matchLabels:
control-plane: controller-manager
policyTypes:
- Egress
- Ingress
egress:
- {} # This allows all egress traffic
ingress:
# This allows ingress traffic from any namespace with the label webhook: enabled
- from:
- namespaceSelector:
matchLabels:
webhook: enabled # Only from namespaces with this label
ports:
- port: 443
protocol: TCP
`
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/*
Copyright 2024 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package policy

import (
"path/filepath"

"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
)

var _ machinery.Template = &Kustomization{}

// Kustomization scaffolds a file that defines the kustomization scheme for the prometheus folder
type Kustomization struct {
machinery.TemplateMixin
}

// SetTemplateDefaults implements file.Template
func (f *Kustomization) SetTemplateDefaults() error {
if f.Path == "" {
f.Path = filepath.Join("config", "policy", "kustomization.yaml")
}

f.TemplateBody = kustomizationTemplate

return nil
}

const kustomizationTemplate = `resources:
- allow-metrics-traffic.yaml
`
Loading

0 comments on commit 675b8b6

Please sign in to comment.