⚠️ replace the kube-rbac-proxy usage with NetworkPolicy

.github/workflows/test-sample-go.yml

@@ -39,8 +39,8 @@ jobs:
           KUSTOMIZATION_FILE_PATH="testdata/project-v4/config/default/kustomization.yaml"
           sed -i '25s/^#//' $KUSTOMIZATION_FILE_PATH
           sed -i '27s/^#//' $KUSTOMIZATION_FILE_PATH
-          sed -i '42s/^#//' $KUSTOMIZATION_FILE_PATH
-          sed -i '46,143s/^#//' $KUSTOMIZATION_FILE_PATH
+          sed -i '45s/^#//' $KUSTOMIZATION_FILE_PATH
+          sed -i '49,145s/^#//' $KUSTOMIZATION_FILE_PATH
       
       - name: Test
         run: |

docs/book/src/reference/metrics.md

@@ -3,13 +3,54 @@
 By default, controller-runtime builds a global prometheus registry and
 publishes [a collection of performance metrics](/reference/metrics-reference.md) for each controller.
 
+## Enabling the Metrics
+
+First, you will need enable the Metrics by uncommenting the following line
+in the file `config/default/kustomization.yaml`
+
+```sh
+# [Metrics] The /metrics endpoint is protected by the NetworkPolicy
+# If you want to expose the metric endpoint of your controller-manager
+# uncomment the following line.
+# - path: manager_metrics_patch.yaml
+```
+
 ## Protecting the Metrics
 
-These metrics are protected by [kube-rbac-proxy](https://github.com/brancz/kube-rbac-proxy)
-by default if using kubebuilder. Kubebuilder v2.2.0+ scaffold a clusterrole which
-can be found at `config/rbac/auth_proxy_client_clusterrole.yaml`.
+These metrics are protected since the release `v3.15.0+` [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
+by default if using Kubebuilder. Kubebuilder scaffold a clusterrole which
+can be found at `config/rbac/metrics_client_cluster_role.yaml`.
+
+<aside class="note warning">
+<h1>Notice for Projects Scaffolded with versions < v3.15.0 and `kube-rbac-proxy`</h1>
+
+Projects built with Kubebuilder versions prior to `v3.15.0`
+utilize [`kube-rbac-proxy`](https://github.com/brancz/kube-rbac-proxy)
+to protect the metrics endpoint.
+
+Kubebuilder has been rebuilding and re-tagging these images for several years.
+However, due to recent infrastructure changes for projects under the Kubernetes umbrella,
+we now require the use of shared infrastructure.
+
+As [`kube-rbac-proxy`](https://github.com/brancz/kube-rbac-proxy) is a process to be donated but
+still not part of kubernetes-sigs, sadly we cannot build and promote these images using this infrastructure.
+Additionally, Google Cloud Platform has deprecated the
+[Container Registry](https://cloud.google.com/artifact-registry/docs/transition/prepare-gcr-shutdown), which
+has been used to promote these images. Ongoing changes and the phase-out of the previous
+GCP infrastructure ([additional information](https://github.com/kubernetes/k8s.io/issues/2647))
+mean that we are no longer able to support, build, or ensure the promotion of these images.
+
+**If you are using OR wish to continue using `kube-rbac-proxy`, we recommend checking the project
+to use the images which are provided by the project.**
 
-You will need to grant permissions to your Prometheus server so that it can
+For the future, with the [Plugins API provided by Kubebuilder](./../plugins/plugins.md),
+it might be possible to create an [external plugin](./../plugins/creating-plugins.md)
+to properly integrate this solution with Kubebuilder and
+provide a helper to allow users to opt-in as please them.
+
+</aside>
+
+Then, you will need to grant permissions to your Prometheus server so that it can
 scrape the protected metrics. To achieve that, you can create a
 `clusterRoleBinding` to bind the `clusterRole` to the service account that your
 Prometheus server uses. If you are using [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus),
@@ -91,6 +132,21 @@ for the metrics exported from the namespace where the project is running
 
 <img width="1680" alt="Screenshot 2019-10-02 at 13 07 13" src="https://user-images.githubusercontent.com/7708031/66042888-a497da80-e515-11e9-9d77-d8a9fc1159a5.png">
 
+## Consuming the Metrics from other Pods.
+
+Note that the only Pods with the label `role: metrics` are able
+to consume the metrics since they are protected by the
+[Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
+which can be found under `config/policy`.
+
+Following an example to create a Pod using Curl to reach out the metrics:
+
+```sh
+kubectl run curl --restart=Never \
+  -n my-project-system --image=curlimages/curl \
+  --labels="role=metrics" --command -- sh -c "curl -v http://<my-project>-controller-manager-metrics-service.<my-project>-system.svc.cluster.local:8080/metrics"
+```
+
 ## Publishing Additional Metrics
 
 If you wish to publish additional metrics from your controllers, this

hack/docs/internal/cronjob-tutorial/generate_cronjob.go

@@ -583,13 +583,6 @@ func updateExample(sp *Sample) {
 		filepath.Join(sp.ctx.Dir, "config/samples/batch_v1_cronjob.yaml"),
 		`# TODO(user): Add fields here`, "")
 	CheckError("fixing samples/batch_v1_cronjob.yaml", err)
-
-	// update default/manager_auth_proxy_patch.yaml
-	err = pluginutil.InsertCode(
-		filepath.Join(sp.ctx.Dir, "config/default/manager_auth_proxy_patch.yaml"),
-		` template:
-    spec:`, ManagerAuthProxySample)
-	CheckError("fixing default/manager_auth_proxy_patch.yaml", err)
 }
 
 func addControllerTest(sp *Sample) {

pkg/plugin/util/util.go

@@ -80,6 +80,36 @@ func InsertCode(filename, target, code string) error {
 	return os.WriteFile(filename, []byte(out), 0644)
 }
 
+// AppendCodeIfNotExist checks if the code does not already exist in the file, and if not, appends it to the end.
+func AppendCodeIfNotExist(filename, code string) error {
+	contents, err := os.ReadFile(filename)
+	if err != nil {
+		return err
+	}
+
+	if strings.Contains(string(contents), code) {
+		return nil // Code already exists, no need to append.
+	}
+
+	return AppendCodeAtTheEnd(filename, code)
+}
+
+// AppendCodeAtTheEnd appends the given code at the end of the file.
+func AppendCodeAtTheEnd(filename, code string) error {
+	f, err := os.OpenFile(filename, os.O_APPEND|os.O_WRONLY, 0644)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if err := f.Close(); err != nil {
+			return
+		}
+	}()
+
+	_, err = f.WriteString(code)
+	return err
+}
+
 // InsertCodeIfNotExist insert code if it does not already exists
 func InsertCodeIfNotExist(filename, target, code string) error {
 	// false positive

pkg/plugins/common/kustomize/v2/scaffolds/api.go

@@ -102,28 +102,37 @@ func (s *apiScaffolder) Scaffold() error {
 
 		// Add scaffolded CRD Editor and Viewer roles in config/rbac/kustomization.yaml
 		rbacKustomizeFilePath := "config/rbac/kustomization.yaml"
-		comment := `
-# For each CRD, "Editor" and "Viewer" roles are scaffolded by
-# default, aiding admins in cluster management. Those roles are
-# not used by the Project itself. You can comment the following lines
-# if you do not want those helpers be installed with your Project.`
-		err = pluginutil.InsertCodeIfNotExist(rbacKustomizeFilePath,
-			"- auth_proxy_client_clusterrole.yaml", comment)
+		err = pluginutil.AppendCodeIfNotExist(rbacKustomizeFilePath,
+			editViewRulesCommentFragment)
 		if err != nil {
-			log.Errorf("Unable to add a comment in the file "+
+			log.Errorf("Unable to append the edit/view roles editViewRulesCommentFragment in the file "+
 				"%s.", rbacKustomizeFilePath)
 		}
 		crdName := strings.ToLower(s.resource.Kind)
 		if s.config.IsMultiGroup() && s.resource.Group != "" {
 			crdName = strings.ToLower(s.resource.Group) + "_" + crdName
 		}
-		err = pluginutil.InsertCodeIfNotExist(rbacKustomizeFilePath, comment,
+		err = pluginutil.InsertCodeIfNotExist(rbacKustomizeFilePath, editViewRulesCommentFragment,
 			fmt.Sprintf("\n- %[1]s_editor_role.yaml\n- %[1]s_viewer_role.yaml", crdName))
 		if err != nil {
 			log.Errorf("Unable to add Editor and Viewer roles in the file "+
 				"%s.", rbacKustomizeFilePath)
 		}
+		// Add an empty line at the end of the file
+		err = pluginutil.AppendCodeIfNotExist(rbacKustomizeFilePath,
+			`
+
+`)
+		if err != nil {
+			log.Errorf("Unable to append empty line at the end of the file"+
+				"%s.", rbacKustomizeFilePath)
+		}
 	}
 
 	return nil
 }
+
+const editViewRulesCommentFragment = `# For each CRD, "Editor" and "Viewer" roles are scaffolded by
+# default, aiding admins in cluster management. Those roles are
+# not used by the Project itself. You can comment the following lines
+# if you do not want those helpers be installed with your Project.`

pkg/plugins/common/kustomize/v2/scaffolds/init.go

@@ -24,6 +24,7 @@ import (
 	"sigs.k8s.io/kubebuilder/v3/pkg/plugins"
 	"sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/kdefault"
 	"sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/manager"
+	"sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/policy"
 	"sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/prometheus"
 	"sigs.k8s.io/kubebuilder/v3/pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/rbac"
 )
@@ -64,10 +65,10 @@ func (s *initScaffolder) Scaffold() error {
 
 	templates := []machinery.Builder{
 		&rbac.Kustomization{},
-		&rbac.AuthProxyRole{},
-		&rbac.AuthProxyRoleBinding{},
-		&rbac.AuthProxyService{},
-		&rbac.AuthProxyClientRole{},
+		&rbac.MetricsRole{},
+		&rbac.MonitoringBinding{},
+		&rbac.MetricsService{},
+		&rbac.MetricsClientRole{},
 		&rbac.RoleBinding{},
 		// We need to create a Role because if the project
 		// has not CRD define the controller-gen will not generate this file
@@ -78,7 +79,9 @@ func (s *initScaffolder) Scaffold() error {
 		&manager.Kustomization{},
 		&manager.Config{Image: imageName},
 		&kdefault.Kustomization{},
-		&kdefault.ManagerAuthProxyPatch{},
+		&kdefault.ManagerMetricsPatch{},
+		&policy.Kustomization{},
+		&policy.NetworkPolicy{},
 		&kdefault.ManagerConfigPatch{},
 		&prometheus.Kustomization{},
 		&prometheus.Monitor{},

pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/kdefault/kustomization.go

@@ -71,12 +71,15 @@ resources:
 #- ../certmanager
 # [PROMETHEUS] To enable prometheus monitor, uncomment all sections with 'PROMETHEUS'.
 #- ../prometheus
+# [NETWORK POLICY] Protect the /metrics endpoint. If you want your controller-manager to expose
+# the /metrics w/o any authn/z, please comment the following line.
+- ../policy
 
 patches:
-# Protect the /metrics endpoint by putting it behind auth.
-# If you want your controller-manager to expose the /metrics
-# endpoint w/o any authn/z, please comment the following line.
-- path: manager_auth_proxy_patch.yaml
+# The /metrics endpoint is protected by the NetworkPolicy
+# If you want to expose the metric endpoint of your controller-manager
+# uncomment the following line.
+#- path: manager_metrics_patch.yaml
 
 {{ if .ComponentConfig -}}
 # Mount the controller config file for loading manager configurations

pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/kdefault/manager_matrics_patch.go

@@ -0,0 +1,61 @@
+/*
+Copyright 2020 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package kdefault
+
+import (
+	"path/filepath"
+
+	"sigs.k8s.io/kubebuilder/v3/pkg/machinery"
+)
+
+var _ machinery.Template = &ManagerMetricsPatch{}
+
+// ManagerMetricsPatch scaffolds a file that defines the patch that enables prometheus metrics for the manager
+type ManagerMetricsPatch struct {
+	machinery.TemplateMixin
+	machinery.ComponentConfigMixin
+}
+
+// SetTemplateDefaults implements file.Template
+func (f *ManagerMetricsPatch) SetTemplateDefaults() error {
+	if f.Path == "" {
+		f.Path = filepath.Join("config", "default", "manager_metrics_patch.yaml")
+	}
+
+	f.TemplateBody = kustomizeMetricsPatchTemplate
+
+	f.IfExistsAction = machinery.Error
+
+	return nil
+}
+
+const kustomizeMetricsPatchTemplate = `# This patch adds the args to allow expose the metrics endpoint
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: controller-manager
+  namespace: system
+spec:
+  template:
+    spec:
+      containers:
+      - name: manager
+        args:
+        - "--health-probe-bind-address=:8081"
+        - "--metrics-bind-address=0.0.0.0:8080"
+        - "--leader-elect"
+`