CMP-2400: exclusion of the namespace for `resource-requests-quota-per-project` #12344

Vincent056 · 2024-08-27T14:17:56Z

We have excluded RHACS from the default namespace check. We don't want to relax our rule too much for the optional operators, as it might increase security risk, instead, the user should use a tailoredprofile to set the exclusion regex.

Added a new variable var_resource_requests_quota_per_project_exempt_regex

CMP-2400

github-actions · 2024-08-27T14:19:25Z

Start a new ephemeral environment with changes proposed in this pull request:

ocp4 (from CTF) Environment (using Fedora as testing environment)

Fedora Testing Environment

Oracle Linux 8 Environment

applications/openshift/general/resource_requests_quota_per_project/rule.yml

Vincent056 · 2024-08-28T02:12:47Z

/test 4.15-e2e-aws-ocp4-moderate
/test 4.16-e2e-aws-ocp4-moderate

github-actions · 2024-08-28T02:31:39Z

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:12344
This image was built from commit: f7c7d3b

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:12344

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:12344 make deploy-local

yuumasato · 2024-08-28T07:36:36Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig

yuumasato · 2024-08-28T12:44:07Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig

Vincent056 · 2024-08-28T16:14:34Z

deployments.apps "compliance-operator" not found

Vincent056 · 2024-08-28T16:14:55Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig

Vincent056 · 2024-08-29T05:35:32Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig

xiaojiey · 2024-08-29T11:58:23Z

@Vincent056 Did I missed anything? Why the variable does not work? Thanks.

% oc get rule upstream-ocp4-resource-requests-quota-per-project  -o=jsonpath={.instructions}
Verify that the every non-control plane namespace has an appropriate ResourceQuota.

To get all the non-control plane namespaces, you can do the
following command $ oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and (true)) | .metadata.name]'

To get all the non-control plane namespaces with a ResourceQuota, you can do the
following command $ oc get --all-namespaces resourcequota -o json | jq '[.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default" and .metadata.namespace != "rhacs-operator" and (true)) | .metadata.namespace] | unique'

Make sure that the namespaces displayed in the commands of the commands match.
Is it the case that Resource requests and limits is not set per project?

% oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and (true)) | .metadata.name]'
[
  "test1",
  "test2"
]
% oc get --all-namespaces resourcequota -o json | jq '[.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default" and .metadata.namespace != "rhacs-operator" and (true)) | .metadata.namespace] | unique'
[]
% cat tp_scc.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: quota-test
  namespace: openshift-compliance
spec:
  description: Test
  setValues:
  - name: upstream-ocp4-var-resource-requests-quota-per-project-exempt-regex
    value: test1|test2
    rationale: test
  extends: upstream-ocp4-stig
  title: My modified nist profile with a custom value
% cat ssb_stig_quota.yaml
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: ocp4-stig-compliance
  namespace: openshift-compliance
profiles:
  - name: quota-test
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
  % oc apply -f tp_scc.yaml 
tailoredprofile.compliance.openshift.io/quota-test created
% oc apply -f ssb_stig_quota.yaml 
scansettingbinding.compliance.openshift.io/ocp4-stig-compliance configured
% oc get suite -w
NAME                   PHASE     RESULT
ocp4-stig-compliance   RUNNING   NOT-AVAILABLE
ocp4-stig-compliance   AGGREGATING   NOT-AVAILABLE
ocp4-stig-compliance   DONE          NON-COMPLIANT
ocp4-stig-compliance   DONE          NON-COMPLIANT
^C%                                                                                                                                                                                                            % oc get ccr | grep resource-requests-quota-per-project
quota-test-resource-requests-quota-per-project                       FAIL     medium

Vincent056 · 2024-08-29T16:55:04Z

let me test that

xiaojiey · 2024-09-02T12:59:21Z

/hold for test

github-actions · 2024-09-04T09:16:09Z

This datastream diff is auto generated by the check Compare DS/Generate Diff

Click here to see the full diff

New content has different text for rule 'xccdf_org.ssgproject.content_rule_resource_requests_quota_per_project'.
--- xccdf_org.ssgproject.content_rule_resource_requests_quota_per_project
+++ xccdf_org.ssgproject.content_rule_resource_requests_quota_per_project
@@ -31,15 +31,15 @@
 Therefore, you need to use a tool that can query the OCP API, retrieve the following:
 /api/v1/resourcequotas
     API endpoint, filter with with the jq utility using the following filter
-    [.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default") | .metadata.namespace] | unique
+    [.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default" and .metadata.namespace != "rhacs-operator" and ({{if ne .var_resource_requests_quota_per_project_exempt_regex "None"}}.metadata.namespace | test("{{.var_resource_requests_quota_per_project_exempt_regex}}") | not{{else}}true{{end}})) | .metadata.namespace] | unique
     and persist it to the local
-    /api/v1/resourcequotas#2e6fd6a1fa7945ee6d06434fd05e4c27822d577045b3d3a4ca5809580cd57f50
+    /api/v1/resourcequotas#4326a181a1e3e8a8e02ffb58e7d3ca9e62ed0e144a5277b1f7551fdbcfeca0a8
     file.
   /api/v1/namespaces
     API endpoint, filter with with the jq utility using the following filter
-    [.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default")]
+    [.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and ({{if ne .var_resource_requests_quota_per_project_exempt_regex "None"}}.metadata.name | test("{{.var_resource_requests_quota_per_project_exempt_regex}}") | not{{else}}true{{end}}))]
     and persist it to the local
-    /api/v1/namespaces#34d4beecc95c65d815d9d48fd4fdcb0c521631852ad088ef74e36d012b0e1e0d
+    /api/v1/namespaces#3ae63defe5cbb61225edb84d8e19f601be933d063305c1ea1e0381297c6258d6
     file.
 
 [reference]:

OVAL for rule 'xccdf_org.ssgproject.content_rule_resource_requests_quota_per_project' differs.
--- oval:ssg-resource_requests_quota_per_project:def:1
+++ oval:ssg-resource_requests_quota_per_project:def:1
@@ -1,4 +1,8 @@
 criteria AND
 criterion oval:ssg-test_file_for_resource_requests_quota_per_project:tst:1
 criterion oval:ssg-test_file_for_resource_requests_quotas_filtered_namespaces:tst:1
+criteria OR
 criterion oval:ssg-test_elements_count_for_resource_requests_quota_per_project:tst:1
+criteria AND
+criterion oval:ssg-test_resource_requests_quota_per_project:tst:1
+criterion oval:ssg-test_resource_requests_quotas_filtered_namespaces:tst:1

OCIL for rule 'xccdf_org.ssgproject.content_rule_resource_requests_quota_per_project' differs.
--- ocil:ssg-resource_requests_quota_per_project_ocil:questionnaire:1
+++ ocil:ssg-resource_requests_quota_per_project_ocil:questionnaire:1
@@ -1,10 +1,10 @@
 Verify that the every non-control plane namespace has an appropriate ResourceQuota.
 
 To get all the non-control plane namespaces, you can do the
-following command $ oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default") | .metadata.name ]'
+following command $ oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and ({{if ne .var_resource_requests_quota_per_project_exempt_regex "None"}}.metadata.name | test("{{.var_resource_requests_quota_per_project_exempt_regex}}") | not{{else}}true{{end}})) | .metadata.name]'
 
 To get all the non-control plane namespaces with a ResourceQuota, you can do the
-following command $ oc get --all-namespaces resourcequota -o json | jq '[.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default") | .metadata.namespace] | unique'
+following command $ oc get --all-namespaces resourcequota -o json | jq '[.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default" and .metadata.namespace != "rhacs-operator" and ({{if ne .var_resource_requests_quota_per_project_exempt_regex "None"}}.metadata.namespace | test("{{.var_resource_requests_quota_per_project_exempt_regex}}") | not{{else}}true{{end}})) | .metadata.namespace] | unique'
 
 Make sure that the namespaces displayed in the commands of the commands match.
       Is it the case that Resource requests and limits is not set per project?

xiaojiey · 2024-09-04T12:18:08Z

Verification pass with content image ghcr.io/complianceascode/k8scontent:12344:

% cat tp_scc.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: quota-test
  namespace: openshift-compliance
spec:
  description: Test
  setValues:
  - name: upstream-ocp4-var-resource-requests-quota-per-project-exempt-regex
    value: test1|test2
    rationale: test
  extends: upstream-ocp4-stig
  title: My modified nist profile with a custom value
% cat ssb_stig_quota.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: ocp4-stig-compliance
  namespace: openshift-compliance
profiles:
  - name: quota-test
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1                                                                                                                                                                                                       
% oc apply -f ssb_stig_quota.yaml 
scansettingbinding.compliance.openshift.io/ocp4-stig-compliance created
% oc get suite -w
NAME                   PHASE     RESULT
ocp4-stig-compliance   RUNNING   NOT-AVAILABLE
ocp4-stig-compliance   AGGREGATING   NOT-AVAILABLE
ocp4-stig-compliance   DONE          NON-COMPLIANT
ocp4-stig-compliance   DONE          NON-COMPLIANT
^C%                                                                                                                                                                                                                 % oc get ccr quota-test-resource-requests-quota-per-project
NAME                                             STATUS   SEVERITY
quota-test-resource-requests-quota-per-project   PASS     medium
% oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and (true)) | .metadata.name]'
[
  "test1",
  "test2"
]
% oc get ccr quota-test-resource-requests-quota-per-project -o=jsonpath={.instructions}
Verify that the every non-control plane namespace has an appropriate ResourceQuota.

To get all the non-control plane namespaces, you can do the
following command $ oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and (.metadata.name | test("test1|test2") | not)) | .metadata.name]'

To get all the non-control plane namespaces with a ResourceQuota, you can do the
following command $ oc get --all-namespaces resourcequota -o json | jq '[.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default" and .metadata.namespace != "rhacs-operator" and (.metadata.namespace | test("test1|test2") | not)) | .metadata.namespace] | unique'

Make sure that the namespaces displayed in the commands of the commands match.
Is it the case that Resource requests and limits is not set per project?%                                                                                                                                           % oc get --all-namespaces resourcequota -o json | jq '[.items[] | select((.metadata.namespace | startswith("openshift") | not) and (.metadata.namespace | startswith("kube-") | not) and .metadata.namespace != "default" and .metadata.namespace != "rhacs-operator" and (.metadata.namespace | test("test1|test2") | not)) | .metadata.namespace] | unique'
[]

% oc get  namespaces -o json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and (.metadata.name | test("test1|test2") | not)) | .metadata.name]'
[]

yuumasato

/lgtm

yuumasato · 2024-09-04T14:50:49Z

/unhold
Based on previous comment

yuumasato · 2024-09-04T14:53:49Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig

yuumasato · 2024-09-04T17:31:24Z

@Vincent056 The assertion files for STIG need to be updated as well.

    helpers.go:872: Result - Name: e2e-stig-resource-requests-quota-per-project - Status: PASS - Severity: medium
    helpers.go:879: E2E-FAILURE: The expected result for the e2e-stig-resource-requests-quota-per-project rule didn't match. Expected 'FAIL', Got 'PASS'

yuumasato · 2024-09-05T14:28:49Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig
/test 4.17-e2e-aws-ocp4-stig

Vincent056 · 2024-09-06T08:05:16Z

vincent@node:~$ cat namespaces.json | jq '[.items[] | select((.metadata.name | startswith("openshift") | not) and (.metadata.name | startswith("kube-") | not) and .metadata.name != "default" and .metadata.name != "rhacs-operator" and (.metadata.name | test("<no value>") | not))]'
[
  {
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
      "annotations": {
        "openshift.io/description": "",
        "openshift.io/display-name": "",
        "openshift.io/sa.scc.mcs": "s0:c27,c9",
        "openshift.io/sa.scc.supplemental-groups": "1000720000/10000",
        "openshift.io/sa.scc.uid-range": "1000720000/10000"
      },
      "creationTimestamp": "2024-09-05T15:16:55Z",
      "labels": {
        "kubernetes.io/metadata.name": "e2e-test",
        "pod-security.kubernetes.io/audit": "restricted",
        "pod-security.kubernetes.io/audit-version": "v1.24",
        "pod-security.kubernetes.io/enforce": "restricted",
        "pod-security.kubernetes.io/enforce-version": "v1.24",
        "pod-security.kubernetes.io/warn": "restricted",
        "pod-security.kubernetes.io/warn-version": "v1.24"
      },
      "name": "e2e-test",
      "resourceVersion": "34631",
      "uid": "1e45dfc3-d45a-48f6-af12-1d6c4227a5fb"
    },
    "spec": {
      "finalizers": [
        "kubernetes"
      ]
    },
    "status": {
      "phase": "Active"
    }
  }
]
vincent@node:~$

the second failed was triggered by https://github.com/ComplianceAsCode/content/blob/b04eec41b6aa4f8059e5514886285262285c3d5d/applications/openshift/networking/configure_network_policies_namespaces/tests/ocp4/e2e-remediation.sh

…quota-per-project Excluded RHACS from the default namespace check. We don't want to releax our rule too much for optional operator, as it might incress security risk, instead user should use tailoredprofile to set the exclusion regex. Added a new variable 'var_resource_requests_quota_per_project_exempt_regex'

This is to fix the e2e failures of resource_requests_quota_per_project caused by manual remediation of antoher rule/

Vincent056 · 2024-09-06T08:23:58Z

/test 4.15-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig
/test 4.17-e2e-aws-ocp4-stig

codeclimate · 2024-09-06T09:04:13Z

Code Climate has analyzed commit 6b62c35 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.5% (0.0% change).

View more on Code Climate.

openshift-ci · 2024-09-06T10:32:54Z

@Vincent056: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/4.17-e2e-aws-ocp4-stig	`6b62c35`	link	true	`/test 4.17-e2e-aws-ocp4-stig`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

yuumasato

/lgtm

Vincent056 mentioned this pull request Aug 27, 2024

OCPBUGS-304: exclude rhacs-operator namespace from resource limit rules #12307

Merged

Vincent056 changed the title ~~CMP-2400: exclusion of the namespace for resource-requests-quota-per-project~~ WIP CMP-2400: exclusion of the namespace for resource-requests-quota-per-project Aug 27, 2024

openshift-ci bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Aug 27, 2024

yuumasato reviewed Aug 27, 2024

View reviewed changes

Vincent056 force-pushed the cmp-2400 branch from 55401dc to c741919 Compare August 28, 2024 02:11

Vincent056 changed the title ~~WIP CMP-2400: exclusion of the namespace for resource-requests-quota-per-project~~ CMP-2400: exclusion of the namespace for resource-requests-quota-per-project Aug 28, 2024

openshift-ci bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Aug 28, 2024

yuumasato added the OpenShift OpenShift product related. label Aug 28, 2024

yuumasato requested review from xiaojiey and BhargaviGudi August 28, 2024 14:17

Vincent056 force-pushed the cmp-2400 branch from c741919 to aeb1dd5 Compare August 29, 2024 05:35

Mab879 added this to the 0.1.75 milestone Aug 29, 2024

openshift-ci bot added the do-not-merge/hold Used by openshift-ci-robot bot. label Sep 2, 2024

Vincent056 force-pushed the cmp-2400 branch from aeb1dd5 to dd1d55f Compare September 4, 2024 09:14

yuumasato approved these changes Sep 4, 2024

View reviewed changes

openshift-ci bot removed the do-not-merge/hold Used by openshift-ci-robot bot. label Sep 4, 2024

yuumasato self-assigned this Sep 4, 2024

Vincent056 force-pushed the cmp-2400 branch from dd1d55f to 3d06a8e Compare September 5, 2024 10:35

Vincent056 added 2 commits September 6, 2024 01:23

Fix e2e manual remediation for resource quota

6b62c35

This is to fix the e2e failures of resource_requests_quota_per_project caused by manual remediation of antoher rule/

Vincent056 force-pushed the cmp-2400 branch from f7c7d3b to 6b62c35 Compare September 6, 2024 08:23

yuumasato approved these changes Sep 6, 2024

View reviewed changes

yuumasato merged commit 7d11b13 into ComplianceAsCode:master Sep 6, 2024
96 of 103 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CMP-2400: exclusion of the namespace for `resource-requests-quota-per-project` #12344

CMP-2400: exclusion of the namespace for `resource-requests-quota-per-project` #12344

Vincent056 commented Aug 27, 2024 •

edited

Loading

github-actions bot commented Aug 27, 2024 •

edited

Loading

Vincent056 commented Aug 28, 2024

github-actions bot commented Aug 28, 2024 •

edited

Loading

yuumasato commented Aug 28, 2024

yuumasato commented Aug 28, 2024

Vincent056 commented Aug 28, 2024

Vincent056 commented Aug 28, 2024

Vincent056 commented Aug 29, 2024

xiaojiey commented Aug 29, 2024

Vincent056 commented Aug 29, 2024

xiaojiey commented Sep 2, 2024

github-actions bot commented Sep 4, 2024

xiaojiey commented Sep 4, 2024

yuumasato left a comment

yuumasato commented Sep 4, 2024

yuumasato commented Sep 4, 2024

yuumasato commented Sep 4, 2024

yuumasato commented Sep 5, 2024

Vincent056 commented Sep 6, 2024 •

edited

Loading

Vincent056 commented Sep 6, 2024

codeclimate bot commented Sep 6, 2024

openshift-ci bot commented Sep 6, 2024

yuumasato left a comment

CMP-2400: exclusion of the namespace for resource-requests-quota-per-project #12344

CMP-2400: exclusion of the namespace for resource-requests-quota-per-project #12344

Conversation

Vincent056 commented Aug 27, 2024 • edited Loading

github-actions bot commented Aug 27, 2024 • edited Loading

Vincent056 commented Aug 28, 2024

github-actions bot commented Aug 28, 2024 • edited Loading

yuumasato commented Aug 28, 2024

yuumasato commented Aug 28, 2024

Vincent056 commented Aug 28, 2024

Vincent056 commented Aug 28, 2024

Vincent056 commented Aug 29, 2024

xiaojiey commented Aug 29, 2024

Vincent056 commented Aug 29, 2024

xiaojiey commented Sep 2, 2024

github-actions bot commented Sep 4, 2024

xiaojiey commented Sep 4, 2024

yuumasato left a comment

Choose a reason for hiding this comment

yuumasato commented Sep 4, 2024

yuumasato commented Sep 4, 2024

yuumasato commented Sep 4, 2024

yuumasato commented Sep 5, 2024

Vincent056 commented Sep 6, 2024 • edited Loading

Vincent056 commented Sep 6, 2024

codeclimate bot commented Sep 6, 2024

openshift-ci bot commented Sep 6, 2024

yuumasato left a comment

Choose a reason for hiding this comment

CMP-2400: exclusion of the namespace for `resource-requests-quota-per-project` #12344

CMP-2400: exclusion of the namespace for `resource-requests-quota-per-project` #12344

Vincent056 commented Aug 27, 2024 •

edited

Loading

github-actions bot commented Aug 27, 2024 •

edited

Loading

github-actions bot commented Aug 28, 2024 •

edited

Loading

Vincent056 commented Sep 6, 2024 •

edited

Loading