feat: Add logic for Kyverno policy recommendation

[Vyom-Yadav]

Signed-off-by: Vyom-Yadav jackhammervyom@gmail.com

---

Kyverno policy recommendation is spread across 3 PRs including this one:

• Modify policy template schema for Kyverno policies kubearmor/policy-templates#1037
• feat: Connect with discovery-engine to recommend Kyverno policies kubearmor/kubearmor-client#303

---

Explanation:

Kyverno policy recommendation dynamically recommends Kyverno policies based on container runtime data.

The dynamically recommended policy right now:

• Restrict automounting of service account token.

---

How it works:

1. Policies are fetched from the latest release of https://github.com/kubearmor/policy-templates (the path is modified at present to my fork).
2. Algorithm determines whether the service account token is being used in pods in the deployment.
3. If it is unused, restrict-automount-sa-token is generated and inserted into the DB.
4. Since the policy recommendation is a cron job, if in subsequent runs, if container runtime data points towards usage of service account token, the policy for that particular deployment, if present, is deleted.

---

Nuances:

1. Policy present in policy-templates matches Pods, but we generate a policy for Deployments.

Kyverno autogen does not work when label selectors are used, see kyverno/kyverno#4410.

Since we right now only recommend policies for Deployments (this logic will be changed when we shift to owner references), autogen for conversion from Pod -> Deployment is internally implemented. This can be easily extended for other owner references too if Kyverno does not support autogen in the future too.

2. Kyverno policies are added to policy_yaml DB.

As internally discussed, we will be adding policies to policy_yaml so we can extract policies from a single place. Other generated policies are right now added to both policy_yaml and other DB's like system_policy, network_policy, etc. The worker protobuf fetches system policies from system_policy, network policies from network_policy, and admission controller policies from policy_yaml table. There is no admission_controller_policy table.

3. Worker has no control over admission controller policy recommendation.

The worker cannot stop or start the admission controller policy recommendation since it is a part of recommend cronjob. The recommend cronjob is automatically started when DE starts.

4. The algorithm makes a few assumptions while checking whether SA token is mounted or not.

• Removing part of path returned by summary matching regular expression ..[0-9]{4}_[0-9]{2}_[0-9]{2}.*:

This is a known k8s feature/issue, read more about it at https://stackoverflow.com/q/50685385/15412365. Since KubeArmor resolved symlinks, this is required to match VolumeMount path.

• Removing the first element of the path if the complete path doesn't match:

Sometimes when SA account token is accessed at /var/run/secrets/kubernetes.io/serviceaccount/token, the path shown in summary is /run/secrets/kubernetes.io/serviceaccount/..2023_03_28_12_23_09.412453730/token. This is most likely a bug in symlink resolution, the correct path while resolving symlinks should be /var/run/secrets/kubernetes.io/serviceaccount/..2023_03_28_12_23_09.412453730/token. Read the discussion at https://accuknox.slack.com/archives/C0229TD83RA/p1680013050747849

---

Process of merging PRs:

1. Merge policy templates PR and publish a new release, so this PR can leverage those. The code won't break even if no Kyverno policies are present in a release.
2. Merge this PR.
3. Merge kubearmor-client PR.

---

Testing:

• Unit tests for validating policy recommendation algorithms have been added.

[Vyom-Yadav]

Final recommended policy example:

apiVersion: kyverno.io/v1
kind: Policy
metadata:
  annotations:
    policies.kyverno.io/minversion: 1.6.0
    recommended-policies.kubearmor.io/description: Don't mount service account token
      when it is not needed
    recommended-policies.kubearmor.io/description-detailed: If the Service Account
      Token is not used by a pod, then it should not be automounted. Service account
      token provide access to the kubeapi-server which potentially increases the surface
      area of attack.
    recommended-policies.kubearmor.io/tags: AUTOMOUNT_SERVICE_ACCOUNT
    recommended-policies.kubearmor.io/title: Restrict Auto-Mount of Service Account
      Tokens
  name: nginx-restrict-automount-sa-token
  namespace: default
spec:
  background: true
  rules:
  - match:
      any:
      - resources:
          kinds:
          - Deployment
    name: validate-automountServiceAccountToken
    preconditions:
      all:
      - key: '{{ request.operation || ''BACKGROUND'' }}'
        operator: NotEquals
        value: DELETE
      - key: '{{ request.object.spec.template.metadata.labels.app || '''' }}'
        operator: Equals
        value: nginx
      - key: '{{ request.object.spec.template.metadata.labels.foo || '''' }}'
        operator: Equals
        value: faa
    validate:
      message: Auto-mounting of Service Account tokens is not allowed.
      pattern:
        spec:
          template:
            spec:
              automountServiceAccountToken: "false"
  validationFailureAction: Audit

[nyrahul]

Awesome PR. LGTM. 👍

[vishnusomank]

LGTM!