Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Provides documentation on Kubearmor container visibility #1028

Merged
merged 3 commits into from
Jan 30, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
300 changes: 300 additions & 0 deletions getting-started/kubearmor_visibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,300 @@
# KubeArmor Visibility

KubeArmor currently supports enabling visibility for containers and hosts.

Visibility for hosts is not enabled by default, however it is enabled by default for containers .

The `karmor` tool provides access to both using `karmor logs`.

<details>
<summary>Available visibility options:</summary>

#### KubeArmor provides visibility on the following behavior of containers
* Process
* Files
* Networks

</details>



### Prerequisites

* If you don't have access to a K8s cluster, please follow [this](https://github.com/kubearmor/KubeArmor/blob/main/getting-started/deployment_guide.md#prerequisites) to set one up.
* karmor CLI tool: [Download and install karmor-cli](https://github.com/kubearmor/KubeArmor/blob/main/getting-started/deployment_guide.md#1-download-and-install-karmor-cli-tool)

### Example: wordpress-mysql

* To deploy [wordpress-mysql](https://github.com/kubearmor/KubeArmor/blob/main/examples/wordpress-mysql/wordpress-mysql-deployment.yaml) app follow [this](https://github.com/kubearmor/KubeArmor/blob/main//examples/wordpress-mysql.md)
* Now we need to deploy some sample policies
```
kubectl apply -f https://github.com/raw/kubearmor/KubeArmor/main/examples/wordpress-mysql/security-policies/ksp-wordpress-block-process.yaml
```
This sample policy blocks execution of the `apt` and `apt-get` commands in wordpress pods with label selector `app: wordpress`.

### Getting Container Visibility


* Checking default visibility

* Container visibility is enabled by default. We can check it using `kubectl describe` and grep `kubearmor-visibility`

```text
POD_NAME=$(kubectl get pods -n wordpress-mysql -l app=wordpress -o jsonpath='{.items[0].metadata.name}') && kubectl describe -n wordpress-mysql pod $POD_NAME | grep kubearmor-visibility

kubearmor-visibility: process, file, network, capabilities
```
* **For pre-existing workloads :** Enable visibility using `kubectl annotate`. Currently KubeArmor supports `process`, `file`, `network`, `capabilities`
```text
kubectl annotate pods <pod-name> -n wordpress-mysql "kubearmor-visibility=process,file,network,capabilities"
```
* Open up a terminal, and watch logs using the `karmor` cli
```text
karmor logs
```
* In another terminal, simulate a policy violation . Try `sleep` inside a pod

```text
POD_NAME=$(kubectl get pods -n wordpress-mysql -l app=wordpress -o jsonpath='{.items[0].metadata.name}') && kubectl -n wordpress-mysql exec -it $POD_NAME -- bash
# apt update
```
* In the terminal running `karmor logs`, the policy violation along with container visibility is shown, in this case for example
<details>
<summary>Click to expand</summary>

```text
== Alert / 2023-01-04 04:58:37.689182 ==
ClusterName: default
HostName: sibashi-asus
NamespaceName: wordpress-mysql
PodName: wordpress-787f45786f-mm2bm
Labels: app=wordpress
ContainerName: wordpress
ContainerID: 9af5246810fd0a732e74d391b32b95f65e4c08e655d1ab10b49b04b148cc1c24
ContainerImage: docker.io/library/wordpress:4.8-apache@sha256:6216f64ab88fc51d311e38c7f69ca3f9aaba621492b4f1fa93ddf63093768845
Type: MatchedPolicy
PolicyName: ksp-wordpress-block-process
Severity: 3
Source: /bin/bash
Resource: /usr/bin/apt update
Operation: Process
Action: Block
Data: syscall=SYS_EXECVE
Enforcer: AppArmor
Result: Permission denied
HostPID: 17462
HostPPID: 17293
PID: 199
PPID: 193
ParentProcessName: /bin/bash
ProcessName: /usr/bin/apt
```

</details>


* The logs can also be generated in JSON format using `karmor logs --json `

<details>
<summary>Click to expand</summary>

```json
{
"Timestamp":1672808328,
"UpdatedTime":"2023-01-04T04:58:48.838991Z",
"ClusterName":"default","HostName":"sibashi-asus",
"NamespaceName":"wordpress-mysql","PodName":"wordpress-787f45786f-mm2bm",
"Labels":"app=wordpress",
"ContainerID":"9af5246810fd0a732e74d391b32b95f65e4c08e655d1ab10b49b04b148cc1c24",
"ContainerName":"wordpress",
"ContainerImage":"docker.io/library/wordpress:4.8-apache@sha256:6216f64ab88fc51d311e38c7f69ca3f9aaba621492b4f1fa93ddf63093768845",
"HostPPID":17293,
"HostPID":17526,
"PPID":193,
"PID":200,
"ParentProcessName":"/bin/bash",
"ProcessName":"/usr/bin/apt",
"PolicyName":"ksp-wordpress-block-process",
"Severity":"3",
"Type":"MatchedPolicy",
"Source":"/bin/bash",
"Operation":"Process",
"Resource":"/usr/bin/apt update",
"Data":"syscall=SYS_EXECVE",
"Enforcer":"AppArmor",
"Action":"Block",
"Result":"Permission denied"
}

```

</details>

### Getting Host Visibility

* Host Visibility is not enabled by default . To enable Host Visibility we need to annotate the node using `kubectl annotate node`
```
kubectl annotate node <node-name> "kubearmor-visibility=process,file,network,capabilities"
```

* To confirm it use `kubectl describe` and grep `kubearmor-visibility`
```text
kubectl describe node <node-name> | grep kubearmor-visibility
```

* Now we can get general telemetry events in the context of the host using `karmor logs` .The logs related to Host Visibility will have type `Type: HostLog`and `Operation: File | Process | Network `

```text
karmor logs --logFilter=all
```

<details>
<summary>Click to expand</summary>

```text
== Alert / 2023-01-04 04:58:37.689182 ==
== Log / 2023-01-27 14:41:49.017709 ==
ClusterName: default
HostName: kubearmor-dev2
Type: HostLog
Source: /usr/bin/dockerd
Resource: /usr/bin/runc --version
Operation: Process
Data: syscall=SYS_EXECVE
Result: Passed
HostPID: 193088
HostPPID: 914
PID: 193088
PPID: 914
ParentProcessName: /usr/bin/dockerd
ProcessName: /usr/bin/runc
== Log / 2023-01-27 14:41:49.018951 ==
ClusterName: default
HostName: kubearmor-dev2
Type: HostLog
Source: /usr/bin/runc --version
Resource: /lib/x86_64-linux-gnu/libc.so.6
Operation: File
Data: syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC
Result: Passed
HostPID: 193088
HostPPID: 914
PID: 193088
PPID: 914
ParentProcessName: /usr/bin/dockerd
ProcessName: /usr/bin/runc
== Log / 2023-01-27 14:41:49.018883 ==
ClusterName: default
HostName: kubearmor-dev2
Type: HostLog
Source: /usr/bin/runc --version
Resource: /etc/ld.so.cache
Operation: File
Data: syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC
Result: Passed
HostPID: 193088
HostPPID: 914
PID: 193088
PPID: 914
ParentProcessName: /usr/bin/dockerd
ProcessName: /usr/bin/runc
== Log / 2023-01-27 14:41:49.020905 ==
ClusterName: default
HostName: kubearmor-dev2
Type: HostLog
Source: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/k3s
Resource: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/portmap
Operation: Process
Data: syscall=SYS_EXECVE
Result: Passed
HostPID: 193090
HostPPID: 5627
PID: 193090
PPID: 5627
ParentProcessName: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/k3s
ProcessName: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/portmap

```
* The logs can also be generated in JSON format using `karmor logs --logFilter=all --json `

<details>
<summary>Click to expand</summary>

```json

{
"Timestamp": 1674889485,
"UpdatedTime": "2023-01-28T07:04:45.436854Z",
"ClusterName": "default",
"HostName": "kubearmor-dev2",
"ParentProcessName": "/usr/bin/containerd-shim",
"ProcessName": "/usr/bin/containerd",
"HostPPID": 80989,
"HostPID": 614074,
"PPID": 80989,
"PID": 614074,
"Type": "HostLog",
"Source": "/usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby",
"Operation": "File",
"Resource": "/lib/x86_64-linux-gnu/libdl.so.2",
"Data": "syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC",
"Result": "Passed",
},
{
"Timestamp": 1674889485,
"UpdatedTime": "2023-01-28T07:04:45.440804Z",
"ClusterName": "default",
"HostName": "kubearmor-dev2",
"ParentProcessName": "/usr/bin/containerd-shim",
"ProcessName": "/usr/bin/containerd",
"HostPPID": 80989,
"HostPID": 614074,
"PPID": 80989,
"PID": 614074,
"Type": "HostLog",
"Source": "/usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby",
"Operation": "File",
"Resource": "/usr/bin/containerd",
"Data": "syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC",
"Result": "Passed",
},
{
"Timestamp": 1674889485,
"UpdatedTime": "2023-01-28T07:04:45.427658Z",
"ClusterName": "default",
"HostName": "kubearmor-dev2",
"ParentProcessName": "/bin/bash",
"ProcessName": "/usr/bin/pgrep",
"HostPPID": 614066,
"HostPID": 614073,
"PPID": 614066,
"PID": 614073,
"Type": "HostLog",
"Source": "/bin/bash",
"Operation": "Process",
"Resource": "/usr/bin/pgrep kubearmor",
"Data": "syscall=SYS_EXECVE",
"Result": "Passed",
},
{
"Timestamp": 1674889485,
"UpdatedTime": "2023-01-28T07:04:45.450454Z",
"ClusterName": "default",
"HostName": "kubearmor-dev2",
"ParentProcessName": "/usr/bin/containerd-shim",
"ProcessName": "/usr/bin/containerd",
"HostPPID": 80989,
"HostPID": 614074,
"PPID": 80989,
"PID": 614074,
"Type": "HostLog",
"Source": "/usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby",
"Operation": "Network",
"Resource": "domain=AF_UNIX type=SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC protocol=0",
"Data": "syscall=SYS_SOCKET",
"Result": "Passed"
}



```