diff --git a/docs/css/extra.css b/docs/css/extra.css index 17c0e0f5..f30c4cc0 100644 --- a/docs/css/extra.css +++ b/docs/css/extra.css @@ -30,4 +30,12 @@ font-weight: bold; opacity: 1; color: white; -} \ No newline at end of file +} + +.md-typeset__table { + min-width: 100%; + } + + .md-typeset table:not([class]) { + display: table; + } diff --git a/docs/dev-guide/testing.md b/docs/dev-guide/testing.md index 0a790bae..65633583 100644 --- a/docs/dev-guide/testing.md +++ b/docs/dev-guide/testing.md @@ -28,7 +28,7 @@ The repository includes a suite of system tests that will do the following: - run KubeHound using the file collector to create a working graph database - query the graph database to ensure all expected vertices and edges have been created correctly -The cluster setup and running instances can be found under [test/setup](./test/setup/) +The cluster setup and running instances can be found under [test/setup](https://github.com/DataDog/KubeHound/tree/main/test/setup) If you need to manually access the system test environment with kubectl and other commands, you'll need to set (assuming you are at the root dir): diff --git a/docs/khaas/advanced-configuration.md b/docs/khaas/advanced-configuration.md new file mode 100644 index 00000000..0a1ba989 --- /dev/null +++ b/docs/khaas/advanced-configuration.md @@ -0,0 +1,52 @@ +# Advanced configuration + +This section covers all the available flags in KubeHound. +!!! note + + If you don't want to specify the bucket every time, you can set it up in your local config file (`./kubehound.yaml` or `$HOME/.config/kubehound.yaml`). See [getting started with KHaaS](./getting-started.md) + +## Manual collection + +In order to use `kubehound` with KHaaS, you need to specify the api endpoint you want to use: + +- `--khaas-server` from the inline flags (by default `127.0.0.1:9000`) + +## Dump and ingest + +In order to use the collector with KHaaS you need to specify the dump location for the k8s resources: + +- `--bucket_url` from the inline flags (i.e. `s3://`). There is no default value for security reason. +- `--region` from the inline flags (i.e. `us-east-1`) to set the region to retrieve the configuration (only for s3). + +!!! warning + + The `kubehound` binary needs to have push access to your cloud storage provider. + +To dump and ingest the current k8s cluster, you just need to run the following command (i.e. using an AWS config): + +```bash +kubehound dump remote --khaas-server 127.0.0.1:9000 --insecure --bucket_url s3:// --region us-east-1 +``` + +The last command will: + +- **dump the k8s resources** to the cloud storage provider. +- send a grpc call to **run the ingestion on the KHaaS** grpc endpoint. + +!!! note + + The ingestion will dump the current cluster being setup, if you want to skip the interactive mode, just specify `-y` or `--non-interactive` + +### Manual ingestion + +If you want to rehydrate (reingesting all the latest clusters dumps), you can use the `ingest` command to run it. + +```bash +kubehound ingest remote --khaas-server 127.0.0.1:9000 --insecure +``` + +You can also specify a specific dump by using the `--cluster` and `run_id` flags. + +```bash +kubehound ingest remote --khaas-server 127.0.0.1:9000 --insecure --cluster my-cluster-1 --run_id 01htdgjj34mcmrrksw4bjy2e94 +``` diff --git a/docs/khaas/deployment.md b/docs/khaas/deployment.md new file mode 100644 index 00000000..87661904 --- /dev/null +++ b/docs/khaas/deployment.md @@ -0,0 +1,71 @@ +# Deploying KHaaS - Ingestor stack + +!!! warning "deprecated" + + The `kubehound-ingestor` has been deprecated since **v1.5.0** and renamed to `kubehound-binary`. + +## Docker deployment + +To run the KubeHound as a Service with `docker` just use the following [compose files](https://github.com/DataDog/KubeHound/tree/main/deployments/kubehound): + +```bash +cd ./deployments/kubehound +docker compose -f docker-compose.yaml -f docker-compose.release.yaml -f docker-compose.release.ingestor.yaml --profile jupyter up -d +``` + +By default the endpoints are only exposed locally: + +- `127.0.0.1:9000` for ingestor endpoint. +- `127.0.0.1:8888` for the UI. + +For the UI 2 profiles (`--profile`) are available, you need to pick one: + +- `jupyter` to spawn a Jupyter backend compatible with Janusgraph endpoint ([aws graph-notebook](https://github.com/aws/graph-notebook)). +- `invana` to spawn the [Invana Studio](https://github.com/invana/invana-studio), a dedicated UI for Janusgraph (this is also deploying the invana backend). **We do not encourage to use as it is not maintained anymore**. + +!!! warning + + You should change the default password by editing `NOTEBOOK_PASSWORD=` in the `docker-compose.yaml` + +## k8s deployment + +To run the KubeHound as a Service on Kubernetes just use the following [helm files](https://github.com/DataDog/KubeHound/tree/main/deployments/k8s): + +```bash +cd ./deployments/k8s +helm install khaas khaas --namespace khaas --create-namespace +``` + +If it succeeded you should see the deployment listed: + +```bash +$ helm ls -A +NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION +khaas khaas 1 2024-07-30 19:04:37.0575 +0200 CEST deployed kubehound-0.0.1 +``` + +!!! Note + + This is an example to deploy KubeHound as a Service in k8s cluster, but you will need to adapt it to your own environment. + +## k8s collector + +When deploying the collector inside a k8s cluster, we need to configure one of the following variable: + +- `KH_K8S_CLUSTER_NAME`: variable indicating the name of the targetted k8s cluster + +### RBAC requirements + +In order for the collector to work it needs access to the k8s API and the following k8s ClusterRole: + +| apiGroups | resources | verb | +| ------------------------- | ------------------------------------------------------------ | ----------- | +| rbac.authorization.k8s.io | roles
rolebindings
clusterroles
clusterrolebindings | get
list | +| | pods
nodes
| get
list | +| discovery.k8s.io | endpointslices | get
list | + +The definition of the k8s RBAC can find here: + +- [clusterRole](https://github.com/DataDog/KubeHound/tree/main/deployments/k8s/khaas/templates/cluster_role.yaml) +- [clusterRoleBinding](https://github.com/DataDog/KubeHound/tree/main/deployments/k8s/khaas/templates/cluster_role_binding.yaml) +- [serviceAccount](https://github.com/DataDog/KubeHound/tree/main/deployments/k8s/khaas/templates/service_account.yaml) diff --git a/docs/khaas/getting-started.md b/docs/khaas/getting-started.md new file mode 100644 index 00000000..33c2b8bb --- /dev/null +++ b/docs/khaas/getting-started.md @@ -0,0 +1,57 @@ +# KubeHound as a Service (KHaaS) + +KHaaS enables you to use KubeHound in a distributive way. It is split in 2 main categories: + +- The ingestor stack which includes the `graphdb`, `storedb`, `UI` and `grpc endpoint`. +- The collector (the kubehound binary) which will dump and send the k8s resources to the KHaaS `grpc endpoint`. + +[![](../images/khaas-architecture.png)](../images/khaas-architecture.png) + +!!! note + + You need to deploy the data storage you want to use ([AWS s3 in our example](https://github.com/DataDog/KubeHound/tree/main/deployments/terraform)). + +## Automatic collection + +KHaaS has been created to be deployed inside a kubernetes cluster. This eases scaling and allows you to set Kubernetes `CronJob` daily dumps of your infrastucture for instance. To configure and deploy it, please refer to the [deployment](deployment.md) section. + +## Manual collection + +In order to use `kubehound` with KHaaS, you need to specify the api endpoint you want to use. Since this is not likely to change in your environment, we recommend using the local config file. By default KubeHound will look for `./kubehound.yaml` or `$HOME/.config/kubehound.yaml`. For instance if the default configuration we set the endpoint with disabled SSL: + +```yaml +ingestor: + blob: + bucket_url: "" # (i.e.: s3://your-bucket) + region: "us-east-1" # (i.e.: us-east-1) + api: + endpoint: "127.0.0.1:9000" + insecure: true +``` + +!!! note + + You can use [kubehound-reference.yaml](https://github.com/DataDog/KubeHound/blob/main/configs/etc/kubehound-reference.yaml) as an example which list every options. + +Once everything is configured you just run the following, it will: + +- **dump the k8s resources** and push it compressed to the cloud storage provider. +- send a grpc call to **run the ingestion on the KHaaS** grpc endpoint. + +```bash +kubehound dump remote +``` + +## Manual ingestion + +If you want to rehydrate (reingesting all the latest clusters dumps), you can use the `ingest` command to run it. + +```bash +kubehound ingest remote +``` + +You can also specify a specific dump by using the `--cluster` and `run_id` flags. + +```bash +kubehound ingest remote --cluster my-cluster-1 --run_id 01htdgjj34mcmrrksw4bjy2e94 +``` diff --git a/mkdocs.yml b/mkdocs.yml index f69d16dc..02ca2cb8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -12,16 +12,16 @@ theme: logo: ./logo.png favicon: ./logo.png palette: - - scheme: default - primary: light blue - toggle: - icon: material/toggle-switch-off-outline - name: Switch to dark mode - - scheme: slate - primary: light blue - toggle: - icon: material/toggle-switch - name: Switch to light mode + - scheme: default + primary: light blue + toggle: + icon: material/toggle-switch-off-outline + name: Switch to dark mode + - scheme: slate + primary: light blue + toggle: + icon: material/toggle-switch + name: Switch to light mode features: - navigation.expand - navigation.indexes @@ -30,19 +30,19 @@ theme: - navigation.tabs.sticky - content.code.copy icon: - admonition: - note: octicons/tag-16 - abstract: octicons/checklist-16 - info: octicons/info-16 - tip: octicons/squirrel-16 - success: octicons/check-16 - question: octicons/question-16 - warning: octicons/alert-16 - failure: octicons/x-circle-16 - danger: octicons/zap-16 - bug: octicons/bug-16 - example: octicons/beaker-16 - quote: octicons/quote-16 + admonition: + note: octicons/tag-16 + abstract: octicons/checklist-16 + info: octicons/info-16 + tip: octicons/squirrel-16 + success: octicons/check-16 + question: octicons/question-16 + warning: octicons/alert-16 + failure: octicons/x-circle-16 + danger: octicons/zap-16 + bug: octicons/bug-16 + example: octicons/beaker-16 + quote: octicons/quote-16 use_directory_urls: true extra_javascript: - path: js/tablesort.js #TODO not working @@ -67,7 +67,7 @@ markdown_extensions: - footnotes - markdown_captions - pymdownx.tabbed: - alternate_style: true + alternate_style: true - pymdownx.emoji: emoji_index: !!python/name:material.extensions.emoji.twemoji emoji_generator: !!python/name:material.extensions.emoji.to_svg @@ -85,8 +85,11 @@ nav: - Getting Started: user-guide/getting-started.md - Advanced config: user-guide/advanced-configuration.md - Local Common Operations: user-guide/common-operations.md - - KubeHound as a Service: user-guide/khaas-101.md - Troubleshooting: user-guide/troubleshooting.md + - KubeHound as a Serivce: + - Getting Started: khaas/getting-started.md + - Advanced config: khaas/advanced-configuration.md + - Deployment: khaas/deployment.md - Developer Guide: - Getting Started: dev-guide/getting-started.md - Testing: dev-guide/testing.md