Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

Filebeat Chart #117

Merged
merged 4 commits into from
May 2, 2019
Merged

Filebeat Chart #117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2a231f2
Fix some incorrect es/kibana default values in docs
tylerjl Apr 25, 2019
b367fa5
Initial commit of Filebeat chart
tylerjl Apr 26, 2019
161c6cf
Remove unused filebeat PDB template
tylerjl Apr 30, 2019
2a00066
[filebeat] python templating tests
tylerjl May 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Please look in the chart directories for the documentation for each chart. These

* [Elasticsearch](./elasticsearch/README.md) - [docker image docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html)
* [Kibana](./kibana/README.md) - [docker image docs](https://www.elastic.co/guide/en/kibana/current/docker.html)
* [Filebeat](./filebeat/README.md) - [docker image docs](https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html)

## Kubernetes versions

Expand Down
4 changes: 2 additions & 2 deletions elasticsearch/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,11 @@ helm install --name elasticsearch elastic/elasticsearch --version 7.0.0-alpha1 -
| `minimumMasterNodes` | The value for [discovery.zen.minimum_master_nodes](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/discovery-settings.html#minimum_master_nodes). Should be set to `(master_eligible_nodes / 2) + 1`. Ignored in Elasticsearch versions >= 7. | `2` |
| `esMajorVersion` | Used to set major version specific configuration | `7` |
| `esConfig` | Allows you to add any config files in `/usr/share/elasticsearch/config/` such as `elasticsearch.yml` and `log4j2.properties`. See [values.yaml](./values.yaml) for an example of the formatting. | `{}` |
| `extraEnvs` | Extra [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config) which will be appended to the `env:` definition for the container | `{}` |
| `extraEnvs` | Extra [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config) which will be appended to the `env:` definition for the container | `[]` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching these Mr Eagle eyes.

| `extraVolumes` | Additional volumes to be passed to the `tpl` function | |
| `extraVolumeMounts` | Additional volumeMounts to be passed to the `tpl` function | |
| `extraInitContainers` | Additional init containers to be passed to the `tpl` function | |
| `secretMounts` | Allows you easily mount a secret as a file inside the statefulset. Useful for mounting certificates and other secrets. See [values.yaml](./values.yaml) for an example | `{}` |
| `secretMounts` | Allows you easily mount a secret as a file inside the statefulset. Useful for mounting certificates and other secrets. See [values.yaml](./values.yaml) for an example | `[]` |
| `image` | The Elasticsearch docker image | `docker.elastic.co/elasticsearch/elasticsearch` |
| `imageTag` | The Elasticsearch docker image tag | `7.0.0` |
| `imagePullPolicy` | The Kubernetes [imagePullPolicy](https://kubernetes.io/docs/concepts/containers/images/#updating-images) value | `IfNotPresent` |
Expand Down
2 changes: 2 additions & 0 deletions filebeat/.helmignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
tests/
.pytest_cache/
11 changes: 11 additions & 0 deletions filebeat/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
description: Filebeat
home: https://github.com/elastic/helm-charts
maintainers:
- email: helm-charts@elastic.co
name: Elastic
name: filebeat
version: 7.0.0-alpha1
appVersion: 7.0.0
sources:
- https://github.com/elastic/beats
icon: https://helm.elastic.co/icons/filebeat.png
1 change: 1 addition & 0 deletions filebeat/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../helpers/common.mk
110 changes: 110 additions & 0 deletions filebeat/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Filebeat Helm Chart

This functionality is in alpha status and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but alpha features are not subject to the support SLA of official GA features.

This helm chart is a lightweight way to configure and run our official [Filebeat docker image](https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html).

## Requirements

* Kubernetes >= 1.8
* [Helm](https://helm.sh/) >= 2.8.0

## Installing

* Add the elastic helm charts repo
```
helm repo add elastic https://helm.elastic.co
```
* Install it
```
helm install --name filebeat elastic/filebeat --version 7.0.0-alpha1
```

## Compatibility

This chart is tested with the latest supported versions. The currently tested versions are:

| 5.x | 6.x | 7.x |
| ------ | ----- | ----- |
| 5.6.16 | 6.7.1 | 7.0.0 |

Examples of installing older major versions can be found in the [examples](./examples) directory.

While only the latest releases are tested, it is possible to easily install old or new releases by overriding the `imageTag`. To install version `7.0.0` of Filebeat it would look like this:

```
helm install --name filebeat elastic/filebeat --version 7.0.0-alpha1 --set imageTag=7.0.0
```


## Configuration
| Parameter | Description | Default |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `filebeatConfig` | Allows you to add any config files in `/usr/share/filebeat` such as `filebeat.yml`. See [values.yaml](./values.yaml) for an example of the formatting. | `{}` |
| `extraEnvs` | Extra [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config) which will be appended to the `env:` definition for the container | `[]` |
| `extraVolumeMounts` | Any extra volumes mounts to define for the Filebeat container | `[]` |
| `extraVolumes` | Any extra volumes to define for the pod | `[]` |
| `hostPathRoot` | Fully-qualified [hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath) that will be used to persist Filebeat registry data | `/var/lib` |
| `image` | The Filebeat docker image | `docker.elastic.co/beats/filebeat` |
| `imageTag` | The Filebeat docker image tag | `7.0.0` |
| `imagePullPolicy` | The Kubernetes [imagePullPolicy](https://kubernetes.io/docs/concepts/containers/images/#updating-images) value | `IfNotPresent` |
| `imagePullSecrets` | Configuration for [imagePullSecrets](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-pod-that-uses-your-secret) so that you can use a private registry for your image | `[]` |
| `managedServiceAccount` | Whether the `serviceAccount` should be managed by this helm chart. Set this to `false` in order to manage your own service account and related roles. | `true` |
| `podAnnotations` | Configurable [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) applied to all Filebeat pods | `{}` |
| `podSecurityContext` | Configurable [podSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for Filebeat pod execution environment | `fsGroup: 1000`<br>`runAsUser: 0`<br>`privileged: false` |
| `livenessProbe` | Parameters to pass to [liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/) checks for values such as timeouts and thresholds. | `failureThreshold: 3`<br>`initialDelaySeconds: 10`<br>`periodSeconds: 10`<br>`successThreshold: 3`<br>`timeoutSeconds: 5` |
| `resources` | Allows you to set the [resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for the `DaemonSet` | `requests.cpu: 100m`<br>`requests.memory: 100Mi`<br>`limits.cpu: 1000m`<br>`limits.memory: 200Mi` |
| `serviceAccount` | The [serviceAccount](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) that Filebeat will use during execution | `filebeat` |
| `secretMounts` | Allows you easily mount a secret as a file inside the `DaemonSet`. Useful for mounting certificates and other secrets. See [values.yaml](./values.yaml) for an example | `[]` |
| `terminationGracePeriod` | Termination period (in seconds) to wait before killing Filebeat pod process on pod shutdown | `30` |
| `tolerations` | Configurable [tolerations](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) | `[]` |
| `updateStrategy` | The [updateStrategy](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets) for the `DaemonSet`. By default Kubernetes will kill and recreate pods on updates. Setting this to `OnDelete` will require that pods be deleted manually. | `RollingUpdate` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link here should be pointing to the daemonset docs instead of the statefulset. https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/#daemonset-update-strategy


## Examples

In [examples/](./examples) you will find some example configurations. These examples are used for the automated testing of this helm chart.

### Default

* Deploy the [default Elasticsearch helm chart](../elasticsearch/README.md#default)
* Deploy Filebeat with the default values
```
cd examples/default
make
```
* You can now setup a port forward for Elasticsearch to observe Filebeat indices
```
kubectl port-forward svc/elasticsearch-master 9200
curl localhost:9200/_cat/indices
```

## Testing

This chart uses [pytest](https://docs.pytest.org/en/latest/) to test the templating logic. The dependencies for testing can be installed from the [`requirements.txt`](../requirements.txt) in the parent directory.

```
pip install -r ../requirements.txt
make pytest
```

You can also use `helm template` to look at the YAML being generated

```
make template
```

It is possible to run all of the tests and linting inside of a docker container

```
make test
```

## Integration Testing

Integration tests are run using [goss](https://github.com/aelsabbahy/goss/blob/master/docs/manual.md) which is a serverspec like tool written in golang. See [goss.yaml](examples/default/test/goss.yaml) for an example of what the tests look like.

To run the goss tests against the default example:
```
cd examples/default
make goss
```
13 changes: 13 additions & 0 deletions filebeat/examples/default/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
default: test

include ../../../helpers/examples.mk

RELEASE := helm-filebeat-default

install:
helm upgrade --wait --timeout=600 --values values.yaml --install $(RELEASE) ../../

test: install goss

purge:
helm del --purge $(RELEASE)
15 changes: 15 additions & 0 deletions filebeat/examples/default/test/goss.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
port:
tcp:5066:
listening: true
ip:
- '127.0.0.1'

mount:
/usr/share/filebeat/data:
exists: true

user:
filebeat:
exists: true
uid: 1000
gid: 1000
Loading