Increase the default PVC size for logstash #7540
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
robbavey
reviewed
Feb 8, 2024
| @@ -301,7 +301,7 @@ WARNING: Volume support for Logstash is a breaking change to earlier versions of | |||
| [discrete] | |||
| == Specifying the volume claim settings | |||
|
|
|||
| By default, a PersistentVolume called `logstash-data` is created, that maps to `/usr/share/logstash/data` for persistent storage, typically used for storage from plugins. The `logstash-data` volume claim is, by default, a small (1Gi) volume, using the standard StorageClass of your Kubernetes cluster, but can be overridden by adding a `spec.volumeClaimTemplate` section named `logstash-data`. | |||
| By default, a PersistentVolume called `logstash-data` is created, that maps to `/usr/share/logstash/data` for persistent storage, typically used for storage from plugins. The `logstash-data` volume claim is, by default, a 1.5Gi volume, using the standard StorageClass of your Kubernetes cluster, but can be overridden by adding a `spec.volumeClaimTemplate` section named `logstash-data`. | |||
|
|
|||
| For production workloads, you should define your own volume claim template with the desired storage capacity and (optionally) the Kubernetes link:https://kubernetes.io/docs/concepts/storage/storage-classes/[storage class] to associate with the persistent volume. To override this volume claim for `data` usages, the name of this volume claim must be `logstash-data`. | |||
|
|
|||
There was a problem hiding this comment.
I think it might be a good idea to add extra information on using persistent volumes, something like:
In particular, when enabling the persistent queue feature, using dead letter queues, or certain plugins that make heavy use of temporary storage, the default volume size will likely be insufficient for production workloads and you should increase the storage capacity, or consider creating separate volumes for these use cases.
Maybe just before the Separate storage section?
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
robbavey
approved these changes
Feb 8, 2024
karenzone
reviewed
Feb 8, 2024
docs/orchestrating-elastic-stack-applications/logstash.asciidoc
Outdated
Show resolved
Hide resolved
docs/orchestrating-elastic-stack-applications/logstash.asciidoc
Outdated
Show resolved
Hide resolved
docs/orchestrating-elastic-stack-applications/logstash.asciidoc
Outdated
Show resolved
Hide resolved
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
|
@karenzone your suggestions look fantastic :) thanks |
barkbay
reviewed
Feb 9, 2024
docs/orchestrating-elastic-stack-applications/logstash.asciidoc
Outdated
Show resolved
Hide resolved
docs/orchestrating-elastic-stack-applications/logstash.asciidoc
Outdated
Show resolved
Hide resolved
barkbay
approved these changes
Feb 9, 2024
There was a problem hiding this comment.
Found 2 issues actually:
- OpenSSL is missing in Logstash UBI image 8.12.1 #7544
- Logstash resources are not reconciled when upgrading from ECK 2.11.x to 2.12.0-SNAPSHOT #7545
But this PR LGTM
Co-authored-by: Michael Morello <michael.morello@gmail.com>
Co-authored-by: Michael Morello <michael.morello@gmail.com>
Co-authored-by: Michael Morello <michael.morello@gmail.com>
|
@barkbay thanks for the suggestions. I think this PR is ready to merge. I will follow up on the new issues you created |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit increases the default PVC size to 1.5G for Logstash.
Persistent queue is a popular feature in Logstash. When PQ is enabled, the original PVC size 1G is too small for Logstash to start.
This Logstash resource should start without error
relates: #7538