Update queue.mem defaults #5150
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
urso
added
libbeat
needs_backport
ruflin
reviewed
Sep 12, 2017
There was a problem hiding this comment.
Change LGTM. I like the timeout set to 1s which is kind of "near realtime" but still ensures that several events are combined for fileoutput for example.
Was flush.timeout in beta2? If yes, a CHANGLOGE entry should be added.
It seems to break quite a few of our tests. I'm a bit surprised as I did kind of expect the opposite that the flush.timeout: 1s makes things more stable.
urso
force-pushed
the
pipeline/queue-defaults
branch
from
36291f3
to
57361de
Compare
|
Rebased on master with #5148 being merged. |
- Enable flush based memory queue by default: Increases chance of pushing bigger batches of events to the outputs. Especially if number of CPUs is limited to 1 or 2 this helps with throughput, as outputs and producers go-routines will interleave less. - Set flush.events=2048 and flush.timeout=1s
urso
force-pushed
the
pipeline/queue-defaults
branch
from
57361de
to
1402b32
Compare
|
added changelog entry |
update queue flush parameters to min_events of 8 and a timeout of 0.1s. This hopefully stabilizes some time sensitive tests.
andrewkroh
approved these changes
Sep 12, 2017
CHANGELOG.asciidoc
Outdated
| @@ -68,6 +68,7 @@ https://github.com/elastic/beats/compare/v6.0.0-beta2...master[Check the HEAD di | |||
| - Changed the hashbang used in the beat helper script from `/bin/bash` to `/usr/bin/env bash`. {pull}5051[5051] | |||
| - Changed beat helper script to use `exec` when running the beat. {pull}5051[5051] | |||
| - Fix reloader error message to only print on actual error {pull}5066[5066] | |||
| - Enable flush timeout by defauly. {pull}5150[5150] | |||
tsg
pushed a commit
to tsg/beats
that referenced
this pull request
Sep 13, 2017
* Update queue.mem defaults - Enable flush based memory queue by default: Increases chance of pushing bigger batches of events to the outputs. Especially if number of CPUs is limited to 1 or 2 this helps with throughput, as outputs and producers go-routines will interleave less. - Set flush.events=2048 and flush.timeout=1s * set queue flush parameters in system tests update queue flush parameters to min_events of 8 and a timeout of 0.1s. This hopefully stabilizes some time sensitive tests. * Fix typo in CHANGELOG (cherry picked from commit 052172f)
ruflin
pushed a commit
that referenced
this pull request
Sep 13, 2017
* Update queue.mem defaults (#5150) * Update queue.mem defaults - Enable flush based memory queue by default: Increases chance of pushing bigger batches of events to the outputs. Especially if number of CPUs is limited to 1 or 2 this helps with throughput, as outputs and producers go-routines will interleave less. - Set flush.events=2048 and flush.timeout=1s * set queue flush parameters in system tests update queue flush parameters to min_events of 8 and a timeout of 0.1s. This hopefully stabilizes some time sensitive tests. * Fix typo in CHANGELOG (cherry picked from commit 052172f) * fix changelog
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
) * Update queue.mem defaults (elastic#5150) * Update queue.mem defaults - Enable flush based memory queue by default: Increases chance of pushing bigger batches of events to the outputs. Especially if number of CPUs is limited to 1 or 2 this helps with throughput, as outputs and producers go-routines will interleave less. - Set flush.events=2048 and flush.timeout=1s * set queue flush parameters in system tests update queue flush parameters to min_events of 8 and a timeout of 0.1s. This hopefully stabilizes some time sensitive tests. * Fix typo in CHANGELOG (cherry picked from commit 8877dd4) * fix changelog
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Requires: #5148
Enable flush based memory queue by default:
Increases chance of pushing bigger batches of events to the outputs.
Especially if number of CPUs is limited to 1 or 2 this helps with
throughput, as outputs and producers go-routines will interleave less.
Set flush.events=2048 and flush.timeout=1s