rate limit calls to posix_fadvise() #145
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There can be a large kernel overhead involved in POSIX_FADV_DONTNEED.
There is no point in calling this per item logged, so rate limit
to at most once per 2MiB written.
With a simple test program that logs 100K items at WARNING level:
Before:
$ time strace -c -e fadvise64 log.test
-log_dir=/dev/shm -logtofiles=true -logtostderr=false
% time seconds usecs/call calls errors syscall
100.00 12.522509 125 99957 fadvise64
real 0m52.671s
user 0m2.194s
sys 0m44.022s
After:
$ time strace -c -e fadvise64 log.test
-log_dir=/dev/shm -logtofiles=true -logtostderr=false
% time seconds usecs/call calls errors syscall
100.00 0.000759 152 5 fadvise64
real 0m4.206s
user 0m1.436s
sys 0m3.153s
Fixes issue #84