add a ms instead of ns to end time of the rebound chunk interval #5351
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
conversion between ms and ns loses ns precision which causes us to not include all the logs
kavirajk
reviewed
Feb 9, 2022
sandeepsukhani
added a commit
that referenced
this pull request
Feb 10, 2022
* add a ms instead of ns to end time of the rebound chunk interval conversion between ms and ns loses ns precision which causes us to not include all the logs * improve error when failing to rebound chunk
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
While rebounding chunk with an interval, we get inclusive end time, while the iterator we use for fetching all the samples considers end time to be non-inclusive. Therefore, we added a
nsto the end time to match the expectation to ensure we included all the requested log lines.But since we have
msprecise chunk intervals, which is what the deletion code works on, we lose the ns precision. So adding 1nsmakes us not include logs that have > 1nsvalue. E.g., if the end time of the chunk is1643958368443and we have a log at1643958368443000064, adding 1nswhile rebounding the second half of that chunk would make us not include the log line at1643958368443000064since the end time we passed to iterator is1643958368443000001.This PR fixes the issue by adding a
msinstead of ansbefore passing the end time to the iterator.