-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add code to wait for segments generated to be loaded on historicals #14322
Merged
cryptoe
merged 26 commits into
apache:master
from
adarshsanjeev:controller-load-segments-wait
Sep 6, 2023
Merged
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
2539f86
Add code to wait for segments generated to be loaded on historicals
adarshsanjeev 92b432d
Add broker client and use it to query broker for segment load status
adarshsanjeev 68210ba
Temp
adarshsanjeev 5059fba
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev 52aef1d
Update query with replication factor
adarshsanjeev a7de731
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev 529a577
Cleanup code
adarshsanjeev 650f75b
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev c49f871
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev f19c8f7
Code cleanup
adarshsanjeev d86ad22
Improve coverage
adarshsanjeev 5f79d68
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev 47da704
Resolve build failures
adarshsanjeev e2b1787
Address review comments
adarshsanjeev b922b2f
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev cc5456b
Address review comments
adarshsanjeev 2d9db10
Address review comments
adarshsanjeev 2401c0c
Address review comments
adarshsanjeev 23c2f89
Update names
adarshsanjeev c323474
Update names
adarshsanjeev 6b99073
Increase coverage
adarshsanjeev 7675c8c
Fix spelling
adarshsanjeev f709e71
Add java docs
adarshsanjeev 41aa0a5
Update error message
adarshsanjeev ca398b7
Merge remote-tracking branch 'origin/master' into controller-load-seg…
adarshsanjeev 7ca9465
Update extensions-core/multi-stage-query/src/main/java/org/apache/dru…
cryptoe File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There would always be one version rite ?
Can we add a check here ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought this initially as well, but looking into the code, we actually generate the version based on the lock we acquire for the segment. So if there are multiple intervals we are replacing into and therefore multiple locks, we would have more than one version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh i did not know that. Thanks for explaining the rational. We could also document this as well :)