agent/core: Implement LFC-aware scaling #1003
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SergeyMelnikov
force-pushed
the
sergey/k8s-1.27.13
branch
2 times, most recently
from
f390e13
to
6c8ae14
Compare
sharnoff
force-pushed
the
sharnoff/agent-wss-scaling
branch
from
b37034e
to
36aab7d
Compare
|
Rebased onto main & squashed all commits -- merging left a lot of commits in the branch history. |
Omrigan
reviewed
Jul 17, 2024
Part of #872. This builds on the metrics that will be exposed by neondatabase/neon#8298. For now, we only look at the working set size metrics over various evenly-spaced windows (all 1 minute apart). The algorithm is somewhat straightforward to implement (see wss.go), but unfortunately seems to be difficult to understand *why* it's expected to work. For more context, refer to the RFC here: https://www.notion.so/neondatabase/cca38138fadd45eaa753d81b859490c6
sharnoff
force-pushed
the
sharnoff/agent-wss-scaling
branch
from
a96af7b
to
644486d
Compare
Omrigan
reviewed
Jul 20, 2024
Omrigan
approved these changes
Jul 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #872.
This builds on the metrics that will be exposed by neondatabase/neon#8298.
For now, we only look at the working set size metrics over various time windows.
The algorithm is somewhat straightforward to implement (see wss.go), but unfortunately seems to be difficult to understand why it's expected to work.
See also: https://www.notion.so/neondatabase/cca38138fadd45eaa753d81b859490c6
Notes for review:
Currently just a draft, pending validation that this does behave as expected.Before merging, I want to:
Add some tests for LFC-based scaling tostate_test.go'sTest_DesiredResourcesFromMetricsOrRequestedUpscaling.Add some more tests inwss_test.goAdd some more comments explaining some of the weirder things (the new algorithm, whydesiredResourcesFromMetricsOrRequestedUpscalingnow needshasMetrics, etc.)As a follow-up, I'd like to resurrect #737 to make this cleaner. I have a WIP branch for that locally; figured it'd be easier to get this PR in first, and then tidy up.
Also, for now this PR builds on #929 for consistency with what's deployed on staging.