-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ML] Reimplement established model memory #35263
Merged
droberts195
merged 8 commits into
elastic:feature-jindex-6x
from
droberts195:reimplement_established_model_memory
Nov 13, 2018
Merged
[ML] Reimplement established model memory #35263
droberts195
merged 8 commits into
elastic:feature-jindex-6x
from
droberts195:reimplement_established_model_memory
Nov 13, 2018
Commits on Nov 5, 2018
-
[ML] Reimplement established model memory
This is the 6.6/6.7 implementation of a master node service to keep track of the native process memory requirement of each ML job with an associated native process. The new ML memory tracker service works when the whole cluster is upgraded to at least version 6.6. For mixed version clusters the old mechanism of established model memory stored on the job in cluster state is used. This means that the old (and complex) code to keep established model memory up to date on the job object cannot yet be removed. When this change is forward ported to 7.0 the old way of keeping established model memory updated will be removed.
Configuration menu - View commit details
-
Copy full SHA for 36b8dbd - Browse repository at this point
Copy the full SHA 36b8dbdView commit details
Commits on Nov 7, 2018
-
Configuration menu - View commit details
-
Copy full SHA for f860da1 - Browse repository at this point
Copy the full SHA f860da1View commit details
Commits on Nov 8, 2018
-
Configuration menu - View commit details
-
Copy full SHA for d4124d4 - Browse repository at this point
Copy the full SHA d4124d4View commit details -
Kick persistent tasks after memory requirement refresh
Persistent tasks will recheck allocations when custom metadata changes, so updating a timestamp in the ML metadata will enable persistent tasks whose allocation was deferred to allow for a memory refresh to have another go at selecting an ML node.
Configuration menu - View commit details
-
Copy full SHA for f30c8cc - Browse repository at this point
Copy the full SHA f30c8ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for 10a1467 - Browse repository at this point
Copy the full SHA 10a1467View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7ac0883 - Browse repository at this point
Copy the full SHA 7ac0883View commit details
Commits on Nov 9, 2018
-
Adding this test showed that there was a problem with using Instant.now() in a cluster state update. It is possible that the cluster state update can get applied on more than one master node if the cluster is unstable, and it's essential that they both apply exactly the same change. Therefore I changed the discriminant to the old cluster state version plus one. The actual time is not required. We just need a field whose change will kick persistent tasks.
Configuration menu - View commit details
-
Copy full SHA for 7492b97 - Browse repository at this point
Copy the full SHA 7492b97View commit details
Commits on Nov 12, 2018
-
Configuration menu - View commit details
-
Copy full SHA for 4e004e4 - Browse repository at this point
Copy the full SHA 4e004e4View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.