chore: deprecate kubernetes priority w/ preemption scheduler

[carolinaecalderon]

Ticket

CM-385

Description

Deprecate the Kubernetes priority with preemption scheduler. Remove references to the helm chart & invalidate master configuration input defaultScheduler: preemption. Set the preemption value for all kubernetes resource managers to false (since now no scheduler has preemption enabled).

Test Plan

No testing needed, but manual testing shows that defining defaultScheduler: preemption in the yaml chart will be overwritten by a blank default value to the master configuration. Also, trying to define defaultScheduler: preemption in a devcluster yaml's master configuration won't be recognized as valid.

Checklist

• Changes have been manually QA'd
• New features have been approved by the corresponding PM
• User-facing API changes have the "User-facing API Change" label
• Release notes have been added as a separate file under docs/release-notes/
  See Release Note for details.
• Licenses have been included for new code which was copied and/or modified from any external code

[netlify]

✅ Deploy Preview for determined-ui canceled.

Name	Link
🔨 Latest commit	57f4395
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/66bd06c5f2b442000895c0f7

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 5 lines in your changes missing coverage. Please review.

Project coverage is 54.19%. Comparing base (9da5620) to head (57f4395).
Report is 9 commits behind head on main.

Files	Patch %	Lines
master/internal/config/resource_manager_config.go	57.14%	3 Missing ⚠️
master/internal/config/config.go	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9763      +/-   ##
==========================================
- Coverage   54.26%   54.19%   -0.08%     
==========================================
  Files        1261     1261              
  Lines      155671   155680       +9     
  Branches     3536     3534       -2     
==========================================
- Hits        84481    84365     -116     
- Misses      71052    71177     +125     
  Partials      138      138              

Flag	Coverage Δ
backend	44.84% <66.66%> (-0.08%)	⬇️
harness	72.34% <ø> (-0.27%)	⬇️
web	53.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...nal/rm/kubernetesrm/kubernetes_resource_manager.go	27.67% <100.00%> (-0.22%)	⬇️
master/internal/rm/kubernetesrm/spec.go	82.32% <100.00%> (ø)
master/internal/config/config.go	58.82% <66.66%> (-11.05%)	⬇️
master/internal/config/resource_manager_config.go	66.40% <57.14%> (-0.81%)	⬇️

... and 8 files with indirect coverage changes

[kkunapuli]

Looking good. Left some questions/comments.

[kkunapuli]

LGTM! Nice work.

[ShreyaLnuHpe]

LGTM!

[tara-det-ai]

LGTM