feat: allow k8srm to connect with a kubeconfig

[stoksc]

Description

Allow users to configure the k8s REST client from a kubeconfig. Promote the master ip/port fields to allow
jobs talking to masters outside the cluster they are in to find the master. Remove _creds_dir and its documentation
and upgrade the local devcluster route byo kubeconfig instead.

Test Plan

Follow the instructions for the dev flow in tools/k8s. Do they work?

Commentary (optional)

Checklist

• Changes have been manually QA'd
• User-facing API changes need the "User-facing API Change" label.
• Release notes should be added as a separate file under docs/release-notes/.
  See Release Note for details.
• Licenses should be included for new code which was copied and/or modified from any external code.

Ticket

RM-15

[netlify]

✅ Deploy Preview for determined-ui canceled.

Name	Link
🔨 Latest commit	3c2590c
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/65e8abc3935bd900089dcf2c

[codecov]

Codecov Report

Attention: Patch coverage is 38.46154% with 16 lines in your changes are missing coverage. Please review.

Project coverage is 47.37%. Comparing base (6ecd81e) to head (3c2590c).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8953      +/-   ##
==========================================
+ Coverage   47.35%   47.37%   +0.01%     
==========================================
  Files        1162     1162              
  Lines      176133   176129       -4     
  Branches     2237     2236       -1     
==========================================
+ Hits        83402    83435      +33     
+ Misses      92573    92536      -37     
  Partials      158      158              

Flag	Coverage Δ
backend	42.73% <38.46%> (+0.08%)	⬆️
harness	63.94% <ø> (-0.01%)	⬇️
web	42.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
master/internal/config/resource_manager_config.go	71.59% <ø> (ø)
...nal/rm/kubernetesrm/kubernetes_resource_manager.go	20.29% <0.00%> (ø)
master/internal/rm/kubernetesrm/pods.go	20.52% <43.47%> (+1.45%)	⬆️

... and 6 files with indirect coverage changes

[stoksc]

I'll update the docs tomorrow, and add tests.

[carolinaecalderon]

works on my machine, just one change needed in the devcluster.yaml

[carolinaecalderon]

LGTM!

[amandavialva01]

Awesome work Bradley! LGTM

[stoksc]

running k8s cpu tests then merging. codecov is a bit obnoxious, that I get knocked for adding a config to a bunch of untested files even though the meat of the change is tested.