Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add slot stats to /agents endpoints #9048

Merged
merged 10 commits into from
Mar 27, 2024
Merged

feat: add slot stats to /agents endpoints #9048

merged 10 commits into from
Mar 27, 2024

Conversation

hamidzr
Copy link
Contributor

@hamidzr hamidzr commented Mar 25, 2024

Description

waiting on web to confirm the schema is sufficient

Test Plan

look at the added stats in the API to see the values are as expected by brand and dev type

  • automated tests

Commentary (optional)

putting this up to get early feedback on the schema changes

not going to land any of the web changes via this pr

Checklist

  • Changes have been manually QA'd
  • User-facing API changes need the "User-facing API Change" label.
  • Release notes should be added as a separate file under docs/release-notes/.
    See Release Note for details.
  • Licenses should be included for new code which was copied and/or modified from any external code.

Ticket

related to https://hpe-aiatscale.atlassian.net/browse/RM-108

@cla-bot cla-bot bot added the cla-signed label Mar 25, 2024
Copy link

netlify bot commented Mar 25, 2024

Deploy Preview for determined-ui canceled.

Name Link
🔨 Latest commit fac1bcf
🔍 Latest deploy log https://app.netlify.com/sites/determined-ui/deploys/66036f6eedea400008b6a60e

scale up the output

turn off topology rendering

intro slot stats

slot stats

fill in summary

make copies

add a flag to exclude slots

comment cleanup

set excludeSlots flag when calling getAgents in webui

cleanup

inform about exact disabled slots

implement it

index via enum names instead of int values

sample out
```
      "slotStats": {
        "deviceTypeCounts": {
          "TYPE_CPU": 4
        },
        "disabledSlots": {},
        "drainingCount": 0,
        "slotStates": {
          "0": "STATE_PULLING"
        },
        "stateCounts": {
          "STATE_PULLING": 1
        }
      },
```

switch to list of strings

new stats

take out old schema?

simplify initializing the stats

also add option to exclude containers?

impelement and use exclude_containers

disable topology view for > MAX_USABLE_NODES

note for topology

reenable pagination

expose a way to togggle requesting slots

no summary flag
Comment on lines 48 to 51
// Map of device type to device stats.
map<string, DeviceStats> type_stats = 6;
// Map of device brands to device stats.
map<string, DeviceStats> brand_stats = 7;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just keep these two

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reminder for myself. my plan is to just these keep these two new structs and we shouldn't need the rest.

@hamidzr hamidzr marked this pull request as ready for review March 26, 2024 17:14
@hamidzr hamidzr requested review from a team as code owners March 26, 2024 17:14
@keita-determined keita-determined changed the title chore: add slot stats to /agents endpoints fix: add slot stats to /agents endpoints Mar 26, 2024
@keita-determined
Copy link
Contributor

changed title with fix prefix because this PR and web PR should be tested in release party

Copy link

codecov bot commented Mar 27, 2024

Codecov Report

Attention: Patch coverage is 61.19403% with 26 lines in your changes are missing coverage. Please review.

Project coverage is 47.81%. Comparing base (1868723) to head (fac1bcf).
Report is 15 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9048   +/-   ##
=======================================
  Coverage   47.80%   47.81%           
=======================================
  Files        1161     1161           
  Lines      143646   143713   +67     
  Branches     2373     2371    -2     
=======================================
+ Hits        68676    68714   +38     
- Misses      74817    74846   +29     
  Partials      153      153           
Flag Coverage Δ
backend 42.97% <83.33%> (+0.03%) ⬆️
harness 63.82% <35.48%> (-0.03%) ⬇️
web 40.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
master/internal/api_agents.go 21.58% <83.33%> (+21.58%) ⬆️
harness/determined/common/api/bindings.py 40.10% <35.48%> (-0.02%) ⬇️

... and 5 files with indirect coverage changes

@hamidzr
Copy link
Contributor Author

hamidzr commented Mar 27, 2024

changed title with fix prefix because this PR and web PR should be tested in release party

I'd change it to feat: that'd also be tested. I think that'd be more accurate.

@hamidzr hamidzr changed the title fix: add slot stats to /agents endpoints feat: add slot stats to /agents endpoints Mar 27, 2024
@hamidzr hamidzr enabled auto-merge (squash) March 27, 2024 00:59
@hamidzr hamidzr merged commit e8dba6d into main Mar 27, 2024
68 of 81 checks passed
@hamidzr hamidzr deleted the hz-customerk branch March 27, 2024 01:16
@hamidzr hamidzr added the to-cherry-pick Pull requests that need to be cherry-picked into the current release label Mar 27, 2024
dai-release bot pushed a commit that referenced this pull request Mar 27, 2024
@hamidzr hamidzr removed their assignment Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed to-cherry-pick Pull requests that need to be cherry-picked into the current release User-facing API Change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants