feat: backend support for inference metric tracking part 1 #7375
Conversation
tayritenour
requested review from
carolinaecalderon
and removed request for
a team
✅ Deploy Preview for determined-ui canceled.
|
| for _, val := range trialIds { | ||
| if err := CanGetTrialsExperimentAndCheckCanDoAction(ctx, val.TrialID, |
There was a problem hiding this comment.
if there's a ton of trial ids, it may be more optimal to join trial_source_info to trials to experiments, get workspace ids, and check permissions once per workspace id.
I won't insist on adding it right now, but perhaps as a TODO or something to keep an eye on from perf perspective.
There was a problem hiding this comment.
Added a comment about this
There was a problem hiding this comment.
do we also usually want a ticket associated with these inline todos?
determined-ci
removed
the
documentation
There was a problem hiding this comment.
@tayritenour asked I take a quick look. I commented on some bits that jumped out w/o fully understanding everything.
| for _, val := range trialIds { | ||
| if err := CanGetTrialsExperimentAndCheckCanDoAction(ctx, val.TrialID, |
There was a problem hiding this comment.
do we also usually want a ticket associated with these inline todos?
Description
MLG-637
Design Doc: https://hpe.sharepoint.com/:w:/t/detai/EVAlau0Y2exMgXyX63inrmEB7rUFHEfB8bDUmdLTxJ4Bjw?e=ZjAZJ7&clickparams=eyJBcHBOYW1lIjoiVGVhbXMtRGVza3RvcCIsIkFwcFZlcnNpb24iOiIyOC8yMzA1MDEwMDQyMiIsIkhhc0ZlZGVyYXRlZFVzZXIiOmZhbHNlfQ%3D%3D
Adds the ability to keep track of what checkpoints were used in a given trial. When leveraged with the new generic metrics APIs, this will allow us to track and find the metrics for all inference runs that utilize a given checkpoint.
Related PRs:
Test Plan
The integration tests pass. This isn't integrated fully for the user to see in a release party.
Commentary (optional)
Checklist
docs/release-notes/.See Release Note for details.
Ticket