Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Archive & Unarchive run #9143

Merged
merged 11 commits into from
Apr 29, 2024
Merged

feat: Archive & Unarchive run #9143

merged 11 commits into from
Apr 29, 2024

Conversation

AmanuelAaron
Copy link
Contributor

Description

Adding ability to archive and unarchive runs

Test Plan

integration tests should pass

Checklist

  • Changes have been manually QA'd
  • User-facing API changes need the "User-facing API Change" label.
  • Release notes should be added as a separate file under docs/release-notes/.
    See Release Note for details.
  • Licenses should be included for new code which was copied and/or modified from any external code.

Ticket

ET-68

@cla-bot cla-bot bot added the cla-signed label Apr 10, 2024
Copy link

netlify bot commented Apr 10, 2024

Deploy Preview for determined-ui canceled.

Name Link
🔨 Latest commit 191a4ae
🔍 Latest deploy log https://app.netlify.com/sites/determined-ui/deploys/662f9aaa283ad10008383329

Copy link

codecov bot commented Apr 10, 2024

Codecov Report

Attention: Patch coverage is 65.31792% with 60 lines in your changes are missing coverage. Please review.

Project coverage is 44.74%. Comparing base (4d20fae) to head (191a4ae).
Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9143      +/-   ##
==========================================
+ Coverage   44.71%   44.74%   +0.02%     
==========================================
  Files        1270     1270              
  Lines      155328   155486     +158     
  Branches     2436     2436              
==========================================
+ Hits        69461    69565     +104     
- Misses      85631    85685      +54     
  Partials      236      236              
Flag Coverage Δ
backend 41.73% <84.76%> (+0.08%) ⬆️
harness 64.23% <35.29%> (-0.05%) ⬇️
web 35.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
master/internal/api_runs.go 73.61% <84.76%> (+3.40%) ⬆️
harness/determined/common/api/bindings.py 40.22% <35.29%> (-0.04%) ⬇️

... and 1 file with indirect coverage changes

@AmanuelAaron AmanuelAaron marked this pull request as ready for review April 10, 2024 17:18
@AmanuelAaron AmanuelAaron requested a review from a team as a code owner April 10, 2024 17:18
Copy link
Contributor

@stoksc stoksc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly lgtm, but if runs are the future we should be covering these new endpoints with performance tests from the start.

@@ -33,6 +34,7 @@ type archiveRunOKResult struct {
ID int32
ExpID *int32
IsMultitrial bool
State *bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you try this with sql.Null[bool] and see how it goes? this is for two reasons: we're focusing more on reliability and continue to see nil pointer exceptions a lot. and that is exactly what this is supposed to help, since sql.Null (or if we had it.. Option[T].. jealous of rust) is basically (better) compile time null safety.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it may be better to have this be a bool. It still works in areas where the field is not being used.

Comment on lines +451 to +458
if filter == nil {
query = query.Where("r.id IN (?)", bun.In(runIDs))
} else {
query, err = filterRunQuery(query, filter)
if err != nil {
return nil, err
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, interface clarify. it's not clear to be from the signature that this is either<runIDs []int32, filter *string>. maybe add a func comment? once again, gah rust for having either<...>.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or, as i'm reading this, since it has if filter == nil in a few places, maybe it just makes sense as two functions. and then you try to not repeat yourself some other way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added error for cases when RunIDs and Filter are both filled. Will add this behaviour for other endpoints in their respective PRs

switch {
case check.Archived && archive:
results = append(results, &apiv1.RunActionResult{
Error: "Run is already archived.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would this be an error? it seems like it would only cause confusion if it is surfaced, not clarity. and i think we should make all new api calls idempotent, it's a really useful property when building distributed systems. wdyt @ashtonG?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing ArchiveExperiments & UnarchiveExperiments do this, so I just wanted to keep inline with the expected behaviour of an Archive action. On further investigation it seems as though ArchiveProject & ArchiveWorkspace return success if they are already archived. So we already have some inconsistencies. I agree that it should return a success

master/internal/api_runs.go Outdated Show resolved Hide resolved
master/internal/api_runs.go Outdated Show resolved Hide resolved
return nil, err
}
for _, acceptID := range acceptedIDs {
results = append(results, &apiv1.RunActionResult{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about for stuff in valid IDs but not in accepted? is that technically possible if something is deleted during this flow.

an aside: I think we should be using db transactions for all these endpoints?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

second aside: make sure the frontend is ok if it doesn't end up getting a result for every row. or update your tests to check this specifically if they don't already?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't believe it's an issue if a run is already deleted and we don't show the result in the response. In the case where a filter is used then it would actually be the preferred behaviour

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the other cases then? Why have we been so particular about showing a response if it doesn't matter now?

Copy link
Contributor

@stoksc stoksc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm modulo feedback from last review (meant to leave this with that)

@AmanuelAaron AmanuelAaron merged commit 703e6bd into main Apr 29, 2024
88 of 100 checks passed
@AmanuelAaron AmanuelAaron deleted the archive-run branch April 29, 2024 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants