Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: job queue test failures #8843

Merged
merged 5 commits into from
Feb 15, 2024
Merged

fix: job queue test failures #8843

merged 5 commits into from
Feb 15, 2024

Conversation

AmanuelAaron
Copy link
Contributor

@AmanuelAaron AmanuelAaron commented Feb 14, 2024

Description

Fixing generic task tests that cause the job queue test to fail

Test Plan

e2e tests should pass, specifically test_job_queue.py & test_generic_tasks.py

Checklist

  • Changes have been manually QA'd
  • User-facing API changes need the "User-facing API Change" label.
  • Release notes should be added as a separate file under docs/release-notes/.
    See Release Note for details.
  • Licenses should be included for new code which was copied and/or modified from any external code.

Ticket

Copy link

netlify bot commented Feb 14, 2024

Deploy Preview for determined-ui canceled.

Name Link
🔨 Latest commit 1dfb520
🔍 Latest deploy log https://app.netlify.com/sites/determined-ui/deploys/65cd4065e581fd00087217fa

Copy link

codecov bot commented Feb 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5b2e32d) 47.58% compared to head (1dfb520) 47.57%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8843      +/-   ##
==========================================
- Coverage   47.58%   47.57%   -0.01%     
==========================================
  Files        1068     1068              
  Lines      170241   170241              
  Branches     2240     2240              
==========================================
- Hits        81009    80999      -10     
- Misses      89074    89084      +10     
  Partials      158      158              
Flag Coverage Δ
backend 43.01% <ø> (-0.02%) ⬇️
harness 64.16% <ø> (-0.01%) ⬇️
web 42.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 3 files with indirect coverage changes

@AmanuelAaron AmanuelAaron marked this pull request as ready for review February 14, 2024 18:11
@AmanuelAaron AmanuelAaron requested a review from a team as a code owner February 14, 2024 18:11
@@ -99,7 +99,7 @@ func (s GenericTaskSpec) SetJobPriority(priority int) error { return nil }

// SetWeight todo.
func (s GenericTaskSpec) SetWeight(weight float64) error {
s.Base.ResourcesConfig.SetWeight(weight)
s.GenericTaskConfig.Resources.SetWeight(weight)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this actually affect priority in scheduling?

I see commands set the group weight in the resource manager

switch err := c.rm.SetGroupWeight(sproto.SetGroupWeight{
Weight: weight,
ResourcePool: c.Config.Resources.ResourcePool,
JobID: c.jobID,
}).(type) {
case nil:
case rmerrors.UnsupportedError:
c.syslog.WithError(err).Debug("ignoring unsupported call to set group weight")
default:
return fmt.Errorf("setting group weight for command: %w", err)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation of toV1Job and SetWeight was originally to just allow det job list to work when a generic task was running. We will still need to fully implement all the functions later. If you see the priority and resource pool functions are still unimplemented

e2e_tests/tests/task/test_generic_tasks.py Outdated Show resolved Hide resolved
master/internal/api_generic_tasks.go Outdated Show resolved Hide resolved
@AmanuelAaron AmanuelAaron added the to-cherry-pick Pull requests that need to be cherry-picked into the current release label Feb 14, 2024
@AmanuelAaron AmanuelAaron changed the title fix: set weight for generic tasks fix: job queue test failures Feb 14, 2024
@AmanuelAaron AmanuelAaron merged commit 08dfa43 into main Feb 15, 2024
78 of 90 checks passed
@AmanuelAaron AmanuelAaron deleted the generic-task-job-fix branch February 15, 2024 17:02
dai-release bot pushed a commit that referenced this pull request Feb 15, 2024
* hotfix for test

* ensure task completion

* remove comment

* update test helpers

* default timeout

(cherry picked from commit 08dfa43)
maxrussell pushed a commit that referenced this pull request Mar 21, 2024
* hotfix for test

* ensure task completion

* remove comment

* update test helpers

* default timeout
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed to-cherry-pick Pull requests that need to be cherry-picked into the current release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants