-
-
Notifications
You must be signed in to change notification settings - Fork 718
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Queue root tasks on scheduler, not workers [with co-assignment] #6598
base: main
Are you sure you want to change the base?
Commits on Jun 18, 2022
-
Idea was that if a `SortedSet` of unrunnable tasks is too expensive, then insertion order is probably _approximately_ priority order, since higher-priority (root) tasks will be scheduled first. This would give us O(1) for all necessary operations, instead of O(logn) for adding and removing. Interestingly, the SortedSet implementation could be hacked to support O(1) `pop` and `popleft`, and inserting a min/max value. In the most common case (root tasks), we're always inserting a value that's greater than the max. Something like this might be the best tradeoff, since it gives us O(1) in the common case but still maintains the sorted gaurantee, which is easier to reason about.
Configuration menu - View commit details
-
Copy full SHA for fdd5fd9 - Browse repository at this point
Copy the full SHA fdd5fd9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 61b6a12 - Browse repository at this point
Copy the full SHA 61b6a12View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4830cf6 - Browse repository at this point
Copy the full SHA 4830cf6View commit details -
improve reasonableness of task-state order
Now task states on the dashboard are listed in the logical order that tasks transition through.
Configuration menu - View commit details
-
Copy full SHA for 7365df4 - Browse repository at this point
Copy the full SHA 7365df4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 66999e1 - Browse repository at this point
Copy the full SHA 66999e1View commit details -
Only support floats for
worker-oversaturation
Simpler, though I think basically just an int of 1 may be the most reasonable.
Configuration menu - View commit details
-
Copy full SHA for f28b9b2 - Browse repository at this point
Copy the full SHA f28b9b2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1380151 - Browse repository at this point
Copy the full SHA 1380151View commit details -
Configuration menu - View commit details
-
Copy full SHA for 117c082 - Browse repository at this point
Copy the full SHA 117c082View commit details -
driveby: WIP color task graph by worker
This is just a hack currently, but maybe it would actually be useful?
Configuration menu - View commit details
-
Copy full SHA for 6a831b2 - Browse repository at this point
Copy the full SHA 6a831b2View commit details -
Revert "driveby: WIP color task graph by worker"
This reverts commit df11f719b59aad11f39a27ccae7b2fd4dfd9243a.
Configuration menu - View commit details
-
Copy full SHA for f9daba4 - Browse repository at this point
Copy the full SHA f9daba4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0edec5d - Browse repository at this point
Copy the full SHA 0edec5dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8a4b2db - Browse repository at this point
Copy the full SHA 8a4b2dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 590aa5b - Browse repository at this point
Copy the full SHA 590aa5bView commit details -
Fix co-assignment logic to consider queued tasks
When there were multiple root task groups, we were just re-using the last worker for every batch because it had nothing processing on it. Unintentionally this also fixes dask#6597 in some cases (because the first task goes to processing, but we measure queued, so we pick the same worker for both task groups)
Configuration menu - View commit details
-
Copy full SHA for b2e7924 - Browse repository at this point
Copy the full SHA b2e7924View commit details
Commits on Jun 20, 2022
-
Revert "unused:
OrderedSet
collection"This reverts commit fdd5fd9.
Configuration menu - View commit details
-
Copy full SHA for c62f82a - Browse repository at this point
Copy the full SHA c62f82aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 51ad734 - Browse repository at this point
Copy the full SHA 51ad734View commit details