Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TaskState to worker.py #4107

Merged
merged 91 commits into from
Oct 15, 2020
Merged

Add TaskState to worker.py #4107

merged 91 commits into from
Oct 15, 2020

Commits on Sep 10, 2020

  1. Add TaskState for some (but not all) relevant dicts

    Starting with:
    * `key`
    * `duration`
    * `priority`
    * `state`
    * `resource_restrictions`
    * `exception`
    * `traceback`
    
    Going to get that working, then handle the rest --
    `dep_state` seems tricky to get right
    gforsyth committed Sep 10, 2020
    Configuration menu
    Copy the full SHA
    1eddc52 View commit details
    Browse the repository at this point in the history
  2. Fixup some more task_state

    gforsyth committed Sep 10, 2020
    Configuration menu
    Copy the full SHA
    eb0d87d View commit details
    Browse the repository at this point in the history
  3. Fix up validation checks

    gforsyth committed Sep 10, 2020
    Configuration menu
    Copy the full SHA
    f4ae9ac View commit details
    Browse the repository at this point in the history
  4. Make sure key exists

    Signed-off-by: Gil Forsyth <gil@forsyth.dev>
    gforsyth committed Sep 10, 2020
    Configuration menu
    Copy the full SHA
    7a36655 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    59f2ce2 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2020

  1. Configuration menu
    Copy the full SHA
    5e49258 View commit details
    Browse the repository at this point in the history
  2. First cut at removing dep_state, dependencies and dependents dicts

    Working plan:
    - All dependencies are also Tasks (they don't necessarily have a `runspec`
    because they might be gathered from another worker)
    - `dependents` and `dependencies` are sets of TaskState objects
    gforsyth committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    604f23c View commit details
    Browse the repository at this point in the history
  3. Add new attributes to init

    Also treating `who_has` as a Task object level dict, where keys are the
    dependency keys and workers is a list of workers who have that data
    
    This is maybe... wrong?
    
    Each dependency TaskState object should be itself (ish) so probably just needs
    to be a list of workers
    gforsyth committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    3ece01f View commit details
    Browse the repository at this point in the history
  4. who_has should be a set

    gforsyth committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    fff2281 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2020

  1. Check key not instance

    gforsyth committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    a7af14d View commit details
    Browse the repository at this point in the history
  2. deadlocking

    gforsyth committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    a5d08ff View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2020

  1. Configuration menu
    Copy the full SHA
    69bac91 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e380517 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a35bddc View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1180f71 View commit details
    Browse the repository at this point in the history
  5. Remove breakpoints

    gforsyth committed Sep 17, 2020
    Configuration menu
    Copy the full SHA
    2f05334 View commit details
    Browse the repository at this point in the history
  6. Remove old notes

    gforsyth committed Sep 17, 2020
    Configuration menu
    Copy the full SHA
    81150e5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    249b196 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2020

  1. Configuration menu
    Copy the full SHA
    2012784 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e441e56 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0486146 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2020

  1. Fix assertion in key_waiting

    gforsyth committed Sep 19, 2020
    Configuration menu
    Copy the full SHA
    b345125 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2020

  1. Configuration menu
    Copy the full SHA
    2bf3259 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    112823f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    dc725bb View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ef4eea4 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    6156e9c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a27ed18 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d0ffad2 View commit details
    Browse the repository at this point in the history
  8. Remove unneeded xfail

    gforsyth committed Sep 21, 2020
    Configuration menu
    Copy the full SHA
    b98b199 View commit details
    Browse the repository at this point in the history
  9. Smol cleanup

    gforsyth committed Sep 21, 2020
    Configuration menu
    Copy the full SHA
    f77b33c View commit details
    Browse the repository at this point in the history
  10. black

    gforsyth committed Sep 21, 2020
    Configuration menu
    Copy the full SHA
    a9b2ff9 View commit details
    Browse the repository at this point in the history
  11. Merge _dep_transitions and _transitions and remove _dep checks

    They are all key checks now
    gforsyth committed Sep 21, 2020
    Configuration menu
    Copy the full SHA
    9f7cf89 View commit details
    Browse the repository at this point in the history
  12. Add flight -> ready entry in `transitions

    This should be the same as flight -> memory
    gforsyth committed Sep 21, 2020
    Configuration menu
    Copy the full SHA
    2ae9a87 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    9446af9 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2020

  1. Add Worker-level count of waiting for data tasks

    Need to think about this more, but we don't want to count these up every time
    the dashboard refreshes
    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    aee4bae View commit details
    Browse the repository at this point in the history
  2. Test fix

    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    ff05783 View commit details
    Browse the repository at this point in the history
  3. remove leftover breakpoint

    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    0b587d6 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    eda1bde View commit details
    Browse the repository at this point in the history
  5. Update failed worker tests

    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    4f47d02 View commit details
    Browse the repository at this point in the history
  6. black again

    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    e1dac2f View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    64f2af3 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    4d6f191 View commit details
    Browse the repository at this point in the history
  9. Remove unneccessary checks

    Things are a bit more stable now
    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    c5eaf37 View commit details
    Browse the repository at this point in the history
  10. Changes to validate_state

    It is OK for there to be tasks that aren't in `self.tasks` yet, but not ok if
    they don't have an assigned `state`
    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    4f94290 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    28e5694 View commit details
    Browse the repository at this point in the history
  12. Remove old skip

    gforsyth committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    5ab969c View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    b32629c View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2020

  1. Update tests to not call put_key_in_memory directly

    Tests (and everything else) should use `update_data` for this kind of (unusual)
    call.
    
    This also avoids special casing an otherwise absent Task object
    gforsyth committed Sep 23, 2020
    Configuration menu
    Copy the full SHA
    d8ce2d0 View commit details
    Browse the repository at this point in the history
  2. continue if there are missing tasks on ready

    Comment in code, too, but I was initially treating this as an error, but it can
    happen as a regular part task execution where a task is re-routed but
    `self.ready` isn't updated before we get here.
    gforsyth committed Sep 23, 2020
    Configuration menu
    Copy the full SHA
    c3c1110 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2020

  1. Use the task key

    gforsyth committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    de950fd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    24e6ed4 View commit details
    Browse the repository at this point in the history
  3. Add ready -> waiting transition

    I think this probably isn't right, but I'm trying it out
    gforsyth committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    44d761c View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2020

  1. Configuration menu
    Copy the full SHA
    ad3c248 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    046360e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4550780 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8f1892b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    3a24aa8 View commit details
    Browse the repository at this point in the history
  6. Handle missing deps with TaskState

    Also removed adding a dep to `_missing_dep_flight` in one of the transition
    functions -- that leads to it being excluded from the dep gather, which seems
    circular in a bad way
    gforsyth committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    24108f2 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    887081e View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    504a51f View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2020

  1. Configuration menu
    Copy the full SHA
    7b85334 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2020

  1. Configuration menu
    Copy the full SHA
    fbf776a View commit details
    Browse the repository at this point in the history
  2. Add type_ to TaskState

    And clean up a few commented out lines
    gforsyth committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    349fe8e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0cc88f8 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    992fbbd View commit details
    Browse the repository at this point in the history
  5. Move startstops to TaskState

    gforsyth committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    b60d4e5 View commit details
    Browse the repository at this point in the history
  6. Remove some older comments

    gforsyth committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    ced3fd4 View commit details
    Browse the repository at this point in the history
  7. Check if state is waiting

    gforsyth committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    2ef35ff View commit details
    Browse the repository at this point in the history
  8. Add TaskState to api docs

    gforsyth committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    f67353b View commit details
    Browse the repository at this point in the history
  9. Restore ts check in transition

    And handle the possibility of a `cause` not existing in `self.tasks`
    gforsyth committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    173c28d View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2020

  1. Restore self.executing check

    gforsyth committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    44e4404 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3ad8972 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1a2c591 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    5c4aa90 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1a3b189 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c2ae508 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    5aaac47 View commit details
    Browse the repository at this point in the history
  8. Add validate check for "flight"

    This doesn't include the `assert ts.key not in self.nbytes` -- since tasks are
    created via `add_task`, they can have `nbytes` specified before-hand
    gforsyth committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    fde532c View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    44135db View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2020

  1. Add docstring, change var name for waiting task count

    And also restore said count to Worker `repr`
    gforsyth committed Oct 9, 2020
    Configuration menu
    Copy the full SHA
    5135c1c View commit details
    Browse the repository at this point in the history
  2. Apply suggestions from code review

    Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
    gforsyth and jrbourbeau authored Oct 9, 2020
    Configuration menu
    Copy the full SHA
    f24365e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ccfd9eb View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2020

  1. Configuration menu
    Copy the full SHA
    0877311 View commit details
    Browse the repository at this point in the history
  2. Replace self.executing with self.executing_count

    This is a redundant bit of state tracking and overcomplicates things.
    Instead, similar to `Worker.tasks` this maintains a count of the
    executing tasks for reporting but relies on `TaskState` objects to keep
    track of the task state.
    
    Note that the removal of the `finally:` block, specifically, is because
    _if_ the key has already been released, then `self.executing_count` is
    already decremented so that particular check doesn't need to be run.
    gforsyth committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    f49fbe8 View commit details
    Browse the repository at this point in the history
  3. Change type_ to type

    gforsyth committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    508927f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    abce5f6 View commit details
    Browse the repository at this point in the history
  5. Fix test

    gforsyth committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    6f9e6a9 View commit details
    Browse the repository at this point in the history
  6. Test fix?

    gforsyth committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    d15e443 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2020

  1. Configuration menu
    Copy the full SHA
    1e7de67 View commit details
    Browse the repository at this point in the history
  2. Update distributed/diagnostics/tests/test_worker_plugin.py

    Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
    gforsyth and jrbourbeau authored Oct 15, 2020
    Configuration menu
    Copy the full SHA
    403ca03 View commit details
    Browse the repository at this point in the history