Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TaskState class to worker.py #4097

Closed
mrocklin opened this issue Sep 5, 2020 · 2 comments
Closed

Add TaskState class to worker.py #4097

mrocklin opened this issue Sep 5, 2020 · 2 comments

Comments

@mrocklin
Copy link
Member

mrocklin commented Sep 5, 2020

Currently we record state in the Worker class with a variety of dictionaries that hold information like the function to be run, the type of the output, the status, the dependencies that we're waiting on, the number of bytes of the output, and so on. We might consider turning all of these collections into a single collection of a TaskState type.

We did this transformation in the Scheduler a while ago see scheduler.py::TaskState and that seems to have been a good choice. We should probably repeat it for the worker. Maybe something like the following (incomplete)

class TaskState:
    def __init__(self, key, run_spec):
        self.key = key
        self.dependencies = ()
        self.dependents = ()
        self.duration = None
        self.priority = None
        self.state = None
        self.dep_state = None
        self.who_has = None
        self.nbytes = None
        self.resource_restrictions = None
        self.actor = None
        self.exception = None
        self.traceback = None
        self.type = None

This has the advantage of being a good learning task

@mrocklin
Copy link
Member Author

mrocklin commented Sep 8, 2020

@gforsyth if you wanted a warmup ...

@gforsyth
Copy link
Contributor

gforsyth commented Sep 9, 2020

Sounds good. I'll pick this up today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants