Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_get_scheduler() exception if dask.multiprocessing missing #3401

Closed
johnomotani opened this issue Oct 15, 2019 · 0 comments · Fixed by #3411
Closed

_get_scheduler() exception if dask.multiprocessing missing #3401

johnomotani opened this issue Oct 15, 2019 · 0 comments · Fixed by #3411

Comments

@johnomotani
Copy link
Contributor

johnomotani commented Oct 15, 2019

These lines were recently changed in #3358

pass
if actual_get is dask.multiprocessing.get:
return "multiprocessing"
else:
return "threaded"

If the 'cloudpickle' package is not installed, then dask.multiprocessing is not available. The try/except that used to be wrapped around if actual_get is dask.multiprocessing.get meant that _get_scheduler() worked in that case, returning "threaded" (I assume this was the expected behaviour). After #3358, _get_scheduler() raised an AttributeError: module 'dask' has no attribute 'multiprocessing' until I installed 'cloudpickle'.

Suggest either reverting the changes that removed the try/except or making 'cloudpickle' a dependency.


To reproduce:

  1. check 'cloudpickle' is not installed, but 'dask' is
  2. execute the following commands
>>> import xarray
>>> xarray.backends.api._get_scheduler()

Expected result: "threaded"
Actual result:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-20da238796b7> in <module>
----> 1 xarray.backends.api._get_scheduler()

~/.local/lib/python3.6/site-packages/xarray/backends/locks.py in _get_scheduler(get, collection)
     87         pass
     88 
---> 89     if actual_get is dask.multiprocessing.get:
     90         return "multiprocessing"
     91     else:

AttributeError: module 'dask' has no attribute 'multiprocessing'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant