Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Xarray's apply_ufunc rather than Dask's map_blocks #1133

Open
tomwhite opened this issue Sep 27, 2023 · 1 comment
Open

Use Xarray's apply_ufunc rather than Dask's map_blocks #1133

tomwhite opened this issue Sep 27, 2023 · 1 comment
Labels
dispatching Issues related to how we send method calls to different backends

Comments

@tomwhite
Copy link
Collaborator

I've been thinking about how we could run (parts of) sgkit on Cubed (#908). One thing that would help is using xarray.map_blocks (or xarray.apply_ufunc) instead of dask.array.map_blocks, since the Xarray version can now dispatch to Dask and Cubed (as well as plain NumPy).
@tomwhite tomwhite added the dispatching Issues related to how we send method calls to different backends label Sep 27, 2023
@tomwhite
Copy link
Collaborator Author

tomwhite commented Aug 19, 2024 •
edited
Loading

From the documentation xarray.apply_ufunc:

Note that due to the overhead xarray.map_blocks() is considerably slower than apply_ufunc.

Also, xarray.map_blocks is marked as experimental and has not been implemented in cubed-xarray, so for those reasons we should prefer xarray.apply_ufunc. This pathway is being explored in #908.

@tomwhite tomwhite changed the title Use Xarray's map_blocks rather than Dask's Use Xarray's apply_ufunc rather than Dask's map_blocks Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dispatching Issues related to how we send method calls to different backends
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@tomwhite