Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regridded output now same dtype as input #102

Merged
merged 17 commits into from
Aug 12, 2021

Conversation

brews
Copy link
Contributor

@brews brews commented Jul 16, 2021

Output regridded data is now the same type as input data. If input is xarray.Dataset with multiple unique dtypes, raises ValueError.

PR includes integration tests for new behavior under different input types/containers.

One pre-existing test needed to be updated because it output ints when the test was expecting precision from float.

Close #99.

This fails without update because passed in data was int. In past this would have output float64 but output is not int, so test was expecting higher precision. Simple work around is to make input same dtype as we expect output to be.
@brews
Copy link
Contributor Author

brews commented Jul 16, 2021

I think this is ready for review!

I'd especially appreciate input in how this is throwing a ValueError if a multi-dtype xr.Dataset is input.

Also, the change I needed to make to the pre-existing unit test makes me a bit nervous because this PR is clearly breaking from past behavior. Just another thing to note. Anyone else nervous about this?

@brews brews marked this pull request as ready for review July 16, 2021 22:16
@brews
Copy link
Contributor Author

brews commented Jul 19, 2021

I think I could use maintainer input on this. pre-commit is giving me some trouble. It looks like it might be a pre-commit/GH actions configuration issue?

Specifically, I'm looking at

blackdoc.................................................................Failed
- hook id: blackdoc
- exit code: 1

Traceback (most recent call last):
  File "/home/runner/.cache/pre-commit/repotmc3t5mu/py_env-python3/bin/blackdoc", line 5, in <module>
    from blackdoc.__main__ import main
  File "/home/runner/.cache/pre-commit/repotmc3t5mu/py_env-python3/lib/python3.9/site-packages/blackdoc/__main__.py", line 10, in <module>
    from .blackcompat import find_project_root, gen_python_files, read_pyproject_toml
  File "/home/runner/.cache/pre-commit/repotmc3t5mu/py_env-python3/lib/python3.9/site-packages/blackdoc/blackcompat.py", line 9, in <module>
    import toml
ModuleNotFoundError: No module named 'toml'

from https://github.com/pangeo-data/xESMF/pull/102/checks?check_run_id=3091031309.

This PR hasn't added or changed dependencies.

I fixed other apparent linting issues by deactivating pre-commit.

@raphaeldussin
Copy link
Contributor

@brews please feel free to add toml package to the various environment yml files under ci
it looks like black does not depend on toml anymore but blackdoc does:

keewis/blackdoc#100

this should take care of the linting test

Copy link
Contributor

@raphaeldussin raphaeldussin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, this is not necessary anymore, fixed in master

@brews
Copy link
Contributor Author

brews commented Jul 21, 2021

@brews please feel free to add toml package to the various environment yml files under ci
it looks like black does not depend on toml anymore but blackdoc does:

keewis/blackdoc#100

this should take care of the linting test

toml is in the environments as of 87285ce. This looks good to go on my end, unless CI/CD says otherwise.

@raphaeldussin
Copy link
Contributor

the handling in the dataset is finicky and decisions need to be made in terms of desired behavior.
the rest looks good.

Change from reviewer. toml is no longer required for blockdoc linting.
@brews brews requested a review from raphaeldussin July 21, 2021 16:38
brews and others added 8 commits July 21, 2021 16:14
This fails without update because passed in data was int. In past this would have output float64 but output is not int, so test was expecting higher precision. Simple work around is to make input same dtype as we expect output to be.
Change from reviewer. toml is no longer required for blockdoc linting.
@huard
Copy link
Contributor

huard commented Aug 12, 2021

@raphaeldussin Could you clarify what needs to be done to merge this ? What decisions do we need to make?

@raphaeldussin
Copy link
Contributor

@brews I have made a PR to your branch here: brews#2
passing a list of different dtypes in the dataset case works fine (see test), please check and update your PR if you are satisfied with the modifications then we can move forward and merge your PR into the main code.

Copy link
Contributor

@raphaeldussin raphaeldussin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend updating PR with modifs from here brews#2

xesmf/frontend.py Outdated Show resolved Hide resolved
@brews
Copy link
Contributor Author

brews commented Aug 12, 2021

Thanks for the bump, @huard. I merged in the suggested corrections from @raphaeldussin. I agree with @raphaeldussin that accepting multiple dtypes from Datasets is ideal behavior. It appears to work in testing so 👍 .

Pending CI/CD checks, I think this is ready for re-review, @raphaeldussin.

@raphaeldussin
Copy link
Contributor

LGTM

@raphaeldussin raphaeldussin merged commit a85025c into pangeo-data:master Aug 12, 2021
@brews brews deleted the respect_dtype branch August 12, 2021 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

regrid output is always float64
3 participants