Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that chunks is tuple of ints upon array creation #1470

Merged
merged 9 commits into from
Jul 27, 2023

Conversation

hanslovsky
Copy link
Contributor

@hanslovsky hanslovsky commented Jul 18, 2023

[Description of PR]

This PR will make sure that chunks of an array are created as tuple[int, ..], even if provided as tuple[float, ...] (#1461). This is already the case for shape and this PR ensures that the behavior is the same between those two.

I opened this as a draft PR with a failing test to ensure that github actions fail as expected. I will add the fix and push separately. Please feel free to squash as needed upon merge of the PR.

Fixes #1461

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst N/A
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@github-actions github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jul 18, 2023
@joshmoore
Copy link
Member

(Launched workflows)

@hanslovsky
Copy link
Contributor Author

Rebased over main

zarr/util.py Outdated
return wrapper


@_as_int_tuple
Copy link
Contributor

@d-v-b d-v-b Jul 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normalize_shape already does shape = tuple(int(s) for s in shape), so what is the purpose of decorating this function here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a safeguard for a future additions of early returns. I can remove the decorator here if desired.

Copy link
Contributor

@d-v-b d-v-b Jul 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about removing the decorator and adding a chunks = tuple(int(s) for s in chunks) line to the end of normalize chunks?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had that as an option in commit 4693cb8 but there are 3 different return statements in normalize_chunks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in that case I would recommend introducing a _chunks variable in that function that gets returned at the end, and replacing the early returns with assignment to _chunks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: None of the early returns in normalize_chunks break my added tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I am happy with any of these options:

  1. decorator on normalize_chunks and normalize_shape
  2. decorator on normalize_chunks
  3. chunks = tuple(int(s) for s in chunks) at the end of normalize_chunks

I'll pick whatever zarr maintainers think is most appropriate for the code base.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I prefer option 3, because at the end of the day this is just about ensuring that functions return the correct type, and I don't think we need decorators for that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reverted to option 3

@hanslovsky hanslovsky changed the title Add failing test for creating group with float chunks Ensure that shape is tuple of ints upon array creation Jul 19, 2023
@hanslovsky hanslovsky changed the title Ensure that shape is tuple of ints upon array creation Ensure that chunks is tuple of ints upon array creation Jul 19, 2023
@github-actions github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Jul 19, 2023
@hanslovsky hanslovsky marked this pull request as ready for review July 19, 2023 14:02
@d-v-b
Copy link
Contributor

d-v-b commented Jul 19, 2023

Looks good!

As an aside, it's weird that we accept booleans for specifying chunks, but I guess it's far too late to change that, unless we were planning a major version increment.

@codecov
Copy link

codecov bot commented Jul 19, 2023

Codecov Report

Merging #1470 (00da620) into main (13cdbff) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main     #1470   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           37        37           
  Lines        14723     14732    +9     
=========================================
+ Hits         14723     14732    +9     
Impacted Files Coverage Δ
zarr/tests/test_creation.py 100.00% <100.00%> (ø)
zarr/util.py 100.00% <100.00%> (ø)

@joshmoore
Copy link
Member

Thanks, guys! Happy to get this in.

Also happy to hear if there is anything that should be done so that mypy could detect that a cast was needed.

@joshmoore joshmoore merged commit 6ed4d78 into zarr-developers:main Jul 27, 2023
19 checks passed
@hanslovsky hanslovsky deleted the fix-1461 branch July 28, 2023 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants