Skip to content

Commit

Permalink
initial refactor for NamedArray (#8075)
Browse files Browse the repository at this point in the history
* initial prototype for NamedArray

* move NDArrayMixin and NdimSizeLenMixin inside named_array

* vendor is_duck_dask_array

* vendor Frozen object

* update import

* move _default sentinel value

* rename subpackage to namedarray per @TomNicholas suggestion

* Remove NdimSizeLenMixin

* fix typing

* add annotations

* Remove NDArrayMixin

* Apply suggestions from code review

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

* fix typing

* fix return type

* revert NDArrayMixin

* [WIP] as_compatible_data refactor

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* duplicate sentinel value and leave the original sentinel object alone

* Apply suggestions from code review

Co-authored-by: Stephan Hoyer <shoyer@google.com>

* use DuckArray

* Apply suggestions from code review

Co-authored-by: Stephan Hoyer <shoyer@google.com>

* use sentinel value from xarray

* remove unused code

* fix variable constructor

* fix as_compatible_data utility function

* move _to_dense and _non_zero to NamedArray

* more typing

* add initial tests

* Apply suggestions from code review

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* attempt to fix some mypy errors

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update core.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update core.py

* All input data can be arraylike

* Update core.py

* Update core.py

* get and set attrs at the same level.

* data doesn't have to be ndarray

* avoid redefining typing use new variable names instead

* import on runtime as well to be able to cast

* requires ufunc and function to be a valid duck array

* Add array_namespace

* Update test_dataset.py

* Update test_dataset.py

* remove Frozen

* update tests

* update tests

* switch to functional API

* add fastpath

* Test making sizes dict[Hashable, int]

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* A lot of errors... Try Mapping instead

* Update groupby.py

* Update types.py

* Apply suggestions from code review

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>
Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update docstrings

* update error messages

* update tests

* test explicitly index array

* update tests

* remove unused types

* Update xarray/tests/test_namedarray.py

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use Self

---------

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>
Co-authored-by: dcherian <deepak@cherian.net>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Stephan Hoyer <shoyer@google.com>
Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>
  • Loading branch information
6 people authored Sep 27, 2023
1 parent 84f5a0d commit c3b5ead
Show file tree
Hide file tree
Showing 10 changed files with 700 additions and 293 deletions.
2 changes: 1 addition & 1 deletion xarray/core/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ def _get_axis_num(self: Any, dim: Hashable) -> int:
raise ValueError(f"{dim!r} not found in array dimensions {self.dims!r}")

@property
def sizes(self: Any) -> Frozen[Hashable, int]:
def sizes(self: Any) -> Mapping[Hashable, int]:
"""Ordered mapping from dimension names to lengths.
Immutable.
Expand Down
4 changes: 2 additions & 2 deletions xarray/core/groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -699,7 +699,7 @@ class GroupBy(Generic[T_Xarray]):

_groups: dict[GroupKey, GroupIndex] | None
_dims: tuple[Hashable, ...] | Frozen[Hashable, int] | None
_sizes: Frozen[Hashable, int] | None
_sizes: Mapping[Hashable, int] | None

def __init__(
self,
Expand Down Expand Up @@ -746,7 +746,7 @@ def __init__(
self._sizes = None

@property
def sizes(self) -> Frozen[Hashable, int]:
def sizes(self) -> Mapping[Hashable, int]:
"""Ordered mapping from dimension names to lengths.
Immutable.
Expand Down
2 changes: 1 addition & 1 deletion xarray/core/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ def dims(self) -> Frozen[Hashable, int] | tuple[Hashable, ...]:
...

@property
def sizes(self) -> Frozen[Hashable, int]:
def sizes(self) -> Mapping[Hashable, int]:
...

@property
Expand Down
Loading

0 comments on commit c3b5ead

Please sign in to comment.