Skip to content

Commit

Permalink
tests for arrays with units (#3238)
Browse files Browse the repository at this point in the history
* create the empty test file

* add tests for data array aggregation functions

* include pint in the ci

* ignore missing type annotations for pint

* really skip the tests if pint is not available

* remove the reason from the importorskip call

* test that the dataarray constructor does not strip the unit

* convert every unit stripped warning to an error

* work around pint not implementing np.allclose yet

* remove the now unnecessary filterwarnings decorator

* xfail all tests that depend on pint having a __array_function__

* treat nans as equal

* implement tests for simple arithmetic operations

* use param's id argument to assign readable names

* add tests for sel() and isel()

* add more readable names for the unary arithmetics

* xfail every test that is not yet xfailing

These don't pass because the constructor raises a unit stripped
warning - fixed in pint#764.

* only xfail if pint is not the current development version

This is test is not really reliable, but sufficient for now.

* always use lists instead of tuples for indexing

* add tests for loc and squeeze

* black

* add names and xfail marks to the parameters

* add tests for interp and interp_like

* implement tests for reindex

* remove the xfail marks where it is not clear yet that pint is to blame

* add tests for reindex_like

* don't pass the new DataArray to a kwarg

* xfail if not on pint dev

* refactor the tests

* add tests for univariate and bivariate ufuncs

* black

* xfail aggregation only if pint does not implement __array_function__ yet

* remove the global filterwarnings mark

apparently, this caused the tests to change behavior, resulting in
different errors, or causing tests to pass that should actually fail.

* add a test case for the repr

* create a pytest mark that explicitly requires pint's __array_function__

* also check the string representation in addition to the repr

* add helpers for creating method tests

* check that simple aggregation methods work

* use format() instead of format strings

* make sure the repr of method calls is different from functions

* merge the two aggregation tests

* explicitly check whether pint supports __array_function__

relying on versions is somewhat fragile.

* provide a fallback for the new base quantity

* check that no warning is raised for both with and without coords

* also check that the repr works both with and without coords

* wrap all aggregation function calls

* xfail every call that fails because of something outside xarray

* xfail tests related to dimension coordinates and indexes

* use the dimensions from the original array

* allow passing arguments to the method on call

* add tests for comparisons

* add tests for detecting, filling and dropping missing values

* mark the missing value tests as requiring pint to support duck arrays

* add tests for isin, where and interpolate_na

* reformat unit ids and add a test parameter for compatible units

* remove an unnecessary xfail

* add tests for the top-level replication functions (*_like)

* check for whatever pint does with *_like functions

* add tests for combine_first

* xfail the bivariate ufunc tests

* xfail the call to np.median

* move the top-level function tests out of the DataArray namespace class

* add cumsum and cumprod to the list of aggregation functions

* add tests for the numpy methods

* check for equal units directly after checking the magnitude

* add tests for content manipulation methods

* add tests for comparing DataArrays (equals, indentical)

* add a test for broadcast_equals

* refactor the comparison operation tests

* rewrite the strip, attach and assert_equal functions and add extract

* preserve multiindex in strip and attach

* attach the unit from element "data" as fallback

* fix some small typos

* compare QuantityScalar and QuantitySequence based on their values

* make the isel test more robust

* add tests for reshaping and reordering

* unify the structure of the tests

* mark the remaining tests as requiring a recent pint version, too

* explicitly handle quantities as parameters

* change the repr of the function / method wrappers

* check whether __init__ and repr / str handle units in data and coords

* generalize array_attach_units

* move the redefinition of DimensionalityError

* identify quantities using isinstance

* typo

* skip tests with a pint version without __array_function__

* compare DataArrays where possible

* mark only the compatible unit as xfailing

* preserve the name of data arrays

* also attach units to x_mm

* Test in more CI environments; documentation

* What's New

* remove a stale function

* use Quantity directly for instance tests

* explicitly set roll_coords to silence a deprecation warning

* skip the whole module if pint does not implement __array_function__

the advantage is that now forgetting to decorate a test case is not possible.

* allow to attach units using the mapping from extract_units

* add tests for computation methods

resampling fails until I figure out how to use it with non-datetime coords.

* add tests for grouped operations

* add a test for rolling_exp

* add a todo note for the module level skip on __array_function__

* add a test for dot

* use attach_units instead of manually attaching

* modify the resample test to actually work

* add a test for to_unstacked_dataset

* update whats-new.rst and installing.rst

* reformat the whats-new.rst entry

* What's New
  • Loading branch information
keewis authored and max-sixty committed Oct 17, 2019
1 parent 1f81338 commit 55b1ac0
Show file tree
Hide file tree
Showing 9 changed files with 1,667 additions and 2 deletions.
1 change: 1 addition & 0 deletions ci/requirements/py36-min-all-deps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ dependencies:
- numba=0.44
- numpy=1.14
- pandas=0.24
# - pint # See py36-min-nep18.yml
- pip
- pseudonetcdf=3.0
- pydap=3.2
Expand Down
3 changes: 2 additions & 1 deletion ci/requirements/py36-min-nep18.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@ name: xarray-tests
channels:
- conda-forge
dependencies:
# Optional dependencies that require NEP18, such as sparse,
# Optional dependencies that require NEP18, such as sparse and pint,
# require drastically newer packages than everything else
- python=3.6
- coveralls
- dask=2.4
- distributed=2.4
- numpy=1.17
- pandas=0.24
- pint=0.9 # Actually not enough as it doesn't implement __array_function__yet!
- pytest
- pytest-cov
- pytest-env
Expand Down
1 change: 1 addition & 0 deletions ci/requirements/py36.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand Down
1 change: 1 addition & 0 deletions ci/requirements/py37-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand Down
1 change: 1 addition & 0 deletions ci/requirements/py37.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand Down
11 changes: 10 additions & 1 deletion doc/installing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,15 @@ For plotting
Alternative data containers
~~~~~~~~~~~~~~~~~~~~~~~~~~~
- `sparse <https://sparse.pydata.org/>`_: for sparse arrays
- `pint <https://pint.readthedocs.io/>`_: for units of measure

.. note::

At the moment of writing, xarray requires a `highly experimental version of pint
<https://github.com/andrewgsavage/pint/pull/6>`_ (install with
``pip install git+https://github.com/andrewgsavage/pint.git@refs/pull/6/head)``.
Even with it, interaction with non-numpy array libraries, e.g. dask or sparse, is broken.

- Any numpy-like objects that support
`NEP-18 <https://numpy.org/neps/nep-0018-array-function-protocol.html>`_.
Note that while such libraries theoretically should work, they are untested.
Expand All @@ -85,7 +94,7 @@ dependencies:
(`NEP-29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_)
- **pandas:** 12 months
- **scipy:** 12 months
- **sparse** and other libraries that rely on
- **sparse, pint** and other libraries that rely on
`NEP-18 <https://numpy.org/neps/nep-0018-array-function-protocol.html>`_
for integration: very latest available versions only, until the technology will have
matured. This extends to dask when used in conjunction with any of these libraries.
Expand Down
13 changes: 13 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,19 @@ What's New
v0.14.1 (unreleased)
--------------------

New Features
~~~~~~~~~~~~
- Added integration tests against `pint <https://pint.readthedocs.io/>`_.
(:pull:`3238`) by `Justus Magin <https://github.com/keewis>`_.

.. note::

At the moment of writing, these tests *as well as the ability to use pint in general*
require `a highly experimental version of pint
<https://github.com/andrewgsavage/pint/pull/6>`_ (install with
``pip install git+https://github.com/andrewgsavage/pint.git@refs/pull/6/head)``.
Even with it, interaction with non-numpy array libraries, e.g. dask or sparse, is broken.

Documentation
~~~~~~~~~~~~~

Expand Down
2 changes: 2 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ ignore_missing_imports = True
ignore_missing_imports = True
[mypy-pandas.*]
ignore_missing_imports = True
[mypy-pint.*]
ignore_missing_imports = True
[mypy-PseudoNetCDF.*]
ignore_missing_imports = True
[mypy-pydap.*]
Expand Down
Loading

0 comments on commit 55b1ac0

Please sign in to comment.