CI: add the pytest-randomly pytest plugin to test env

doc/source/developing/testing.rst

@@ -74,28 +74,35 @@ run:
 Handling yt dependencies
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
-We attempt to make yt compatible with a wide variety of upstream software
-versions. However, sometimes a specific version of a project that yt depends on
+Our dependencies are specified in ``setup.cfg``. Hard dependencies are found in
+``options.install_requires``, while optional dependencies are specified in
+``options.extras_require``. The ``full`` target contains the specs to run our
+test suite, which are intended to be as modern as possible (we don't set upper
+limits to versions unless we need to). The ``minimal`` target is used to check
+that we don't break backward compatibility wiht old versions of upstream
+projects by accident. It is intended to pin stricly our minimal supported
+versions. The ``test`` target specifies the tools neeed to run the tests, but
+not needed by yt itself.
+
+**Python version support.**
+When a new Python version is released, it takes about
+a month or two for yt to support it, since we're dependent on bigger projects
+like numpy and matplotlib. We vow to follow numpy's deprecation plan regarding
+our supported versions for Python and numpy, defined formally in `NEP 29
+<https://numpy.org/neps/nep-0029-deprecation_policy.html>`_. However, we try to
+avoid bumping our minimal requirements shortly before a yt release.
+
+**Third party dependencies.**
+We  However, sometimes a specific version of a project that yt depends on
 causes some breakage and must be blacklisted in the tests or a more
 experimental project that yt depends on optionally might change sufficiently
 that the yt community decides not to support an old version of that project.
 
-To handle cases like this, the versions of upstream software projects installed
-on the machines running the yt test suite are pinned to specific version
-numbers that must be updated manually. This prevents breaking the yt tests when
-a new version of an upstream dependency is released and allows us to manage
-updates in upstream projects at our pace.
-
-If you would like to add a new dependency for yt (even an optional dependency)
-or would like to update a version of a yt dependency, you must edit the
-``tests/test_requirements.txt`` file, this path is relative to the root of the
-repository. This file contains an enumerated list of direct dependencies and
-pinned version numbers. For new dependencies, simply append the name of the new
-dependency to the end of the file, along with a pin to the latest version
-number of the package. To update a package's version, simply update the version
-number in the entry for that package.
-
-Finally, we also run a set of tests with "minimal" dependencies installed. Please make sure any new tests you add that depend on an optional dependency are properly set up so that the test is not run if the dependency is not installed. If for some reason you need to update the listing of packages that are installed for the "minimal" dependency tests, you will need to edit ``tests/test_minimal_requirements.txt``.
+**Note.**
+Some of our optional dependencies are not trivial to install and their support
+may vary across platforms. To manage such issue, we currently use requirement
+files in additions to ``setup.cfg``. They are found in
+``tests/*requirements.txt`` and used in ``tests/ci_install.sh``.
 
 How to Write Unit Tests
 ^^^^^^^^^^^^^^^^^^^^^^^

setup.cfg

@@ -37,15 +37,16 @@ project_urls =
 [options]
 packages = find:
 install_requires =
-    IPython>=1.0
+    ipython>=1.0
     matplotlib>=2.0.2,<3.5
     more-itertools>=8.4
-    numpy>=1.10.4
+    numpy>=1.13.3
+    pyyaml>=4.2b1
     setuptools>=19.6
     sympy>=1.2
     toml>=0.10.2
     tqdm>=3.4.0
-    unyt>=2.7.2
+    unyt>=2.8.0
 python_requires = >=3.6
 include_package_data = True
 scripts = scripts/iyt
@@ -58,10 +59,46 @@ nose.plugins.0.10 =
     answer-testing = yt.utilities.answer_testing.framework:AnswerTesting
 
 [options.extras_require]
+full =
+    astropy~=4.0.1
+    cartopy~=0.18.0
+    f90nml>=1.1.2
+    fastcache~=1.0.2
+    firefly_api>=0.0.2
+    glueviz~=0.13.3
+    h5py~=3.1.0
+    libconf~=1.0.1
+    miniballcpp>=0.2.1
+    mpi4py~=3.0.3
+    netCDF4~=1.5.3
+    pandas~=1.1.2
+    pooch>=0.7.0
+    pyaml~=17.10.0
+    pykdtree~=1.3.1
+    pyqt5~=5.15.2
+    requests~=2.20.0
+    scipy~=1.5.0
+    xarray~=0.16.1
 hub =
     girder_client
 mapserver =
     bottle
+minimal =
+    ipython==1.0.0
+    matplotlib==2.0.2
+    more-itertools==8.4
+    numpy==1.13.3
+    sympy==1.2
+    unyt==2.8.0
+test =
+    codecov~=2.0.15
+    coverage~=4.5.1
+    nose~=1.3.7
+    nose-exclude
+    nose-timer~=1.0.0
+    pytest>=6.1
+    pytest-randomly>=3.7.0
+    pytest-xdist~=2.1.0
 
 [flake8]
 max-line-length = 88

tests/ci_install.sh

@@ -30,43 +30,34 @@ cp tests/matplotlibrc .
 
 # Step 1: pre-install required packages
 if [[ "${RUNNER_OS}" == "Windows" ]] && [[ ${dependencies} != "minimal" ]]; then
-    # Install some dependencies using conda (if not doing a minimal run)
-    CYTHON=$(grep cython tests/test_prerequirements.txt)
-    NUMPY=$(grep numpy tests/test_prerequirements.txt)
-
-    CARTOPY=$(grep cartopy tests/test_requirements.txt)
-    H5PY=$(grep h5py tests/test_requirements.txt)
-    MATPLOTLIB=$(grep matplotlib tests/test_requirements.txt)
-    SCIPY=$(grep scipy tests/test_requirements.txt)
-    conda config --set always_yes yes
-    conda info -a
-    conda install --quiet --yes -c conda-forge \
-      $CYTHON $NUMPY $CARTOPY $H5PY $MATPLOTLIB $SCIPY
+    # windows_conda_requirements.txt is a survivance of test_requirements.txt
+    # keep in sync: setup.cfg
+    while read requirement; do conda install --yes $requirement; done < tests/windows_conda_requirements.txt
 else
     python -m pip install --upgrade pip
     python -m pip install --upgrade wheel
     python -m pip install --upgrade setuptools
 fi
 
-# Step 2: install required packages (depending on whether the build is minimal)
+# Step 2: install deps and yt
 if [[ ${dependencies} == "minimal" ]]; then
-    # Ensure numpy and cython are installed so dependencies that need to be built
-    # don't error out
-    # The first numpy to support py3.6 is 1.12, but numpy 1.13 matches
-    # unyt so we'll match it here.
-    python -m pip install numpy==1.13.3 cython==0.26.1
-    python -m pip install -r tests/test_minimal_requirements.txt
+    python -m pip install -e .[test,minimal]
 else
-    # Getting cartopy installed requires getting cython and numpy installed
-    # first; this is potentially going to be fixed with the inclusion of
-    # pyproject.toml in cartopy.
-    # These versions are pinned, so we will need to update/remove them when
-    # the hack is no longer necessary.
-    python -m pip install -r tests/test_prerequirements.txt
-    CFLAGS="$CFLAGS -DACCEPT_USE_OF_DEPRECATED_PROJ_API_H" python -m pip install -r tests/test_requirements.txt
-fi
+    # Cython and numpy as build-time requirements to the following optional deps in yt
+    # - cartopy
+    # - netcdf4
+    # - pyqt5
+    # The build system are however not specified properly in these projects at the moment
+    # which means we have to install the build-time requirements first.
+    # It is possible that these problems will be fixed in the future if upstream projects
+    # include a pyproject.toml file or use any pip-comptatible solution to remedy this.
+    python -m pip install numpy>=1.19.4 cython>=0.29.21
 
-# Step 3: install yt
-python -m pip install -e .
+    # this is required for cartopy. It should normally be specified in our setup.cfg as
+    # cartopy[plotting]
+    # However it doesn't work on Ubuntu 18.04 (used in CI at the time of writing)
+    pip install shapely --no-binary=shapely
+    CFLAGS="$CFLAGS -DACCEPT_USE_OF_DEPRECATED_PROJ_API_H" python -m pip install -e .[test,full]
+fi
 
 set +x

tests/windows_conda_requirements.txt

@@ -0,0 +1,6 @@
+numpy>=1.19.4
+cython>=0.29.21
+cartopy~=0.18.0
+h5py~=3.1.0
+matplotlib<3.5
+scipy~=1.5.0