The _FillValue in the NLDAS params needs to be explicity changed to the
globalNdv value so the masked areas can be reconstructed on the other
side of an MPI broadcast

* Re-include removed HRRR precipitaion for use in Short-Range forecasts
* Fix issue with MRMS missing data when used as an overlay in AnA

* change negative value replacement from 0 to config_options.globalNdv
* remove log warning (since negatives are most likely missing values in
  radar-only files)

Removed line continuations ('\') in the "Set up ESMF and wgrib2" run step. They were contributing to the CI failures.

…ad config_data symlinks and config_data folder. Now if the folder doesn't exist it triggers the download to populate it. Added hook to allow either esmpy or ESMF python library name (name changed in ESMPy v 8.4.0)

… Updated CI test to run tests out of a docker container instead of building everything as part of the workflow (building ESMF was taking > 20 minutes). Container name is currently andygaydos/wrf_hydro_forcing - this will need to be changed to point to final location of docker image.

… on local repo clone. Added propagation of exit status to python scripts

…r produce errors. Additional YAML key checks (using try/except blocks). Updated ioMod.py to remove temporary grib2.tbl file from scratch dir when finished with it. Uncommented NETCDF variable definition, as it was causing 'not defined' errors

…rocesses. Added pytest-mpi to requirements.

….py to test various MPI n_procs (none, 1, 4, and 8) for consistency

…t, and indexing an array with a string instead of a number). Reworked test fixtures to more easily test outputs of various configurations (time of day, day of year, AnA vs forecast)

…lated to CFS-specific parameters in config.py and bias_correction.py

* Regridding subclasses with abstract class

* New YAML configuration changes

* Revert changes to AnA metadata handling

* removed July 2022 changes to AnA metadata handling and reverted to
  v2.1 behavior.
* removed unneeded special-case code to artificially limit Alaska AnA
  cycle/forecast counting

* AK Short-Range NBM support (NCAR#90)

* Shift NBM cycles back 3 hours for AK Short-Range to match HRRR

* Fix unintentional Alaska MR NBM shift

* Onboarding updates

* Fix parallel bug in NBM regridding

* Fix incorrect time divisor in AK ExtAnA

* Add spatial metadata globals to output files (NCAR#94)

* Support NBM as primary forcing input (NCAR#93)

* Additional NBM support

* Add support for NBM input with external terrain file

* Add creep-fill for NBM

* Remove precipitation from CONUS HRRR forcing (NCAR#92)

* Resolved merge conflicts with the main branch

* Create README.md

* Change `model_total_valid_times` attrib to int32

* Change globalNdv to -999999

* Add  to ignored global metadata

* Don't mask NLDAS params when reading netCDF var

The _FillValue in the NLDAS params needs to be explicity changed to the
globalNdv value so the masked areas can be reconstructed on the other
side of an MPI broadcast

* Adding pytest unit tests and related changes

* Adding change to python version in the workflow

* Adding change to requirements file by removing ESMpy in the workflow

* Adding change to requirements file by removing unnecessary packages in the workflow

* Adding change to requirements file by removing unnecessary packages in the workflow

* Adding change to requirements file by removing unnecessary packages in the workflow

* Adding change to requirements file by removing unnecessary packages in the workflow

* Adding change to python version

* Adding change to linux system type

* Adding change to linux system type

* Adding change to linux system type

* Adding change to linux system type

* Adding change to linux system type

* Adding change to linux system type

* Removed disutils from requirements

* Removed packages from requirements

* Removed packages from requirements

* Removed packages from requirements

* Removed packages from requirements

* Removed packages from requirements

* Removed packages from requirements

* Adding python path dependencies

* Adding python path dependencies

* Adding python path dependencies

* Adding python path dependencies

* Adding WrfHydroForcing before core in import statements

* Removing WrfHydroForcing before core in import statements

* change to python path

* change to python path

* change to setup

* change to setup

* change to setup

* python 3.8

* python 3.8

* python 3.8

* python 3.8

* python 3.8

* Restructured the tests and added relative paths

* Added conda init to github workflow

* Modified conda init to github workflow

* Modified conda init to github workflow

* Modified conda activate to source in github workflow

* Modified conda activation in a non interactive terminal in github workflow

* Added Installed /opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/wrf_hydro_mfe-1.0-py3.8.egg in github workflow

* Added ESMPy install

* Added ESMPy install

* Added ESMPy install

* Added ESMPy install

* Added ESMPy install

* Added ESMPy install

* Delete configOptions_empty.yaml

* Delete configOptions_invalid1.yaml

* Delete configOptions_invalid2.yaml

* Delete configOptions_missing.yaml

* Delete configOptions_missingKey.yaml

* Delete configOptions_valid.yaml

* Delete inputForcings_empty.yaml

* Delete inputForcings_invalid.yaml

* Delete inputForcings_missing.yaml

* Delete inputForcings_valid.yaml

* Delete report.html

* Fix issues with MRMS overlaying HRRR/RAP

* Re-include removed HRRR precipitaion for use in Short-Range forecasts
* Fix issue with MRMS missing data when used as an overlay in AnA

* Set negative MRMS values to globalNdv silently

* change negative value replacement from 0 to config_options.globalNdv
* remove log warning (since negatives are most likely missing values in
  radar-only files)

* Update tests.yml

Removed line continuations ('\') in the "Set up ESMF and wgrib2" run step. They were contributing to the CI failures.

* Updated ESMF_7_1_0r source URL (now hosted on Github)

* Wrong spelling of ESMF tarball filename.

* More paths fixed in tests.yml

* More tests.yml fixes in conda setup

* Added ubuntu install of mpich

* Argh. Need sudo for installing packages on runner machine

* Trying newer version of ESMF

* More mods for building ESMFpy and wgrib2

* wgrib2 and ESMF working, trying to get pytest working

* Added downloading of test data from google drive to tests. Removed dead config_data symlinks and config_data folder. Now if the folder doesn't exist it triggers the download to populate it. Added hook to allow either esmpy or ESMF python library name (name changed in ESMPy v 8.4.0)

* Changed np.int to int

* Added run_tests.py to download test data first before running pytest. Updated CI test to run tests out of a docker container instead of building everything as part of the workflow (building ESMF was taking > 20 minutes). Container name is currently andygaydos/wrf_hydro_forcing - this will need to be changed to point to final location of docker image.

* Changed URL for gdrive download. Added run step to change permissions on local repo clone. Added propagation of exit status to python scripts

* Bug fix

* Whoops gdrive mod didn't work...

* Changed docker image tag name to wrfhydro/wrf_hydro_forcing:latest

* Changed np.int and np.bool to int and bool respectively, as the former produce errors. Additional YAML key checks (using try/except blocks). Updated ioMod.py to remove temporary grib2.tbl file from scratch dir when finished with it. Uncommented NETCDF variable definition, as it was causing 'not defined' errors

* Added initial mpi tests. Modified run_tests.py to use mpirun with 4 processes. Added pytest-mpi to requirements.

* Added tests and assertions for scattering an array using ESMF and mpi

* Added geoMod tests. Added mpi scatter_array tests. Modified run_tests.py to test various MPI n_procs (none, 1, 4, and 8) for consistency

* Forgot to add fixtures.py - a common set of pytest test fixtures

* Added bias correction tests

* Fixed bug in bias_correction.py (variable referenced before assignment, and indexing an array with a string instead of a number). Reworked test fixtures to more easily test outputs of various configurations (time of day, day of year, AnA vs forecast)

* Added bias correction tests for CFS. Fixed yaml configuration bugs related to CFS-specific parameters in config.py and bias_correction.py

---------

Co-authored-by: ishita9 <ishitas@ucar.edu>
Co-authored-by: ishita9 <36771676+ishita9@users.noreply.github.com>
Co-authored-by: ishita9 <ishita9@github.com>
Co-authored-by: Ryan Cabell <rcabell@ucar.edu>
Co-authored-by: Andrew Gaydos <gaydos@derecho4.hsn.de.hpc.ucar.edu>

…n tests to shared fixtures.py file