-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor updates #2
base: refactor
Are you sure you want to change the base?
Commits on Mar 28, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3cc6bb1 - Browse repository at this point
Copy the full SHA 3cc6bb1View commit details
Commits on Mar 31, 2023
-
Configuration menu - View commit details
-
Copy full SHA for c61cb6d - Browse repository at this point
Copy the full SHA c61cb6dView commit details -
Configuration menu - View commit details
-
Copy full SHA for c23bb81 - Browse repository at this point
Copy the full SHA c23bb81View commit details
Commits on Apr 13, 2023
-
Don't mask NLDAS params when reading netCDF var
The _FillValue in the NLDAS params needs to be explicity changed to the globalNdv value so the masked areas can be reconstructed on the other side of an MPI broadcast
Configuration menu - View commit details
-
Copy full SHA for a930309 - Browse repository at this point
Copy the full SHA a930309View commit details
Commits on Aug 16, 2023
-
Fix issues with MRMS overlaying HRRR/RAP
* Re-include removed HRRR precipitaion for use in Short-Range forecasts * Fix issue with MRMS missing data when used as an overlay in AnA
Configuration menu - View commit details
-
Copy full SHA for 06e460d - Browse repository at this point
Copy the full SHA 06e460dView commit details
Commits on Aug 18, 2023
-
Set negative MRMS values to globalNdv silently
* change negative value replacement from 0 to config_options.globalNdv * remove log warning (since negatives are most likely missing values in radar-only files)
Configuration menu - View commit details
-
Copy full SHA for 33d1f7e - Browse repository at this point
Copy the full SHA 33d1f7eView commit details
Commits on Aug 23, 2023
-
Configuration menu - View commit details
-
Copy full SHA for fdb9d44 - Browse repository at this point
Copy the full SHA fdb9d44View commit details -
Removed line continuations ('\') in the "Set up ESMF and wgrib2" run step. They were contributing to the CI failures.
Configuration menu - View commit details
-
Copy full SHA for 84eb47e - Browse repository at this point
Copy the full SHA 84eb47eView commit details
Commits on Aug 24, 2023
-
Configuration menu - View commit details
-
Copy full SHA for e575aa7 - Browse repository at this point
Copy the full SHA e575aa7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 814e856 - Browse repository at this point
Copy the full SHA 814e856View commit details -
Configuration menu - View commit details
-
Copy full SHA for f676797 - Browse repository at this point
Copy the full SHA f676797View commit details -
Configuration menu - View commit details
-
Copy full SHA for bb1d23a - Browse repository at this point
Copy the full SHA bb1d23aView commit details -
Configuration menu - View commit details
-
Copy full SHA for ce4c23f - Browse repository at this point
Copy the full SHA ce4c23fView commit details -
Configuration menu - View commit details
-
Copy full SHA for cf3d025 - Browse repository at this point
Copy the full SHA cf3d025View commit details -
Configuration menu - View commit details
-
Copy full SHA for 985be5e - Browse repository at this point
Copy the full SHA 985be5eView commit details -
Configuration menu - View commit details
-
Copy full SHA for e69082f - Browse repository at this point
Copy the full SHA e69082fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 075a21f - Browse repository at this point
Copy the full SHA 075a21fView commit details
Commits on Aug 25, 2023
-
Added downloading of test data from google drive to tests. Removed de…
…ad config_data symlinks and config_data folder. Now if the folder doesn't exist it triggers the download to populate it. Added hook to allow either esmpy or ESMF python library name (name changed in ESMPy v 8.4.0)
Configuration menu - View commit details
-
Copy full SHA for 2bad68c - Browse repository at this point
Copy the full SHA 2bad68cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 134b2df - Browse repository at this point
Copy the full SHA 134b2dfView commit details
Commits on Aug 29, 2023
-
Added run_tests.py to download test data first before running pytest.…
… Updated CI test to run tests out of a docker container instead of building everything as part of the workflow (building ESMF was taking > 20 minutes). Container name is currently andygaydos/wrf_hydro_forcing - this will need to be changed to point to final location of docker image.
Configuration menu - View commit details
-
Copy full SHA for 90230cd - Browse repository at this point
Copy the full SHA 90230cdView commit details -
Changed URL for gdrive download. Added run step to change permissions…
… on local repo clone. Added propagation of exit status to python scripts
Configuration menu - View commit details
-
Copy full SHA for 3889630 - Browse repository at this point
Copy the full SHA 3889630View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d4c139 - Browse repository at this point
Copy the full SHA 4d4c139View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5765d74 - Browse repository at this point
Copy the full SHA 5765d74View commit details -
Configuration menu - View commit details
-
Copy full SHA for d382e14 - Browse repository at this point
Copy the full SHA d382e14View commit details
Commits on Sep 1, 2023
-
Changed np.int and np.bool to int and bool respectively, as the forme…
…r produce errors. Additional YAML key checks (using try/except blocks). Updated ioMod.py to remove temporary grib2.tbl file from scratch dir when finished with it. Uncommented NETCDF variable definition, as it was causing 'not defined' errors
Andrew Gaydos committedSep 1, 2023 Configuration menu - View commit details
-
Copy full SHA for 405659e - Browse repository at this point
Copy the full SHA 405659eView commit details
Commits on Sep 8, 2023
-
Added initial mpi tests. Modified run_tests.py to use mpirun with 4 p…
…rocesses. Added pytest-mpi to requirements.
Configuration menu - View commit details
-
Copy full SHA for cf47ae3 - Browse repository at this point
Copy the full SHA cf47ae3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 66071e0 - Browse repository at this point
Copy the full SHA 66071e0View commit details
Commits on Sep 11, 2023
-
Added geoMod tests. Added mpi scatter_array tests. Modified run_tests…
….py to test various MPI n_procs (none, 1, 4, and 8) for consistency
Configuration menu - View commit details
-
Copy full SHA for 63e8c71 - Browse repository at this point
Copy the full SHA 63e8c71View commit details -
Configuration menu - View commit details
-
Copy full SHA for 090cfd2 - Browse repository at this point
Copy the full SHA 090cfd2View commit details
Commits on Sep 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for c242506 - Browse repository at this point
Copy the full SHA c242506View commit details
Commits on Sep 18, 2023
-
Fixed bug in bias_correction.py (variable referenced before assignmen…
…t, and indexing an array with a string instead of a number). Reworked test fixtures to more easily test outputs of various configurations (time of day, day of year, AnA vs forecast)
Configuration menu - View commit details
-
Copy full SHA for 5be220e - Browse repository at this point
Copy the full SHA 5be220eView commit details
Commits on Sep 28, 2023
-
Added bias correction tests for CFS. Fixed yaml configuration bugs re…
…lated to CFS-specific parameters in config.py and bias_correction.py
Configuration menu - View commit details
-
Copy full SHA for 90f65b1 - Browse repository at this point
Copy the full SHA 90f65b1View commit details
Commits on Oct 31, 2023
-
* Regridding subclasses with abstract class * New YAML configuration changes * Revert changes to AnA metadata handling * removed July 2022 changes to AnA metadata handling and reverted to v2.1 behavior. * removed unneeded special-case code to artificially limit Alaska AnA cycle/forecast counting * AK Short-Range NBM support (NCAR#90) * Shift NBM cycles back 3 hours for AK Short-Range to match HRRR * Fix unintentional Alaska MR NBM shift * Onboarding updates * Fix parallel bug in NBM regridding * Fix incorrect time divisor in AK ExtAnA * Add spatial metadata globals to output files (NCAR#94) * Support NBM as primary forcing input (NCAR#93) * Additional NBM support * Add support for NBM input with external terrain file * Add creep-fill for NBM * Remove precipitation from CONUS HRRR forcing (NCAR#92) * Resolved merge conflicts with the main branch * Create README.md * Change `model_total_valid_times` attrib to int32 * Change globalNdv to -999999 * Add to ignored global metadata * Don't mask NLDAS params when reading netCDF var The _FillValue in the NLDAS params needs to be explicity changed to the globalNdv value so the masked areas can be reconstructed on the other side of an MPI broadcast * Adding pytest unit tests and related changes * Adding change to python version in the workflow * Adding change to requirements file by removing ESMpy in the workflow * Adding change to requirements file by removing unnecessary packages in the workflow * Adding change to requirements file by removing unnecessary packages in the workflow * Adding change to requirements file by removing unnecessary packages in the workflow * Adding change to requirements file by removing unnecessary packages in the workflow * Adding change to python version * Adding change to linux system type * Adding change to linux system type * Adding change to linux system type * Adding change to linux system type * Adding change to linux system type * Adding change to linux system type * Removed disutils from requirements * Removed packages from requirements * Removed packages from requirements * Removed packages from requirements * Removed packages from requirements * Removed packages from requirements * Removed packages from requirements * Adding python path dependencies * Adding python path dependencies * Adding python path dependencies * Adding python path dependencies * Adding WrfHydroForcing before core in import statements * Removing WrfHydroForcing before core in import statements * change to python path * change to python path * change to setup * change to setup * change to setup * python 3.8 * python 3.8 * python 3.8 * python 3.8 * python 3.8 * Restructured the tests and added relative paths * Added conda init to github workflow * Modified conda init to github workflow * Modified conda init to github workflow * Modified conda activate to source in github workflow * Modified conda activation in a non interactive terminal in github workflow * Added Installed /opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/wrf_hydro_mfe-1.0-py3.8.egg in github workflow * Added ESMPy install * Added ESMPy install * Added ESMPy install * Added ESMPy install * Added ESMPy install * Added ESMPy install * Delete configOptions_empty.yaml * Delete configOptions_invalid1.yaml * Delete configOptions_invalid2.yaml * Delete configOptions_missing.yaml * Delete configOptions_missingKey.yaml * Delete configOptions_valid.yaml * Delete inputForcings_empty.yaml * Delete inputForcings_invalid.yaml * Delete inputForcings_missing.yaml * Delete inputForcings_valid.yaml * Delete report.html * Fix issues with MRMS overlaying HRRR/RAP * Re-include removed HRRR precipitaion for use in Short-Range forecasts * Fix issue with MRMS missing data when used as an overlay in AnA * Set negative MRMS values to globalNdv silently * change negative value replacement from 0 to config_options.globalNdv * remove log warning (since negatives are most likely missing values in radar-only files) * Update tests.yml Removed line continuations ('\') in the "Set up ESMF and wgrib2" run step. They were contributing to the CI failures. * Updated ESMF_7_1_0r source URL (now hosted on Github) * Wrong spelling of ESMF tarball filename. * More paths fixed in tests.yml * More tests.yml fixes in conda setup * Added ubuntu install of mpich * Argh. Need sudo for installing packages on runner machine * Trying newer version of ESMF * More mods for building ESMFpy and wgrib2 * wgrib2 and ESMF working, trying to get pytest working * Added downloading of test data from google drive to tests. Removed dead config_data symlinks and config_data folder. Now if the folder doesn't exist it triggers the download to populate it. Added hook to allow either esmpy or ESMF python library name (name changed in ESMPy v 8.4.0) * Changed np.int to int * Added run_tests.py to download test data first before running pytest. Updated CI test to run tests out of a docker container instead of building everything as part of the workflow (building ESMF was taking > 20 minutes). Container name is currently andygaydos/wrf_hydro_forcing - this will need to be changed to point to final location of docker image. * Changed URL for gdrive download. Added run step to change permissions on local repo clone. Added propagation of exit status to python scripts * Bug fix * Whoops gdrive mod didn't work... * Changed docker image tag name to wrfhydro/wrf_hydro_forcing:latest * Changed np.int and np.bool to int and bool respectively, as the former produce errors. Additional YAML key checks (using try/except blocks). Updated ioMod.py to remove temporary grib2.tbl file from scratch dir when finished with it. Uncommented NETCDF variable definition, as it was causing 'not defined' errors * Added initial mpi tests. Modified run_tests.py to use mpirun with 4 processes. Added pytest-mpi to requirements. * Added tests and assertions for scattering an array using ESMF and mpi * Added geoMod tests. Added mpi scatter_array tests. Modified run_tests.py to test various MPI n_procs (none, 1, 4, and 8) for consistency * Forgot to add fixtures.py - a common set of pytest test fixtures * Added bias correction tests * Fixed bug in bias_correction.py (variable referenced before assignment, and indexing an array with a string instead of a number). Reworked test fixtures to more easily test outputs of various configurations (time of day, day of year, AnA vs forecast) * Added bias correction tests for CFS. Fixed yaml configuration bugs related to CFS-specific parameters in config.py and bias_correction.py --------- Co-authored-by: ishita9 <ishitas@ucar.edu> Co-authored-by: ishita9 <36771676+ishita9@users.noreply.github.com> Co-authored-by: ishita9 <ishita9@github.com> Co-authored-by: Ryan Cabell <rcabell@ucar.edu> Co-authored-by: Andrew Gaydos <gaydos@derecho4.hsn.de.hpc.ucar.edu>
Configuration menu - View commit details
-
Copy full SHA for db65282 - Browse repository at this point
Copy the full SHA db65282View commit details
Commits on Nov 1, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 9f912e6 - Browse repository at this point
Copy the full SHA 9f912e6View commit details -
Added downscaling tests. Moved some reusable code from bias correctio…
…n tests to shared fixtures.py file
Configuration menu - View commit details
-
Copy full SHA for 60a470e - Browse repository at this point
Copy the full SHA 60a470eView commit details