Update develop-ref after #2701 #2703
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://github.com/avatars/in/15368?s=60&v=4){kind=small}
…at causes GHA disk to exceed capacity when too many data volumes are created. Use specific commit of coveralls python action to ensure it will not change
…were accidentally added with PR #1984
…n a pull request from a fork because the authentication fails
… to push this change and the NOAA OAR reporting label to all the METplus repos.
…lso fixing the get_lablels.sh to search for up to 200 existing labels. Also work on the log messages.
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
* prevent divide by zero if run_count is 0 * Per #2547, add support for setting -valid_beg and -valid_end command line arguments. Added changes to make ASCII2NC wrapper able to run for all runtime frequencies * Refactored system_util preprocess_file function to reduce cognitive complexity and add quotation marks around 'filenames' that contain spaces (typically python embedding commands) so explicit handling of these cases don't need to be done downstream. Added unit tests to test more python embedding cases * remove logic to add quotes around input file since it is handled already in preprocess_file * changed find_input_files function to return None or a time_info dictionary to be consistent across wrappers * added ReformatPointWrapper to use as parent class for ASCII2NC, MADIS2NC, PB2NC, and Point2Grid wrappers to consistently handle tools that reformat point observation data. Moved verbosity to the end of commands * clean up pb2nc wrapper to be more consistent with other ReformatPoint wrappers * per #2547, added glossary entries for new config variables to set -valid_beg/end and added commented example to basic use case config file * added glossary entries for *_RUNTIME_FREQ variables * Per #2513, remove TC_RMW_MAX_RANGE_KM
* Updated for new data * Updated to match new data * Updates due to new data/updates to data * Updates due to new data/data variables * Changes due to changes to data variables, date * Updates due to changes to data * Update the date to reflect the new data (with updates to variables) * fixed error with formatting * Remove redundant instructions. * For testing * Update use_case_groups.json * Revert to original location of input data to use the same data for all three FV3 Physics Tendency use cases. * Revert to original location of data from tarball * Remove typo in file directory name * Update use_case_groups.json Returned 10-12 to follow 11 * Update use_case_groups.json revert to false for testing the short range use cases for FV3 physics tendency * Update use_case_groups.json fix alignment of opening curly brace * Update use_case_groups.json revert to original formatting * removed config variable that should be set by the user because it is specific to the user's environment --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
* refactoring to reduce duplicate/redundant code, reduce cognitive complexity to satisfy SonarQube, etc
* change wording of error log to satisfy test
* set -input_thresh argument if set for add, derive, and sum methods. refactor setting of method arguments, e.g. -add, -sum, etc., cleanup
* refactor how level is handled in find_data function so that if the level has already been set, it will use that value, otherwise try to get it from {data_type}_level, e.g. fcst_level, otherwise set it to 0 to prevent errors
* refactor duplicate code into function to satisfy SonarQube
* removed unused variable
* use find_data function to find input files to be consistent with other wrappers, only allow multiple input files to be found for a given call to find_data if using the -derive method
* fix typo in key
* suppress warnings when files aren't found because it is expected
* formatting to be consistent in doc string
* per #2578, add MISSING before file path that is not found if input_thresh is set and less than 1.0, added unit test to ensure correct behavior occurs
* add documentation blocks for new functions, ci-run-all-diff
* use pytest fixture instead of local function
* add support for setting -vld_thresh argument
* per #2578, added documentation and tests for setting -input_thresh and -vld_thresh
* moved verbosity argument to end of command to more easily change it when debugging
* refactor to reduce cognitive complexity to satisfy SonarQube
* update usage statement to include METplus version number
* add copy button for code blocks to easily copy/paste commands * turn off copy button always visible but leave code block so it can be turned on easily if desired
Update METplus-6.0.0 development schedule.
* Initial commit for QBO driver * Adding initial documentation for Stratosphere * Updating documentation typo * Updates to Polar plot * Updates to Stratosphere use cases and documentation * Updates to polar use case * Updating documentation * Updates to Polar and QBO * Updated documentation for use cases that cannot be run in actions * Removed some unnecessary imports * Fixed staging dir settings * documentation update * More documentation edits * More updates * Fixed link in documentation * Updating documentation * Updated bullet list * More updates * Added some additional comments to the .conf file * Fixed typo in output file name in documentation * Updated output section of documentation * Updated documentation * Fixed list that was not working * Updated broken link * Fixed broken link again * Fixed another typo * Trying to fix documentation error * Trying to fix indent error * fixed capital typo * Added clarification
…command to upload asset
* Per #2656, update the read_iodav2_mpr.py scripts to write 3 additional values of NA in each MPR line corresponding to the OBS_CLIMO_CDF, FCST_CLIMO_MEAN, and FCST_CLIMO_STDEV columns. * Per #2656, update the ugrid_lfric_mpr.py to write 3 additional values of NA in each MPR line corresponding to the OBS_CLIMO_CDF, FCST_CLIMO_MEAN, and FCST_CLIMO_STDEV columns.
* Per 2679, updating user support documentation * Per #2679, fixing formatting error * Update user_support.rst * Update user_support.rst --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
* Per #2452, updating MET official release instructions * Update to indicate the "DTC" web server machine Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Updated language for tar file Co-authored-by: John Halley Gotway <johnhg@ucar.edu> --------- Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
* Refactor to prevent calling sys.exit from functions that could cause the calling script to exit unintentionally. Instead return None or False and only sys.exit a non-zero value from a script that is called explicitly. Refactored validate config logic to reduce cognitive complexity to satisfy SonarQube * per #2596, ensure the same RUN_ID is set for a given METplus run even if instances are used in the PROCESS_LIST * fixed linter complaints * remove unused imports * cleanup for SonarQube and linter * remove unused imports * import parent wrappers directly so that PyCharm can follow class hierarchy * per #2612, add support for processing groups of forecast leads * remove call to prune_empty function and ci-run-all-diff * fixed check on DataFrame to properly check for an empty one * remove prune_empty function that is no longer used * update conda environment requirements for metdataio * set label to Group<n> for LEAD_SEQ_<n> if LEAD_SEQ_<n>_LABEL is not set (instead of erroring), use forecast lead label for lead if set, log lead group label and formatted list of leads for each run * increase timeout for request from 60 to 90 * handle exception and output an error message with a suggestion to rerun the failed GHA jobs when a hiccup with DockerHub prevents the list of available Docker data volumes from being obtained. FYI @jprestop * cleanup to address SonarQube complaints, move call to exit out of main function so a script calling the function does not exit when the function errors * cleanup to appease SonarQube * per #2612, add support for creating groups of forecast leads by specifying a list of forecast leads and a time interval to create groups of leads * per #2612, add unit tests for function that gets groups of forecast leads and added tests for new groups capabilities * various refactoring to satisfy SonarQube complaints * renamed config LEAD_SEQ_DIVISIONS to LEAD_SEQ_GROUP_SIZE and LEAD_SEQ_DIVISIONS_LABEL to LEAD_SEQ_GROUP_LABEL * per #2612, add documentation for new config variables including examples * fix empty check for DataFrame * fix rst typo * minor cleanup to satisfy linter * Per #2612, if runtime freq is RUN_ONCE_PER_LEAD and groups of forecast leads are specified, then the wrapper will be run once per lead GROUP instead of once for each unique lead in the lead groups. Update unit test to reflect new behavior
…all to cartopy feature download script -- see SciTools/cartopy#2263
* remove support for old environment variables CLIMO_MEAN_FILE and CLIMO_STDEV_FILE that were used in METplus versions earlier than v4.0.0 * fixed SonarQube bugs * add better error checking for DockerHub queries that occasionally fail in GHA * update pycharm files * per #2622, support setting climo_mean and climo_stdev dictionary values inside fcst and obs dictionaries for grid_stat, ensemble_stat, point_stat, and series_analysis. Quietly add support for setting climo dictionaries inside the ens dictionary for gen_ens_prod -- this support is not needed because there is only 1 dictionary that may contain the climo dicts, but adding support for completeness so it can be assumed that {APP_NAME}_{DICT}_{SUBDICT}_{VAR} is supported, e.g. GEN_ENS_PROD_ENS_CLIMO_MEAN_REGRID_METHOD * turn on use cases that started failing from MET changes * turn off all use cases from push events * reorder assert so that expected vs. actual in PyCharm diff are accurate * remove commented code * added unit tests for climo VAR<n> variables that were missing * add VAR<n> variables used to specify climo_mean/stdev field values
* exclude build directory from PyCharm project * refactor to satisfy SonarQube complaints * Per #2651, add support for setting -aggr argument with file path. Refactor functions to resolve SonarQube complaints. Enhance unit tests to test multiple init times and -aggr argument * add new config variables to basic use case * change settings to match MET unit test * added new config variables to set -aggr argument in SeriesAnalysis wrapper to documentation * Per #2651, add new use case to demonstrate using the -aggr argument to SeriesAnalysis * turn on new use case to test it in GHA * remove blank line * turn off use case to prepare for PR
23 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#2701 added a new use case (SeriesAnalysis_aggr) that produces new output
Created by @georgemccabe