-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix shell check and merge develop #2
Fix shell check and merge develop #2
Commits on Nov 2, 2023
-
Corrects missing local variable reference (NOAA-EMC#2023)
Fixes missing `$` in variable reference in archive job. Resolves NOAA-EMC#2022
Configuration menu - View commit details
-
Copy full SHA for 247cdf8 - Browse repository at this point
Copy the full SHA 247cdf8View commit details -
Move analysis post to new stand-alone task (NOAA-EMC#1979)
This PR addresses issue NOAA-EMC#1130. A new task `postanl.sh` has been added such that the `anl` post-processing is not it's own task within the Rocoto workflow. This change prevents the GDAS experiment from breaking due to failures caused by the `anl` not being available for the first cycle. Resolves NOAA-EMC#1130
Configuration menu - View commit details
-
Copy full SHA for 31b05a9 - Browse repository at this point
Copy the full SHA 31b05a9View commit details -
Create separate GSI monitor jobs and remove from vrfy job (NOAA-EMC#1983
) This PR will move the GSI monitor tasks from the `vrfy` job into their own separate jobs in the rocoto mesh. This takes care of part of issue NOAA-EMC#235. Changes in this PR: 1. Create new individual GSI monitor jobs in the rocoto mesh to match how it is done in operations. The new jobs are: `gdasverfozn`, `gdasverfrad`, `gdasvminmon`, and `gfsvminmon`. The names correspond to the names in operations and the ecf script names. The jobs are only added to cycled mode, none of the jobs are run in forecast-only mode. 2. Updates to GSI monitor scripts: * Update `ush/jjob_header.sh` arguments in JJOB scripts to call new individual job names and configs. * Introduce `COM_*` variables and use of the `generate_com` utility. This results in the outputs landing in `COM` subfolders, like occurs in operations, instead of landing in the online archive. * Some variable cleanup in JJOB and ush scripts, includes cleaning up usage of `DATA` variable and its removal at the end of the jobs. * Add missing exit statements to `JGFS_ATMOS_VMINMON` and `JGDAS_ATMOS_VERFRAD`. 3. Create new rocoto job scripts for the new individual GSI monitor jobs. 4. Create new configs for the new individual GSI monitor jobs. 5. Add new individual GSI monitor jobs to resource configuration, using the same resources as in operations. 6. Move and rename GSI monitor job switches from `config.vrfy` to `config.base`. Also add switches to setup system. New switches to control each job being added to the rocoto mesh are: `DO_VERFOZN`, `DO_VERFRAD`, `DO_VMINMON` 7. Add GSI monitor job output to archival job and into resulting HPSS tarballs or local tarballs. This now matches what is done in operations. 8. Remove the GSI monitor tasks from the rocoto `vrfy` job and associated `config.vrfy`. Example output locations and files (not showing contents of oznmon/horiz or oznmon/time for brevity, see archival script for full list of output files): ``` kate.friedman@dlogin09:/lfs/h2/emc/ptmp/kate.friedman/comrot/testmonitor2> ll gdas.20211221/00/products/atmos/*mon gdas.20211221/00/products/atmos/minmon: total 112 -rw-r--r-- 1 kate.friedman emc 14030 Oct 25 15:00 2021122100.costs.txt -rw-r--r-- 1 kate.friedman emc 84932 Oct 25 15:00 2021122100.cost_terms.txt -rw-r--r-- 1 kate.friedman emc 808 Oct 25 15:00 2021122100.gnorms.ieee_d -rw-r--r-- 1 kate.friedman emc 808 Oct 25 15:00 2021122100.reduction.ieee_d -rw-r--r-- 1 kate.friedman emc 80 Oct 25 15:00 gnorm_data.txt gdas.20211221/00/products/atmos/oznmon: total 8 drwxr-sr-x 2 kate.friedman emc 4096 Oct 25 15:05 horiz drwxr-sr-x 2 kate.friedman emc 4096 Oct 25 15:05 time gdas.20211221/00/products/atmos/radmon: total 21036 -rw-r--r-- 1 kate.friedman emc 231 Oct 25 15:12 bad_diag.2021122100 -rw-r--r-- 1 kate.friedman emc 9035 Oct 25 15:12 bad_pen.2021122100 -rw-r--r-- 1 kate.friedman emc 1449 Oct 25 15:12 low_count.2021122100 -rw-r--r-- 1 kate.friedman emc 20523403 Oct 25 15:07 radmon_angle.tar.gz -rw-r--r-- 1 kate.friedman emc 217272 Oct 25 15:08 radmon_bcoef.tar.gz -rw-r--r-- 1 kate.friedman emc 502151 Oct 25 15:10 radmon_bcor.tar.gz -rw-r--r-- 1 kate.friedman emc 264480 Oct 25 15:12 radmon_time.tar.gz -rw-r--r-- 1 kate.friedman emc 684 Oct 25 15:12 warning.2021122100 kate.friedman@dlogin09:/lfs/h2/emc/ptmp/kate.friedman/comrot/testmonitor2> ll gfs.20211221/00/products/atmos/minmon/ total 88 -rw-r--r-- 1 kate.friedman emc 10530 Oct 25 14:55 2021122100.costs.txt -rw-r--r-- 1 kate.friedman emc 63882 Oct 25 14:55 2021122100.cost_terms.txt -rw-r--r-- 1 kate.friedman emc 808 Oct 25 14:55 2021122100.gnorms.ieee_d -rw-r--r-- 1 kate.friedman emc 608 Oct 25 14:55 2021122100.reduction.ieee_d -rw-r--r-- 1 kate.friedman emc 80 Oct 25 14:55 gnorm_data.txt ``` Lingering work and considerations: 1. Issue NOAA-EMC#1925 2. Consider not hardcoding the list of types in archival and perhaps tie it to something reliable or configurable (maybe into `config.verfozn`?): `subtyplist="gome_metop-b omi_aura ompslp_npp ompsnp_n20 ompsnp_npp ompstc8_n20 ompstc8_npp sbuv2_n19"` Resolves NOAA-EMC#1908
Configuration menu - View commit details
-
Copy full SHA for 93bc918 - Browse repository at this point
Copy the full SHA 93bc918View commit details -
Configuration menu - View commit details
-
Copy full SHA for bd4c56d - Browse repository at this point
Copy the full SHA bd4c56dView commit details
Commits on Nov 3, 2023
-
Fix staging of MOM data (NOAA-EMC#2028)
During the update to stage_ic, the copying of the additional res_N files for 0p25 was omitted. These are now properly copied. Resolves NOAA-EMC#2027
Configuration menu - View commit details
-
Copy full SHA for cbdc097 - Browse repository at this point
Copy the full SHA cbdc097View commit details -
Add two ucx modules in load_ufswm_modules.sh to solve C768 and C1152 …
…S2SW job hanging issue on WCOSS2 (NOAA-EMC#2021)
Configuration menu - View commit details
-
Copy full SHA for 2563806 - Browse repository at this point
Copy the full SHA 2563806View commit details -
Implement IAU Cycling Type with Marine Data Assimilation Enabled (NOA…
…A-EMC#1944) Allows cycling in IAU mode with the S2S UFS model, soca and the GSI. Most of the work related to making IAU work was done by @JessicaMeixner-NOAA . Thanks @JessicaMeixner-NOAA ! Resolves NOAA-EMC#1943 Refs NOAA-EMC#1776
Configuration menu - View commit details
-
Copy full SHA for 8d55126 - Browse repository at this point
Copy the full SHA 8d55126View commit details
Commits on Nov 6, 2023
-
Streamline CI reporting to PRs (NOAA-EMC#2026)
Cleaned up the logs for CI PRs sent to the GitHub Messages: - Eliminates redundant accumulative reporting - Reports on all created or skipped experiment instantiations in a single message - Gives one line report once for each experiment completion - Single final report completion of CI See some examples in forked [PR 195](TerrenceMcGuinness-NOAA#195) Resolves NOAA-EMC#2007
Configuration menu - View commit details
-
Copy full SHA for 75269e4 - Browse repository at this point
Copy the full SHA 75269e4View commit details -
Reduce gdas builds to 8 NOAA-EMC#2029 (NOAA-EMC#2036)
Reduce the default number of build jobs for the GDASApp to 8 from 16. This is needed for Orion as the build crashes during a linking step. Though not verified, it appears this may be caused by using too much memory with 16 builds. The issue disappears when using 8 build jobs. Resolves NOAA-EMC#2029
Configuration menu - View commit details
-
Copy full SHA for 34a73cf - Browse repository at this point
Copy the full SHA 34a73cfView commit details
Commits on Nov 7, 2023
-
Update UFS for dust fix and remove nitrates by default (NOAA-EMC#1989)
This PR updates the `ufs-weather-model` to the latest hash that included the dust fix (PR NOAA-EMC#1922). Along with this I removed the nitrates by default in support of the GEFSv13 EP4 and EP5 experiments. Removed unneeded inputs that should help with speed improvements.
Configuration menu - View commit details
-
Copy full SHA for 5183c43 - Browse repository at this point
Copy the full SHA 5183c43View commit details -
Configuration menu - View commit details
-
Copy full SHA for 83058ad - Browse repository at this point
Copy the full SHA 83058adView commit details -
Merge remote-tracking branch 'EMC/develop' into mergedev_develop_mdto…
…y_hr3 Conflicts: Externals.cfg sorc/checkout.sh
Configuration menu - View commit details
-
Copy full SHA for 59718ce - Browse repository at this point
Copy the full SHA 59718ceView commit details