-
Notifications
You must be signed in to change notification settings - Fork 168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix file promotion and updated versioned FIX_DIR paths in workflow #966
Comments
- Source new fix.ver file to get fix subfolder version values for setting up fix folder symlinks. - Change "fix_NEW" to "fix" for FIX_DIR paths: use newly reorganized and versioning fix set. - Remove "fix_" from /fix folder symlink names. - Update /fix folder symlink creation to use version variables from fix.ver. - Update GSI and GDAS fix symlinks. Refs: #966
- Change FV3JEDI_FIX in related configs to change the path subfolder from "fix_jedi" to "gdas". Refs: #966
- Remove "fix_" text from fix file paths in several configs. Refs: #966
- Update /scripts and /ush scripts to remove the "fix_" text from fix subfolder paths. Refs: #966
Ran control (with fix_NEW) and test (using updated fix paths) on Orion (both atmos-only and coupled). The test output matches the control run. atmos-only run spot checking:
|
Ran |
@GeorgeGayno-NOAA Below are the UFS_UTILS files I had to modify for the updated fix subfolders (see clone on Orion):
We'll need you to make these same changes in UFS_UTILS develop and then we'll update the UFS_UTILS version as the related global-workflow changes go in. I have another issue (#974) that competes with this work a bit. May need to coordinate when we commit these fix changes for after we update to the newer UFS_UTILS tag. Not 100% sure at the moment. |
Just opened an issue to track this: ufs-community/UFS_UTILS#687 |
Awesome, thanks @GeorgeGayno-NOAA ! |
Just a note here to make sure we're on the same page: link_fixdirs should probably take in the versions just like we do for modulefiles. Not sure if Kate's current changes do that, but the final solution should. |
- Create a new version folder to hold version files. - Create a new fix.ver to hold fix file versions. Refs: #966
Great point @WalterKolczynski-NOAA , thanks! @GeorgeGayno-NOAA Please see the link script changes I made in my g-w branch here: fdd640f I also created a I updated my changes to my UFS_UTILS copy to show what the symlinks should now be based on the contents of
|
Global workflow recently updated their fix directory structure to incorporate versioning using dates within the directory tree. Additionally, some files were moved and directories renamed (including removing all the 'fix_' prefixes of subdirectories). This commit updates the fix paths in UFS utils to match the new directory structure, and also take version numbers for each of the fix component directories, similar to how versions are used for modulefiles.) Fixes ufs-community#687 Refs: NOAA-EMC/global-workflow#966
Global workflow recently updated their fix directory structure to incorporate versioning using dates within the directory tree. Additionally, some files were moved and directories renamed (including removing all the 'fix_' prefixes of subdirectories). This commit updates the fix paths in UFS utils to match the new directory structure, and also take version numbers for each of the fix component directories, similar to how versions are used for modulefiles.) Fixes ufs-community#687 Refs: NOAA-EMC/global-workflow#966
- Source new fix.ver file to get fix subfolder version values for setting up fix folder symlinks. - Change "fix_NEW" to "fix" for FIX_DIR paths: use newly reorganized and versioning fix set. - Remove "fix_" from /fix folder symlink names. - Update /fix folder symlink creation to use version variables from fix.ver. - Update GSI and GDAS fix symlinks. Refs: NOAA-EMC#966
- Change FV3JEDI_FIX in related configs to change the path subfolder from "fix_jedi" to "gdas". Refs: NOAA-EMC#966
- Remove "fix_" text from fix file paths in several configs. Refs: NOAA-EMC#966
- Update /scripts and /ush scripts to remove the "fix_" text from fix subfolder paths. Refs: NOAA-EMC#966
- Create a new version folder to hold version files. - Create a new fix.ver to hold fix file versions. Refs: NOAA-EMC#966
Global workflow recently updated their fix directory structure to incorporate versioning using dates within the directory tree. Additionally, some files were moved and directories renamed (including removing all the 'fix_' prefixes of subdirectories). This commit updates the fix paths in UFS utils to match the new directory structure, and also take version numbers for each of the fix component directories, similar to how versions are used for modulefiles.) Fixes ufs-community#687 Refs: NOAA-EMC/global-workflow#966
@WalterKolczynski-NOAA fyi, the following changes will need to be made to the GLDAS scripts for this issue:
|
Global workflow recently updated their fix directory structure to incorporate versioning using dates within the directory tree. Additionally, some files were moved and directories renamed (including removing all the 'fix_' prefixes of subdirectories). This commit updates the fix paths in UFS utils to match the new directory structure, and also take version numbers for each of the fix component directories, similar to how versions are used for modulefiles.) Fixes ufs-community#687 Refs: NOAA-EMC/global-workflow#966
An earlier change had inadvertently stopped checking out GSI Monitor, GSI Utils, and GLDAS when GDAS is used instead of GSI. Refs: NOAA-EMC#966
First wave of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
Second wave of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
Wave 3 of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
Wave 4 of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
The last merge from develop introduced a minor syntax error in the link script due to a bad conflict resolution. The issue is now corrected. Refs: NOAA-EMC#966
GLDAS scripts were recent moved into the workflow repo and need to be updated for the new fix structure like other components. Refs: NOAA-EMC#622, NOAA-EMC#966
Moved the setting of FIXDIR higher so it errors on an unknown machine sooner. Also removed redundant version file load. Refs: NOAA-EMC#966
- Source new fix.ver file to get fix subfolder version values for setting up fix folder symlinks. - Change "fix_NEW" to "fix" for FIX_DIR paths: use newly reorganized and versioning fix set. - Remove "fix_" from /fix folder symlink names. - Update /fix folder symlink creation to use version variables from fix.ver. - Update GSI and GDAS fix symlinks. Refs: NOAA-EMC#966
- Change FV3JEDI_FIX in related configs to change the path subfolder from "fix_jedi" to "gdas". Refs: NOAA-EMC#966
- Remove "fix_" text from fix file paths in several configs. Refs: NOAA-EMC#966
- Update /scripts and /ush scripts to remove the "fix_" text from fix subfolder paths. Refs: NOAA-EMC#966
- Create a new version folder to hold version files. - Create a new fix.ver to hold fix file versions. Refs: NOAA-EMC#966
An earlier change had inadvertently stopped checking out GSI Monitor, GSI Utils, and GLDAS when GDAS is used instead of GSI. Refs: NOAA-EMC#966
First wave of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
Second wave of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
Wave 3 of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
Wave 4 of linter fixes for removing gfs-utils code/fix reorg. Refs: NOAA-EMC#966
The last merge from develop introduced a minor syntax error in the link script due to a bad conflict resolution. The issue is now corrected. Refs: NOAA-EMC#966
GLDAS scripts were recent moved into the workflow repo and need to be updated for the new fix structure like other components. Refs: NOAA-EMC#622, NOAA-EMC#966
Moved the setting of FIXDIR higher so it errors on an unknown machine sooner. Also removed redundant version file load. Refs: NOAA-EMC#966
Removes all of the code and scripts that were moved to the new [gfs-utils repo](https://github.com/NOAA-EMC/gfs-utils) and adjusts workflow scripts to build and use them from the new location. Some of the build scripts had unnecessary calls to machine-setup that are removed because the lower-level script already has the same functionality. This PR also includes updates to use the new fix organization. This includes the addition of a fix versions file, updates to the link script, and some changes in the fix directories used by scripts to account for files that have been relocated. The versions file sets the version number for each component of fix, so that fix can be more easily maintained and documented. The initial versions are all the same, and correspond to the old fix_NEW directory (other than some directories have been renamed or reorganized). Except gdas, which has already had a new set of fix files added. The fix update also required an update to UFS-Utils. Finally, the link script has been updated to match the syntax of the rest of the build system ([checkout.sh](sorc/checkout.sh) and [build_all.sh](sorc/build_all.sh)). [link_workflow.sh](sorc/link_workflow.sh) now detects the machine automatically instead of requiring an argument, and dev mode, which used to be set using `emc` as an argument, is now assumed. To run in ops mode (copy instead of link), the `-o` option is used. The full syntax is now simply: ``` ./link_workflow.sh [-o] Options: -o: Run in operations (NCO) mode (copy instead of link) ``` Fixes #356 Fixes #966
Description
This issue will document the promotion of
fix_NEW
tofix
and the associated workflow linking updates to the newly restructured and versioned fix file sets on supported platforms. The initial promotion tasks will happen over a two day period and then the workflow updates will occur.Checklist
August 10th (promotion day 1)
fix
tofix_20210923
(Hera)fix
tofix_20210923
(WCOSS2 Cactus)fix
tofix_20210923
(WCOSS2 Dogwood)fix
tofix_20210923
(Orion)fix
tofix_20210923
(Jet)fix
in to replacefix
(Hera)fix
from Hera in to replacefix
on WCOSS2 Cactusfix
from Hera in to replacefix
on WCOSS2 Dogwoodfix
from Hera in to replacefix
on Orionfix
from Hera in to replacefix
on Jetfix_NEW
(fix_NEW_test
) into global space for sanity testing on Herafix_NEW_test
on HeraAugust 11th (promotion day 2)
fix_NEW
tofix_NEW_20220726
(Hera)fix_NEW
tofix_NEW_20220726
(WCOSS2 Cactus)fix_NEW
tofix_NEW_20220726
(WCOSS2 Dogwood)fix_NEW
tofix_NEW_20220726
(Orion)fix_NEW
tofix_NEW_20220726
(Jet)fix_NEW
with symlinks to updatedfix
(Hera)fix_NEW
with symlinks to updatedfix
(WCOSS2 Cactus)fix_NEW
with symlinks to updatedfix
(WCOSS2 Dogwood)fix_NEW
with symlinks to updatedfix
(Orion)fix_NEW
with symlinks to updatedfix
(Jet)August 19th (cleanup)
fix_NEW_20220726
(Hera)fix_NEW_20220726
(WCOSS2 Cactus)fix_NEW_20220726
(WCOSS2 Dogwood)fix_NEW_20220726
(Orion)fix_NEW_20220726
(Jet)August 26th (cleanup)
fix_20210923
(Hera)fix_20210923
(WCOSS2 Cactus)fix_20210923
(WCOSS2 Dogwood)fix_20210923
(Orion)fix_20210923
(Jet)Workflow updates
link_workflow.sh
FIX_DIR
s to point to newfix
(Implement fix reorg and remove gfs-utils code #1009)FIX_DIR
paths inlink_workflow.sh
(Implement fix reorg and remove gfs-utils code #1009)The text was updated successfully, but these errors were encountered: