Update to obsproc/v1.2.0 and prepobs/v1.1.0 #2903

KateFriedman-NOAA · 2024-09-11T18:26:00Z

Description

This PR updates the develop branch to use the newer operational obsproc/v1.2.0 and prepobs/v1.1.0. The obsproc/prepobs installs in glopara space on supported platforms use tags cut from the dev/gfsv17 branches in the respective repos. The installation of prepobs/v1.1.0 on WCOSS2 is called "gfsv17_v1.1.0" to help avoid GFSv16 users using it instead of the operational module.

Also, the HOMEobsproc path is updated to set an empty default for obsproc_run_ver. This both removes the need to set a default (and constantly update it, which is duplication) and avoid the unset variable error when the fcst jobs use their own load module script that does not know obsproc_run_ver:

export HOMEobsproc="${BASE_GIT:-}/obsproc/v${obsproc_run_ver:-}"

This PR also reverts the prepobs and fit2obs installs on MSU back to the glopara space from the temporary /work/noaa/global/kfriedma/glopara space installs.

Lastly, this PR also includes updates to complete issue #2844 (merge build.spack.ver and run.spack.ver).

Resolves #2291
Resolves #2840
Resolves #2844

Type of change

Regular updates to upstream components

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)

How has this been tested?

Cycled tests on WCOSS2 and Hera.

- Update to obsproc/v1.2.0 and prepobs/v1.1.0 - Revert back to glopara installs on Orion/Hercules - Remove default version for obsproc in config.base Refs NOAA-EMC#2291 Refs NOAA-EMC#2840 Refs NOAA-EMC#2841

Change Orion BASE_GIT to the same as Hercules (git_rocky9). Refs NOAA-EMC#2291 Refs NOAA-EMC#2840

Add ":-" to obsproc_run_ver variable setting for HOMEobsproc; the run.spack.ver version file is not currently sourced during forecast jobs and thus obsproc_run_ver is not set; it's not needed for forecast jobs however so allow it to be unset Refs NOAA-EMC#2291

Update module_base.gaea.lua to use the prepobs_run_ver instead of a hardcoded version. Refs NOAA-EMC#2840

Install of prepobs/v1.1.0 on WCOSS2 was installed under the folder name "1.1.0_gfsv17" to keep the v16 package from using it and to differentiate it from the ops version. Refs NOAA-EMC#2840

Refs NOAA-EMC#2840

This helps GFSv16 users avoid using this install instead of the operational module version. Refs NOAA-EMC#2840

aerorahul

looks ok

DavidHuber-NOAA

Looks good. I see that Jet is still using /lfs4 for Fit2Obs, but I think this is being covered in #2878, correct?

KateFriedman-NOAA · 2024-09-11T18:35:34Z

Awaiting install of prepobs/v1.1.0 on MSU before CI tests should start.

aerorahul · 2024-09-11T18:36:27Z

Before tests are run, do you see an issue in perhaps combining build.spack.ver and run.spack.ver into a single spack.ver and updating the host specific build and run version files?
A little busy work, but small enough and relevant to the changes in this PR.
Thoughts?

KateFriedman-NOAA · 2024-09-11T18:36:40Z

Looks good. I see that Jet is still using /lfs4 for Fit2Obs, but I think this is being covered in #2878, correct?

Yes, I believe @InnocentSouopgui-NOAA will be updating that path for Fit2Obs on Jet. They are likely also updating the prepobs path too but this PR will do that before their PR.

KateFriedman-NOAA · 2024-09-11T18:38:11Z

Before tests are run, do you see an issue in perhaps combining build.spack.ver and run.spack.ver into a single spack.ver and updating the host specific build and run version files? A little busy work, but small enough and relevant to the changes in this PR. Thoughts?

Sure, I can do that in this PR. Stay tuned...

KateFriedman-NOAA · 2024-09-11T18:39:13Z

Will now combine changes for issue #2844 into this PR.

Update machine ver files to source combined spack.ver Refs NOAA-EMC#2844

DavidHuber-NOAA

*.spack.ver merge looks good!

aerorahul

lgtm

emcbot · 2024-09-17T18:01:47Z

CI Passed on Hercules in Build# 4
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2903


Experiment C48_ATM_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 11:35:32 CDT 2024
Experiment C48_S2SW_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 12:54:51 CDT 2024
Experiment C96_atm3DVar_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 12:55:30 CDT 2024
Experiment C48_S2SWA_gefs_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 13:01:01 CDT 2024
Experiment C96C48_hybatmDA_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 13:01:31 CDT 2024

emcbot · 2024-09-17T18:41:55Z

Checkout Failed on Hera in Build# 6: Could not perform submodule update

emcbot · 2024-09-17T18:57:19Z

Checkout Failed on Hera in Build# 7: Could not perform submodule update

emcbot · 2024-09-18T00:11:46Z

CI Passed on Hera in Build# 8
Built and ran in directory /scratch1/NCEPDEV/global/CI/2903


Experiment C48mx500_3DVarAOWCDA_9a8a9001 Completed 2 Cycles: *SUCCESS* at Tue Sep 17 21:50:09 UTC 2024
Experiment C48_ATM_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 21:56:21 UTC 2024
Experiment C96C48_hybatmDA_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 23:03:55 UTC 2024
Experiment C96_atm3DVar_9a8a9001 Completed 3 Cycles: *SUCCESS* at Tue Sep 17 23:11:48 UTC 2024
Experiment C96C48_ufs_hybatmDA_9a8a9001 Completed 2 Cycles: *SUCCESS* at Tue Sep 17 23:34:25 UTC 2024
Experiment C48_S2SW_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 23:40:30 UTC 2024
Experiment C48_S2SWA_gefs_9a8a9001 Completed 1 Cycles: *SUCCESS* at Tue Sep 17 23:40:33 UTC 2024
Experiment C96C48_hybatmaerosnowDA_9a8a9001 Completed 3 Cycles: *SUCCESS* at Wed Sep 18 00:11:37 UTC 2024

This PR updates the `develop` branch to use the newer operational `obsproc/v1.2.0` and `prepobs/v1.1.0`. The obsproc/prepobs installs in glopara space on supported platforms use tags cut from the `dev/gfsv17` branches in the respective repos. The installation of `prepobs/v1.1.0` on WCOSS2 is called "gfsv17_v1.1.0" to help avoid GFSv16 users using it instead of the operational module. Also, the `HOMEobsproc` path is updated to set an empty default for `obsproc_run_ver`. This both removes the need to set a default (and constantly update it, which is duplication) and avoid the unset variable error when the fcst jobs use their own load module script that does not know `obsproc_run_ver`: ``` export HOMEobsproc="${BASE_GIT:-}/obsproc/v${obsproc_run_ver:-}" ``` This PR also reverts the prepobs and fit2obs installs on MSU back to the glopara space from the temporary `/work/noaa/global/kfriedma/glopara` space installs. Lastly, this PR also includes updates to complete issue NOAA-EMC#2844 (merge `build.spack.ver` and `run.spack.ver`). Resolves NOAA-EMC#2291 Resolves NOAA-EMC#2840 Resolves NOAA-EMC#2844

KateFriedman-NOAA added 9 commits August 26, 2024 15:41

Update obsproc and prepobs versions

7008246

- Update to obsproc/v1.2.0 and prepobs/v1.1.0 - Revert back to glopara installs on Orion/Hercules - Remove default version for obsproc in config.base Refs NOAA-EMC#2291 Refs NOAA-EMC#2840 Refs NOAA-EMC#2841

Update Orion BASE_GIT

f36c329

Change Orion BASE_GIT to the same as Hercules (git_rocky9). Refs NOAA-EMC#2291 Refs NOAA-EMC#2840

Merge branch 'NOAA-EMC:develop' into feature/obsproc_prepobs_versions

ac1724d

Update Gaea modulefile to use prepobs variable

668254e

Update module_base.gaea.lua to use the prepobs_run_ver instead of a hardcoded version. Refs NOAA-EMC#2840

Update prepobs_run_ver for WCOSS2

f9f5a48

Install of prepobs/v1.1.0 on WCOSS2 was installed under the folder name "1.1.0_gfsv17" to keep the v16 package from using it and to differentiate it from the ops version. Refs NOAA-EMC#2840

Merge branch 'NOAA-EMC:develop' into feature/obsproc_prepobs_versions

14e43e1

Remove "_gfsv17" from the prepobs version

9d0edb1

Refs NOAA-EMC#2840

Add "gfsv17_" to prepobs path before version

77f16e2

This helps GFSv16 users avoid using this install instead of the operational module version. Refs NOAA-EMC#2840

KateFriedman-NOAA requested review from aerorahul, WalterKolczynski-NOAA and DavidHuber-NOAA September 11, 2024 18:26

aerorahul previously approved these changes Sep 11, 2024

View reviewed changes

KateFriedman-NOAA marked this pull request as ready for review September 11, 2024 18:34

DavidHuber-NOAA previously approved these changes Sep 11, 2024

View reviewed changes

Merge build.spack.ver and run.spack.ver

c225660

Update machine ver files to source combined spack.ver Refs NOAA-EMC#2844

KateFriedman-NOAA dismissed stale reviews from DavidHuber-NOAA and aerorahul via c225660 September 11, 2024 19:01

DavidHuber-NOAA previously approved these changes Sep 11, 2024

View reviewed changes

KateFriedman-NOAA requested a review from aerorahul September 11, 2024 19:19

aerorahul previously approved these changes Sep 11, 2024

View reviewed changes

KateFriedman-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Sep 12, 2024

Merge branch 'NOAA-EMC:develop' into feature/obsproc_prepobs_versions

258f5d3

emcbot removed the CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS label Sep 12, 2024

emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Sep 17, 2024

KateFriedman-NOAA mentioned this pull request Sep 17, 2024

Move to contrib installation of spack-stack on Jet #2878

Merged

23 tasks

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Sep 17, 2024

KateFriedman-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Sep 17, 2024

emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Sep 17, 2024

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Sep 17, 2024

TerrenceMcGuinness-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Sep 17, 2024

WalterKolczynski-NOAA approved these changes Sep 18, 2024

View reviewed changes

WalterKolczynski-NOAA merged commit 7588d2b into NOAA-EMC:develop Sep 18, 2024
10 of 11 checks passed

KateFriedman-NOAA deleted the feature/obsproc_prepobs_versions branch September 18, 2024 12:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to obsproc/v1.2.0 and prepobs/v1.1.0 #2903

Update to obsproc/v1.2.0 and prepobs/v1.1.0 #2903

KateFriedman-NOAA commented Sep 11, 2024 •

edited

Loading

aerorahul left a comment

DavidHuber-NOAA left a comment

KateFriedman-NOAA commented Sep 11, 2024

aerorahul commented Sep 11, 2024

KateFriedman-NOAA commented Sep 11, 2024

KateFriedman-NOAA commented Sep 11, 2024

KateFriedman-NOAA commented Sep 11, 2024

DavidHuber-NOAA left a comment

aerorahul left a comment

emcbot commented Sep 17, 2024

emcbot commented Sep 17, 2024

emcbot commented Sep 17, 2024

emcbot commented Sep 18, 2024

Update to obsproc/v1.2.0 and prepobs/v1.1.0 #2903

Update to obsproc/v1.2.0 and prepobs/v1.1.0 #2903

Conversation

KateFriedman-NOAA commented Sep 11, 2024 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

aerorahul left a comment

Choose a reason for hiding this comment

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

KateFriedman-NOAA commented Sep 11, 2024

aerorahul commented Sep 11, 2024

KateFriedman-NOAA commented Sep 11, 2024

KateFriedman-NOAA commented Sep 11, 2024

KateFriedman-NOAA commented Sep 11, 2024

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

aerorahul left a comment

Choose a reason for hiding this comment

emcbot commented Sep 17, 2024

emcbot commented Sep 17, 2024

emcbot commented Sep 17, 2024

emcbot commented Sep 18, 2024

KateFriedman-NOAA commented Sep 11, 2024 •

edited

Loading