Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port the GSI to Acorn #793

Merged
merged 5 commits into from
Oct 24, 2024

Conversation

DavidHuber-NOAA
Copy link
Collaborator

@DavidHuber-NOAA DavidHuber-NOAA commented Sep 24, 2024

Description

This ports the GSI to Acorn to allow testing of new library installations before official installations go to Cactus and Dogwood. This is a prerequisite (but does not fix) for #642, #662, and #665.

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?
Ctests were run on Acorn. I do not currently have rstprod access on the system, so only the HAFS and RTMA tests passed.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • New and existing tests pass with my changes

@RussTreadon-NOAA
Copy link
Contributor

Install and build DavidHuber-NOAA:feature/acorn at 542b6f7 on Acorn. Run ctests with the following results

Test project /lfs/h2/emc/da/noscrub/russ.treadon/git/gsi/acorn/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  727.26 sec
2/6 Test #6: global_enkf ......................   Passed  851.52 sec
3/6 Test #2: rtma .............................   Passed  968.93 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1153.76 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1213.47 sec
6/6 Test #1: global_4denvar ...................   Passed  1682.33 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1682.34 sec

All tests pass!

Note: Use DavidHuber-NOAA:feature/acorn as updat and contrl.

Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some requested changes in gsi_acornn.intel.lua and optional comments on sub_acorn.

modulefiles/gsi_acorn.intel.lua Outdated Show resolved Hide resolved
modulefiles/gsi_acorn.intel.lua Outdated Show resolved Hide resolved
ush/module-setup.sh Outdated Show resolved Hide resolved
ush/sub_acorn Outdated Show resolved Hide resolved
@RussTreadon-NOAA
Copy link
Contributor

@DavidHuber-NOAA , @aerorahul is on leave until 11/4. Is there anyone else who can complete the second peer review?

Today I will install DavidHuber-NOAA:feature/acorn on Cactus, Hera, Hercules, & Orion to run ctests.

@DavidHuber-NOAA
Copy link
Collaborator Author

@BinLiu-NOAA @ZhanZhang-NOAA Since you both had helped with the original HAFS GSI port to Acorn, would one of you mind reviewing this PR?

@RussTreadon-NOAA
Copy link
Contributor

Cactus (WCOSS2) ctest results
Install develop at 665f9b4 and DavidHuber-NOAA:feature/acorn at b0aa72d on Orion. Run ctests with following results

Test project /lfs/h2/emc/da/noscrub/russ.treadon/git/gsi/pr793/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  1332.26 sec
2/6 Test #6: global_enkf ......................   Passed  1337.77 sec
3/6 Test #5: hafs_3denvar_hybens ..............   Passed  1641.24 sec
4/6 Test #4: hafs_4denvar_glbens ..............   Passed  1882.29 sec
5/6 Test #2: rtma .............................   Passed  2532.07 sec
6/6 Test #1: global_4denvar ...................   Passed  2644.91 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 2644.92 sec

Hera ctest results
Repeat the above installation on Hera. Run ctests with the following results

Test project /scratch1/NCEPDEV/da/Russ.Treadon/git/gsi/pr793/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  494.59 sec
2/6 Test #6: global_enkf ......................   Passed  760.73 sec
3/6 Test #2: rtma .............................   Passed  973.52 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1108.23 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1170.90 sec
6/6 Test #1: global_4denvar ...................   Passed  1804.89 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1804.93 sec

Hercules ctest results
Repeat the above installation on Hercules. Run ctests with the following results

Test project /work/noaa/da/rtreadon/git/gsi/pr793/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  605.12 sec
2/6 Test #6: global_enkf ......................   Passed  905.35 sec
3/6 Test #2: rtma .............................   Passed  1325.60 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1392.84 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1522.91 sec
6/6 Test #1: global_4denvar ...................   Passed  1923.31 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1923.32 sec

Orion ctest results
Repeat the above installation on Orion. Run ctests with the following results

Test project /work2/noaa/da/rtreadon/git/gsi/pr793/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  1094.18 sec
2/6 Test #6: global_enkf ......................   Passed  1279.48 sec
3/6 Test #2: rtma .............................   Passed  1749.78 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  3081.64 sec
5/6 Test #4: hafs_4denvar_glbens ..............***Failed  3383.25 sec
6/6 Test #1: global_4denvar ...................***Failed  4023.81 sec

67% tests passed, 2 tests failed out of 6

Total Test time (real) = 4023.82 sec

The following tests FAILED:
          1 - global_4denvar (Failed)
          4 - hafs_4denvar_glbens (Failed)
Errors while running CTest

global_4denvar failed due to

The runtime for global_4denvar_hiproc_updat is 1041.124062 seconds.  This has exceeded maximum allowable threshold time of 801.522912 seconds,
resulting in Failure of timethresh2 the regression test.

hafs_4denvar_glbens failed for the same reason

The runtime for hafs_4denvar_glbens_loproc_updat is 978.292937 seconds.  This has exceeded maximum allowable threshold time of 814.187132 seconds,
resulting in Failure time-thresh of the regression test.

Neither of these are fatal failures. A rerun of the ctests yields a Passed result for all tests

Test project /work/noaa/da/rtreadon/git/gsi/pr793/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #6: global_enkf ......................   Passed  727.86 sec
2/6 Test #3: rrfs_3denvar_rdasens .............   Passed  966.84 sec
3/6 Test #2: rtma .............................   Passed  1747.13 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  3141.75 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  3201.12 sec
6/6 Test #1: global_4denvar ...................   Passed  3721.94 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 3721.95 sec

Summary
All ctests eventually Passed on Cactus, Hera, Hercules, and Orion. This is an expected results since this PR does not modify source code. Nor is the build on these machines altered by this PR.

Copy link
Contributor

@RussTreadon-NOAA RussTreadon-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve.

@RussTreadon-NOAA
Copy link
Contributor

@DavidHuber-NOAA , if you have addressed @aerorahul 's review comments, please go ahead and resolve the conversations. Thanks.

ctests have been run on all platforms with expected results. Once we get a second peer review and approval we can schedule this PR for merger into develop.

@RussTreadon-NOAA
Copy link
Contributor

@BinLiu-NOAA @ZhanZhang-NOAA : If you have time, your review of this PR is much appreciated. Closure of this PR is a prerequisite to making progress on several open GSI issues. Hence my reaching out to you.

@RussTreadon-NOAA
Copy link
Contributor

We would like to merge this PR into develop no later than next Wednesday, 10/30/2024.

Copy link
Contributor

@BinLiu-NOAA BinLiu-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. And thanks for working on this and enabling Acorn support!

@RussTreadon-NOAA
Copy link
Contributor

Thank you @BinLiu-NOAA for your approval.

@DavidHuber-NOAA , I'll reach out to the GSI Handling Review team to schedule merger of this PR into develop.

@RussTreadon-NOAA RussTreadon-NOAA merged commit b0e3cba into NOAA-EMC:develop Oct 24, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants