Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JEDI ATM lgetkf observer and solver jobs #2833

Merged

Conversation

RussTreadon-NOAA
Copy link
Contributor

@RussTreadon-NOAA RussTreadon-NOAA commented Aug 14, 2024

Description

This PR adds JEDI ATM lgetkf observer and solver jobs to global-workflow. This is approach is akin GSI-based eobs and eupd. Splitting the single JEDI ATM lgetkf job into separate observer and solver jobs improves memory and computational efficiency.

Resolves #2415

Dependency: GDASApp PR #1252

DO NOT MERGE THIS PR into global-workflow deveop until

  • GDASApp PR #1252 is merged into GDASApp develop
  • the gdas.cd hash in this PR is updated
  • JEDI format ATMS radiance bias correction files are staged in ICSDIR_ROOT for C96C48_ufs_hybatmDA CI (g-w issue #2836)

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES

How has this been tested?

The following tests have been conducted

  • run GDASApp ctests inside g-w. All tests Passed
  • run g-w CI for C96C48_ufs_hybatmDA, C48mx500_3DVarAOWCDA, and C96_atmaerosnowDA on WCOSS2
  • run cycled UFSDA-based 3dEnVar test over the period 2024050212 to 2024050300

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code

RussTreadon-NOAA and others added 30 commits March 20, 2024 18:28
@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Aug 27, 2024
@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Aug 27, 2024
@emcbot
Copy link

emcbot commented Aug 27, 2024

CI Passed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/2833


Experiment C48mx500_3DVarAOWCDA_067331be Completed 2 Cycles: *SUCCESS* at Tue Aug 27 17:43:14 UTC 2024
Experiment C48_ATM_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 17:43:14 UTC 2024
Experiment C48_S2SWA_gefs_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 19:14:52 UTC 2024
Experiment C48_S2SW_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 20:16:26 UTC 2024
Experiment C96_atm3DVar_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 20:22:32 UTC 2024
Experiment C96C48_hybatmDA_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 20:22:33 UTC 2024
Experiment C96C48_ufs_hybatmDA_067331be Completed 2 Cycles: *SUCCESS* at Tue Aug 27 20:22:34 UTC 2024
Experiment C96C48_hybatmaerosnowDA_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 20:34:50 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Aug 27, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Aug 27, 2024
@emcbot
Copy link

emcbot commented Aug 27, 2024

CI Update on Wcoss2 at 08/27/24 08:45:11 PM
============================================
Cloning and Building global-workflow PR: 2833
with PID: 216598 on host: clogin03

@emcbot emcbot added CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Aug 27, 2024
@emcbot
Copy link

emcbot commented Aug 27, 2024

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Tue Aug 27 20:55:28 UTC 2024 on clogin03
---------------------------------------------------
Build: Completed at 08/27/24 09:41:01 PM
Case setup: Completed for experiment C48_ATM_067331be
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_067331be
Case setup: Skipped for experiment C48_S2SWA_gefs_067331be
Case setup: Completed for experiment C48_S2SW_067331be
Case setup: Completed for experiment C96_atm3DVar_extended_067331be
Case setup: Skipped for experiment C96_atm3DVar_067331be
Case setup: Completed for experiment C96C48_hybatmaerosnowDA_067331be
Case setup: Completed for experiment C96C48_hybatmDA_067331be
Case setup: Completed for experiment C96C48_ufs_hybatmDA_067331be

@emcbot emcbot added CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Aug 28, 2024
@emcbot
Copy link

emcbot commented Aug 28, 2024

CI Passed on Hercules in Build# 2
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2833


Experiment C48_ATM_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 18:28:17 CDT 2024
Experiment C96C48_hybatmDA_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 19:29:07 CDT 2024
Experiment C96_atm3DVar_067331be Completed 3 Cycles: *SUCCESS* at Tue Aug 27 19:34:53 CDT 2024
Experiment C48_S2SWA_gefs_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 20:36:09 CDT 2024
Experiment C48_S2SW_067331be Completed 1 Cycles: *SUCCESS* at Tue Aug 27 20:41:31 CDT 2024

@emcbot emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Aug 28, 2024
@emcbot
Copy link

emcbot commented Aug 28, 2024

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_067331be *** SUCCESS *** at 08/27/24 11:07:25 PM
Experiment C48_S2SW_067331be *** SUCCESS *** at 08/27/24 11:14:20 PM
Experiment C96C48_hybatmDA_067331be *** SUCCESS *** at 08/27/24 11:56:34 PM
Experiment C96C48_hybatmaerosnowDA_067331be *** SUCCESS *** at 08/28/24 12:42:26 AM
Experiment C96C48_ufs_hybatmDA_067331be *** SUCCESS *** at 08/28/24 01:42:21 AM
Experiment C96_atm3DVar_extended_067331be *** SUCCESS *** at 08/28/24 09:14:34 AM

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 9ad7d3e into NOAA-EMC:develop Aug 28, 2024
10 of 11 checks passed
@RussTreadon-NOAA
Copy link
Contributor Author

Thank you @WalterKolczynski-NOAA !

@RussTreadon-NOAA RussTreadon-NOAA deleted the feature/ensda_obs branch August 28, 2024 16:24
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 30, 2024
* origin/develop:
  Add 3 and 9 hr increment files to IC staging (NOAA-EMC#2876)
  Add diffusion/diag B for aerosol DA and some other needed changes (NOAA-EMC#2738)
  Correct ocean `MOM.res_#` stage copy (NOAA-EMC#2868)
  Support coupling on AWS (NOAA-EMC#2859)
  Add JEDI ATM lgetkf observer and solver jobs (NOAA-EMC#2833)
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Sep 9, 2024
* origin/develop:
  Create JEDI class (NOAA-EMC#2805)
  Restructure the bufr sounding job    (NOAA-EMC#2853)
  Add an archive task to GEFS system to archive files locally (NOAA-EMC#2816)
  Reenable Orion Cycling Support (NOAA-EMC#2877)
  Eliminate race conditions and remove DATAROOT last in cleanup (NOAA-EMC#2893)
  Update aerosol climatology to 2013-2024 mean (NOAA-EMC#2888)
  Add ability to run CI test C96_atm3DVar.yaml to Gaea-C5 (NOAA-EMC#2885)
  Support global-workflow GEFS C48 on Google Cloud (NOAA-EMC#2861)
  Add 3 and 9 hr increment files to IC staging (NOAA-EMC#2876)
  Add diffusion/diag B for aerosol DA and some other needed changes (NOAA-EMC#2738)
  Correct ocean `MOM.res_#` stage copy (NOAA-EMC#2868)
  Support coupling on AWS (NOAA-EMC#2859)
  Add JEDI ATM lgetkf observer and solver jobs (NOAA-EMC#2833)
  Fix gdas build on Gaea and add Gaea to available CI list (NOAA-EMC#2857)
  Support ATM forecast only on Google (NOAA-EMC#2832)
  Add GEFS C48 support on AWS (NOAA-EMC#2818)
  Update omega calculation (NOAA-EMC#2751)
  Add snow DA update and recentering for the EnKF forecasts (NOAA-EMC#2690)
  support ATM forecast only on Azure (NOAA-EMC#2827)
  Convert staging job to python and yaml (NOAA-EMC#2651)
  Fixed test on UNAVAILBLE in python Rocoto check (NOAA-EMC#2842)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add JEDI ATM equivalent to GSI eobs
6 participants