Bring Feature/ufo geovals up to date. #798
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s multiscale EnVar and parallel ensemble IO for EnVar for FV3-LAM (NOAA-EMC#504) The following capabilities developed by OU MAP lab are included (1) Further development for simultaneous multiscale EnVar for both global and regional DA (1a) spatial scale-dependent localization (SDL; contributed by Ting Lei and Daryl Kleist/EMC) is implemented in EnVar as described in Huang et al 2021, MWR for the global NWP application. (1b) variable-dependent localization (VDL) method by Wang and Wang 2022, JAMES is implemented in EnVar. (2)Development of parallel ensemble IO for EnVar for FV3-LAM Implement an approach to simultaneously read in all ensemble members for EnVar. Specifically, parallel ensemble IO for both conventional and radar EnVar for FV3-LAM is implemented by reading in all ensemble members simultaneously. (3) Direct assimilation of radar reflectivity for EnVar for RRFS The direct radar reflectivity assimilation approach by Wang and Wang 2017, MWR is implemented and tested for FV3-LAM. Fixes NOAA-EMC#468
**Description** Added code to divide hx_modens by qsat, for consistency with treatment of q as q/qsat. Fixes issue NOAA-EMC#544
merge changes made in recent GFS v16.3.x implementations into develop
…le for FV3-LAM EnVar (NOAA-EMC#541) This PR adds options (weight_ens_gfs and weight_ens_fv3sar) to tune weights of mixed ensemble for FV3-LAM EnVar (NOAA-EMC#538). Regression tests for global 3dvar/4denvar/4dvar are not completed yet, but for other tests, issues are not found except for "failed the scalability test" and "exceeded maximum allowable hardware memory limit" on Orion. Fixes NOAA-EMC#538 Co-authored-by: Sho Yokota <syokota@Orion-login-1.HPC.MsState.Edu>
**Description** This update includes several functions added for dBZ analysis in EnKF: 1, Add dBZ ncdiag output for EnKF analysis 2, Enhanced binary dBZ diag output for EnKF analysis 3, Fix spurious analysis increments when assimilating reflectivity (from OU MAP) 4, Added EnKF interface for read in dBZ from FV3LAM ensemble forecast (from OU MAP) Fixed a bug for read in subdomain surface restart files Fixes NOAA-EMC#534
**Description** The ncio and ncdiag modules have been promoted from para to prod on WCOSS2. This PR replaces the para load of these modules with prod on WCOSS2. Fixes NOAA-EMC#554
…me covariances in EnVar (NOAA-EMC#542) This PR modifies options (i_ensloccov4{scl,var,tim} -> r_ensloccov4{scl,var,tim}) to tune cross-scale/variable/time covariances in EnVar (NOAA-EMC#539). Regression tests for global 3dvar/4denvar/4dvar are not completed yet, but for other tests, issues are not found except for "failed the scalability test" and "exceeded maximum allowable hardware memory limit" on Orion. Fixes NOAA-EMC#539 Co-authored-by: Sho Yokota <syokota@Orion-login-1.HPC.MsState.Edu>
…C#499) The enhancements made to the EnKF IO interface are the following: (1) For enkf I/O: add the handling of precipitation hydrometeors to parallel netcdf I.O --- preparation of next implementation
… plus changes to make code work for multiscale (bugs introduced in develop). (NOAA-EMC#570)
… conventional data simultaneously without side effects in EnVar (NOAA-EMC#543) This PR removes forcing pure EnVar and adding an option (if_use_w_vr) to assimilate radar reflectivity and conventional data simultaneously without side effects in EnVar (NOAA-EMC#540). Regression tests for global 3dvar/4denvar/4dvar are not completed yet, but for other tests, issues are not found except for "failed the scalability test" and "exceeded maximum allowable hardware memory limit" on Orion. Fixes NOAA-EMC#540 Co-authored-by: Sho Yokota <syokota@Orion-login-1.HPC.MsState.Edu>
Resubmit PR for cleaning the unneeded item in the previous PR In correspondence to the EMC GSI Issue#566, this PR contains a quick adding of the clipping of negative values of sphum (q) in the analysis of FV3-LAM EnKF. This part of codes are not tested in the current GSI regression tests, which, hence, are not run. The current codes are verified using local FV3-LAM case. It is found the differences from this changes exist for sphum ( maximum values about 0.003 (units) and 0.3 K for T (sensible T). The latter is because the sphum would be used when the analysis TV is converted to T. All differences are on spontaneous points and values are reasonable as expected. Hence, the code is regarded as verified. Fixes NOAA-EMC#566 **DUE DATE for this PR is 6/15/2023.** If this PR is not merged into `develop` by this date, the PR will be closed and returned to the developer.
… values instead of zeros (NOAA-EMC#578)
<!-- PLEASE READ --> <!-- Before opening a PR, please note these guidelines: - Each PR should only address ONE topic and have an associated issue - No hardcoded or paths to personal directories should be present - No temporary or backup files should be committed - Any code that was disabled by being commented out should be removed --> **Description** Initialization of the operational RRFSv1 will include assimilation of flash-extent density (FED) observations from the GOES Geostationary Lightning Mapper (GLM). The current PR is the first of at least 3 that will be needed to introduce the capability of FED assimilation into the code and regional workflow. The new capabilities that are added to GSI are: * reading NetCDF FED observations * applying an observation operator that maps the model state to FED. Much of the code was originally developed by Rong Kong at OU-CAPS (Kong et al. 2020, Wang et al. 2021, Kong et al. 2022; https://doi.org/10.1175/MWR-D-19-0192.1, https://doi.org/10.1175/MWR-D-20-0406.1, https://doi.org/10.1175/MWR-D-21-0326.1). Recently, the observation operator has been modified by Amanda Back and Ashley Sebok based on tests with regional, convection-allowing FV3 forecasts. The new observation operator includes a cap of 8 flashes / minute for both the observed and simulated FED. The observation operator is specific to the 3-km regional FV3 application in RRFS. Development of a more general observation operator is left to future work. Fixes NOAA-EMC#588 <!-- Please include relevant motivation and context. --> <!-- Please include a summary of the change and which issue is fixed. --> <!-- List any dependencies that are required for this change. --> <!-- Please provide reference to the issue this pull request is addressing. --> <!-- For e.g. Fixes #IssueNumber --> **Type of change** Please delete options that are not relevant. - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update **How Has This Been Tested?** Initial tests were with NOAA-EMC GSI-EnKF code obtained in April 2023 and modified to include the assimilation of FED observations. A prototype of RRFSv1 was cycled hourly for 2.5 days, and the EnKF assimilation included FED data assimilation. For the current PR, only the GSI observer with FED (and radar reflectivity) observations was tested. It produces identical results to those obtained in April 2023. <!-- Please describe the tests that you ran to verify your changes and on the platforms these tests were conducted. --> <!-- Provide instructions so we can reproduce. --> <!-- Please also list any relevant details for your test configuration --> **Checklist** - [ ] My code follows the style guidelines of this project - [X] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] New and existing tests pass with my changes - [ ] Any dependent changes have been merged and published **DUE DATE for this PR is 8/24/2023.** If this PR is not merged into `develop` by this date, the PR will be closed and returned to the developer. --------- Co-authored-by: Ming Hu <Ming.Hu@noaa.gov>
…#618) (NOAA-EMC#619) **Description** PR NOAA-EMC#591 removed jacobian information from the netcdf ozone diagnostic file. This caused `enkf.x` to crash. This PR adds the removed ozone jacobian arrays back to the netcdf ozone diagnostic file. Fixes NOAA-EMC#618 **Type of change** - [x] Bug fix (non-breaking change which fixes an issue) **How Has This Been Tested?** The revised code was tested in the 20210814 18 gdas cycle of a C192L127 enkf parallel. The updated `gsi.x` created an oznstat file which was successfully processed by `enkf.x`. **Checklist** - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my own code - [x] New and existing tests pass with my changes
* caveat - the wcoss2 build remains at intel/19 --------- Co-authored-by: David Huber <david.huber@noaa.gov> Co-authored-by: Natalie Perlin <Natalie.Perlin@noaa.gov> Co-authored-by: RussTreadon-NOAA <Russ.Treadon@noaa.gov>
**Description** This PR updates the `fix` submodule to bring in GFS v16.3.9 updates to `global_convinfo.txt` and `global_ozinfo.txt`. Issue NOAA-EMC#620 provides additional information on the GFS v16.3.9 updates. Please **note** the following - The change to `global_ozinfo.txt` alters analysis results if ompsnp observation from NOAA-20, NOAA-21, or NPP are processed by gsi.x or `enkf.x`. - The change to `global_convinfo.txt` alters analysis results if PlanetIQ GPSRO (type 267) is processed by `gsi.x` or `enkf.x`. Fixes NOAA-EMC#620 **Type of change** - [x] Breaking change (fix or feature that would cause existing functionality to not work as expected) See the **Note** above for potential impact on analysis results. **How Has This Been Tested?** ctests have been run on Hera, Orion, and WCOSS2 (Cactus) with results posted in issue NOAA-EMC#620. **Checklist** - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my own code
**Description** Bring in HAFSv1 related maxobs changes and the capability of assimilating GOES-R high-resolution AMVs. Resolved NOAA-EMC#599 **Type of change** Use "maxobs" as a condition to check whether the number of observations exceeds the limit, to avoid the out of bound/dimension issue in read_anowbufr.f90 read_dbz_nc.f90 read_gmi.f90 read_goesglm.f90 read_radar.f90 read_radar_wind_ascii.f90 This update also added the capability of assimilating the CIMSS enhanced GOES-R AMVs in a new "satwhr" bufr file. Please delete options that are not relevant. - [x] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) **How Has This Been Tested?** This updates passed the HAFS related regression tests. All tests are performed on Orion. **Checklist** - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] New and existing tests pass with my changes - [x] Any dependent changes have been merged and published **DUE DATE for this PR is 9/21/2023.** If this PR is not merged into develop by this date, the PR will be closed and returned to the developer. --------- Co-authored-by: jswhit <Jeffrey.S.Whitaker@noaa.gov> Co-authored-by: Ting.Lei@noaa.gov <ting.lei@noaa.gov> Co-authored-by: Jili Dong <Jili.Dong@noaa.gov> Co-authored-by: Bin Liu <Bin.Liu@noaa.gov> Co-authored-by: daprediction <leiting2002@gmail.com> Co-authored-by: TingLei-NOAA <Ting.Lei@gmail.com> Co-authored-by: TingLei-NOAA <63461756+TingLei-NOAA@users.noreply.github.com> Co-authored-by: xulu <Xu.Lu@noaa.gov> Co-authored-by: Li Bi <li.bi@noaa.gov> Co-authored-by: edward.safford <edward.safford@noaa.gov> Co-authored-by: MichaelLueken-NOAA <63728921+MichaelLueken-NOAA@users.noreply.github.com> Co-authored-by: RussTreadon-NOAA <russ.treadon@noaa.gov> Co-authored-by: Michael Lueken <michael.lueken@noaa.gov> Co-authored-by: AndrewEichmann-NOAA <Andrew.Eichmann@noaa.gov> Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com> Co-authored-by: Emily Liu <emily.liu@noaa.gov> Co-authored-by: emilyhcliu <36091766+emilyhcliu@users.noreply.github.com> Co-authored-by: Andrew Collard <40322596+ADCollard@users.noreply.github.com> Co-authored-by: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
…EMC#626) The fix involves setting the pressure to 0.001Pa for ozone obs that have zero pressure (to avoid Inf when log(p) calculated), and turning off ob-space vertical localization. Has no effect on current operational setup which uses model-space vertical localization (modelspace_vloc=T). --------- Co-authored-by: jswhit2 <Jeffrey.S.Whitaker@noaa.gov>
NOAA-EMC#594) This update improves the efficiency of the GSI, especially for multiscale runs. Details can be found in issue NOAA-EMC#585
**Description** This PR fixes two types of bugs discovered when cycling `gsi.x` and `enkf.x` with intel/2022 in the global workflow 1. modify variables written to netcdf diagnostic files by `gsi.x` to be consistent with codes which read netcdf diagnostic files 2. modify `lrun_subdirs=.true.` option of `gsi.x` to properly handle the case in which sub-directories already exist in the run directory Fixes NOAA-EMC#623 **Type of change** - [x] Bug fix (non-breaking change which fixes an issue) **How Has This Been Tested?** Ctests have been on Hera, Orion, and WCOSS2 (Cactus) with acceptable behavior. A global parallel covering the period 2021073106 through 2021080118 has been run on Hera, Orion, and WCOSS2 (Cactus). All global workflow jobs ran as expected. **Checklist** - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my own code - [x] New and existing tests pass with my changes
1) Change RRFS case from rrfs_3denvar_glbens to rrfs_3denvar_rdasens. The new RRFS case has the same configuration as RRFS_A warm cycles. 2) delete case netcdf_fv3_regional 3) add RRFS EnKF case for conventional observations: rrfs_enkf_conv
note; while this commit enables gsi.x and enkf.x to be built on Orion Rocky 9,
both executables run 2x slower on Orion Rocky 9 than Orion Centos 7
…into feature/ufo_geovals
…into feature/ufo_geovals
CoryMartin-NOAA
approved these changes
Oct 23, 2024
CoryMartin-NOAA
merged commit e7f6a0c
into
NOAA-EMC:feature/ufo_geovals
2 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes to preserve functionality of feature/ufo_geovals after OS change to Hera. Plus various updates.
How Has This Been Tested?
Checklist