Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving empty and non-existent data table handling inside data_override #933

Merged
merged 6 commits into from
Mar 31, 2022

Conversation

GFDL-Eric
Copy link
Contributor

@GFDL-Eric GFDL-Eric commented Mar 9, 2022

Description
Instead of having these checks within the various couplers, we are moving the handling of empty and non-existent data_table and data_table.yaml files within data_override. Note that this involved hardcoding the file_id returned by open_and_parse_file for yaml files in order for that to be acted upon within data_override.

Fixes #934

How Has This Been Tested?
Compiles on skylake. This is mostly runtime changes and they have not been tested.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules
  • New check tests, if applicable, are included
  • make distcheck passes

…a_table.yaml from within data_override, so that any routine can call data_override_init without worry.
…a_table.yaml from within data_override, so that any routine can call data_override_init without worry... with no lines over 120 and NOTE import in data_override.
@bensonr
Copy link
Contributor

bensonr commented Mar 10, 2022

I've tested the if-test for diag_table existence in two FV3 solo tests (no grid_spec and an empty data_table) and it works as expected.

Copy link
Contributor

@rem1776 rem1776 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this needs a merge with main to get rid of the tap-driver.sh changes

@GFDL-Eric
Copy link
Contributor Author

I'm pretty sure this is already based on the 2022.01-beta2 tag, which is the latest commit? Unless another PR made a change?

@GFDL-Eric
Copy link
Contributor Author

Now I get it... I rolled back the tap-driver.sh stuff to what's currently in main.

@rem1776
Copy link
Contributor

rem1776 commented Mar 24, 2022

@GFDL-Eric That file used to be auto-generated by autoreconf until #800 added it to the repo with some changes, so that probably messed up the merge. I see you fixed it now though

rem1776
rem1776 previously approved these changes Mar 24, 2022
@@ -367,7 +371,14 @@ subroutine read_table(data_table)

! Read coupler_table
open(newunit=iunit, file='data_table', action='READ', iostat=io_status)
if(io_status/=0) call mpp_error(FATAL, 'data_override_mod: Error in opening file data_table')
if(io_status/=0) then
if(io_status==29) then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this compiler or system dependent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh, you're right, I thought I was being clever here and using a "standard". I guess I can just use INQUIRE instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't feel bad, i ALWAYS write write (6,*)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've now pushed a change that should also obviate the need to go through the reading table portion if the file doesn't exist... let me know if anything looks like it won't work with that.

Copy link
Contributor

@bensonr bensonr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the early exit for a non-existent data_table, are things initialized the same as if one had an empty data_table? I ask to make sure that future data_override calls will be handled in the same fashion.

@GFDL-Eric
Copy link
Contributor Author

@bensonr, the only practical implications of exiting the read_table subroutine at that point (as opposed to going through with an empty table) are that the ntable, ntable_lima, and ntable_new values are not set to 0. However, those values are not used outside of the read_table subroutine. The line that sets table_size equal to 0 in read_table is sufficient for triggering the "return" in data_override_init for an empty data table or non-existent file.

@bensonr
Copy link
Contributor

bensonr commented Mar 31, 2022

@GFDL-Eric - Does that mean subsequent calls to data_override will also return?

@GFDL-Eric
Copy link
Contributor Author

not as it's currently written... that's gonna be a problem!

@GFDL-Eric
Copy link
Contributor Author

hmm, actually, subsequent data_override_init calls should return because table_size should already be an established module variable after the first data_override_init call. As for calls to data_override itself with an empty table, assuming a do i=1,0 just gets skipped, those subroutines should also return. Also, nobody should be calling data_override (2D or 3D) with an empty data table in the first place.

@bensonr
Copy link
Contributor

bensonr commented Mar 31, 2022

@GFDL-Eric - I'm clear on the behavior of data_override_init. It's the behavior of data_override called directly without any conditional or trigger around it. I've looked through an ESM codebase and only see unprotected instances in FMSCoupler and MOM6 (coupler functionality that seems to have been moved within) that could be a problem if an absent data_table isn't handled the same way as an empty data_table. With the exception of requiring a grid_spec.

@GFDL-Eric
Copy link
Contributor Author

@bensonr For the data_override calls (both 2D and 3D), the first section in both subroutines attempts to find the index within the table. Here are the lines from 2D:

  if(PRESENT(override)) override = .false.
  index1 = -1
  do i = 1, table_size
     if( trim(gridname) /= trim(data_table(i)%gridname)) cycle
     if( trim(fieldname) /= trim(data_table(i)%fieldname_code)) cycle
     index1 = i                               ! field found
     exit
  enddo
  if(index1 .eq. -1) return  ! NO override was performed

data_table values are set to the default values on the first run of data_override_init, regardless of empty or non-existent data_table. Similarly, table_size is set to 0 on that first init. This should result in index1 remaining at -1 and the return being triggered on the last line, provided data_override_init has been called once (module_is_initialized = .true.). If it hasn't, it appears that 2D will fail, but 3D will not, because 3D has:

if(.not.module_is_initialized) &
       call mpp_error(FATAL,'Error: need to call data_override_init first')

We can add this protection to 2D to be safe, although it would only be applicable in cases where data_override_init was never called. Should we just go ahead and add it?

@bensonr
Copy link
Contributor

bensonr commented Mar 31, 2022

@GFDL-Eric - sounds like we are good as is. I'll approve it here shortly.

@rem1776 rem1776 merged commit bf18b24 into NOAA-GFDL:main Mar 31, 2022
MinsukJi-NOAA added a commit to NOAA-EMC/FMS that referenced this pull request Apr 15, 2022
* get_unit warning now only prints on root_pe (NOAA-GFDL#872)

Co-authored-by: Eric Stofferahn <Eric.Stofferahn@noaa.gov>

* Update changelog, version numbers and build info (NOAA-GFDL#873)

Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>

* chore: Change version number to next development version (NOAA-GFDL#874)

* CI: Update build action for yaml parser (NOAA-GFDL#871)

Update build actions
Change images to organization's repo and fix mixed mode parser test failure

* test(parser): Change real comparison value to double (NOAA-GFDL#886)

Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>

* Make 'unsupported kind' error messages more descriptive. Change author of constants4.F90

* Add doxygen comment to valid_types in sat_vapor_pres/sat_vapor_pres_k.F90

* Modify codes for r8-r4 conversion to remove compiler warnings

* Remove OVERLOAD_R8 directives regarding send_data_*d_r8 subroutines

* Add Doxygen comments to constants4.F90

* Updates AM4 regression test suite to run intel 21 only on PW (NOAA-GFDL#893)

* feat: emc changes for mixedmode (NOAA-GFDL#857) (NOAA-GFDL#898)

* Change send_data interface and any necessary routines for mixed real precision support
* Adds an r4 size constants file

BREAKING CHANGE: changes some argument types in diag_manager, tracer_manager, sat_vapor_pres, and time_manager from real to class(*)

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* feat: adds option to override (ignore) checksum test when reading restarts (NOAA-GFDL#889)

* update to fms2_io to allow reads to ignore embedded checksums on a fileobj basis
* added fms2_io test cases for the ignore_checksum option to restart reads
tests the domain, domain_wrap, and bc restart options

* docs: add CI information file (NOAA-GFDL#899)

* test: Adds check_nml_error after reading a namelist, so it could crash only as expected (NOAA-GFDL#904)

* Revert "feat: emc changes for mixedmode (NOAA-GFDL#857) (NOAA-GFDL#898)" (NOAA-GFDL#914)

This reverts commit 516a5ef.

* fix: clean up unused/uninitialized variables and other warnings (NOAA-GFDL#859)

Remove unused variables throughout and changes/removals for other warnings such as uninitialized values and implicit casts

* docs: update function style and branch names

* Changes master to main in CONTRIBUTING.md

* Claifies function return documentation in CODE_STYLE.md

* feat: adds support for logical variables in the yaml parser (NOAA-GFDL#907)

* Adds support for logical variables in the yaml parser

* correctly determines if a string is true or false

* fix: add check for affinity to self-initialize and update test program (NOAA-GFDL#917)

* fix: adds casts to class(*) calls to match doubles in C routines (NOAA-GFDL#920)

Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>

* build: add intel code coverage build option to autotools (NOAA-GFDL#895)

* refactor: change to inclusive variable names (NOAA-GFDL#926)

* feat: Allow maximum number of restart variables to be set at build time (NOAA-GFDL#909)

* fix: Fixes for linter action and code style (NOAA-GFDL#869)

* test: Test script updates and input tests (NOAA-GFDL#800)

Rewrites test script to use added testing shell functions
Adds previously skipped or removed unit tests and support for testing with input files

Co-authored-by: Seth Underwood <Seth.Underwood@noaa.gov>
Co-authored-by: Uriel Ramirez <uriel.ramirez@noaa.gov>

* chore: 2022.01 release changes (NOAA-GFDL#941)

* chore: change version number to next development version (NOAA-GFDL#945)

* Add option for position independent code (NOAA-GFDL#930)

* fix: document and change parameter names for grid versions (NOAA-GFDL#918)

* fix: Moving empty and non-existent data table handling inside data_override (NOAA-GFDL#933)

Enables the processing of an empty or non-existent data_table or data_table.yaml from within data_override, so that any routine can call data_override_init without worry

* fix: Removal of internal FMS use of fms_io and mpp_io (NOAA-GFDL#928)

* fix: mpp changes to solve compile issues with serial builds (NOAA-GFDL#949)

* feat: adds build option for compiling with different sets of constants (NOAA-GFDL#929)

* feat: add and aggregate mpp_chksum unit tests (NOAA-GFDL#946)

* feat: add module for string utility routines (NOAA-GFDL#911)

Adds module and accompanying test for common string operations

* revert: reinstated the mpp_io routines and put them at the end for future elimination. (NOAA-GFDL#952)

* fix: add back in small_fac parameter to constants .h files (NOAA-GFDL#954)

* fix: root pe bug with fms2_io::flush_file (NOAA-GFDL#958)

* fix: clean up string routines for fms_string_utils_mod (NOAA-GFDL#953)

* Make changes to lines with more than 120 columns

Co-authored-by: Ryan Mulhall <35538242+rem1776@users.noreply.github.com>
Co-authored-by: Eric Stofferahn <Eric.Stofferahn@noaa.gov>
Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>
Co-authored-by: Tom Robinson <33458882+thomas-robinson@users.noreply.github.com>
Co-authored-by: Rusty Benson <6594772+bensonr@users.noreply.github.com>
Co-authored-by: uramirez8707 <49168881+uramirez8707@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@outlook.com>
Co-authored-by: Seth Underwood <Seth.Underwood@noaa.gov>
Co-authored-by: Uriel Ramirez <uriel.ramirez@noaa.gov>
Co-authored-by: Matthew Thompson <matthew.thompson@nasa.gov>
Co-authored-by: Eric <7784797+GFDL-Eric@users.noreply.github.com>
MinsukJi-NOAA added a commit to NOAA-EMC/FMS that referenced this pull request Apr 15, 2022
* get_unit warning now only prints on root_pe (NOAA-GFDL#872)

Co-authored-by: Eric Stofferahn <Eric.Stofferahn@noaa.gov>

* Update changelog, version numbers and build info (NOAA-GFDL#873)

Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>

* chore: Change version number to next development version (NOAA-GFDL#874)

* CI: Update build action for yaml parser (NOAA-GFDL#871)

Update build actions
Change images to organization's repo and fix mixed mode parser test failure

* test(parser): Change real comparison value to double (NOAA-GFDL#886)

Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>

* Make 'unsupported kind' error messages more descriptive. Change author of constants4.F90

* Add doxygen comment to valid_types in sat_vapor_pres/sat_vapor_pres_k.F90

* Modify codes for r8-r4 conversion to remove compiler warnings

* Remove OVERLOAD_R8 directives regarding send_data_*d_r8 subroutines

* Add Doxygen comments to constants4.F90

* Updates AM4 regression test suite to run intel 21 only on PW (NOAA-GFDL#893)

* feat: emc changes for mixedmode (NOAA-GFDL#857) (NOAA-GFDL#898)

* Change send_data interface and any necessary routines for mixed real precision support
* Adds an r4 size constants file

BREAKING CHANGE: changes some argument types in diag_manager, tracer_manager, sat_vapor_pres, and time_manager from real to class(*)

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* feat: adds option to override (ignore) checksum test when reading restarts (NOAA-GFDL#889)

* update to fms2_io to allow reads to ignore embedded checksums on a fileobj basis
* added fms2_io test cases for the ignore_checksum option to restart reads
tests the domain, domain_wrap, and bc restart options

* docs: add CI information file (NOAA-GFDL#899)

* test: Adds check_nml_error after reading a namelist, so it could crash only as expected (NOAA-GFDL#904)

* Revert "feat: emc changes for mixedmode (NOAA-GFDL#857) (NOAA-GFDL#898)" (NOAA-GFDL#914)

This reverts commit 516a5ef.

* fix: clean up unused/uninitialized variables and other warnings (NOAA-GFDL#859)

Remove unused variables throughout and changes/removals for other warnings such as uninitialized values and implicit casts

* docs: update function style and branch names

* Changes master to main in CONTRIBUTING.md

* Claifies function return documentation in CODE_STYLE.md

* feat: adds support for logical variables in the yaml parser (NOAA-GFDL#907)

* Adds support for logical variables in the yaml parser

* correctly determines if a string is true or false

* fix: add check for affinity to self-initialize and update test program (NOAA-GFDL#917)

* fix: adds casts to class(*) calls to match doubles in C routines (NOAA-GFDL#920)

Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>

* build: add intel code coverage build option to autotools (NOAA-GFDL#895)

* refactor: change to inclusive variable names (NOAA-GFDL#926)

* feat: Allow maximum number of restart variables to be set at build time (NOAA-GFDL#909)

* fix: Fixes for linter action and code style (NOAA-GFDL#869)

* test: Test script updates and input tests (NOAA-GFDL#800)

Rewrites test script to use added testing shell functions
Adds previously skipped or removed unit tests and support for testing with input files

Co-authored-by: Seth Underwood <Seth.Underwood@noaa.gov>
Co-authored-by: Uriel Ramirez <uriel.ramirez@noaa.gov>

* chore: 2022.01 release changes (NOAA-GFDL#941)

* chore: change version number to next development version (NOAA-GFDL#945)

* Add option for position independent code (NOAA-GFDL#930)

* fix: document and change parameter names for grid versions (NOAA-GFDL#918)

* fix: Moving empty and non-existent data table handling inside data_override (NOAA-GFDL#933)

Enables the processing of an empty or non-existent data_table or data_table.yaml from within data_override, so that any routine can call data_override_init without worry

* fix: Removal of internal FMS use of fms_io and mpp_io (NOAA-GFDL#928)

* fix: mpp changes to solve compile issues with serial builds (NOAA-GFDL#949)

* feat: adds build option for compiling with different sets of constants (NOAA-GFDL#929)

* feat: add and aggregate mpp_chksum unit tests (NOAA-GFDL#946)

* feat: add module for string utility routines (NOAA-GFDL#911)

Adds module and accompanying test for common string operations

* revert: reinstated the mpp_io routines and put them at the end for future elimination. (NOAA-GFDL#952)

* fix: add back in small_fac parameter to constants .h files (NOAA-GFDL#954)

* fix: root pe bug with fms2_io::flush_file (NOAA-GFDL#958)

* fix: clean up string routines for fms_string_utils_mod (NOAA-GFDL#953)

* chore: update libFMS module with new routines (NOAA-GFDL#912)

* fix: removes fms_c.c and fms_c.h (NOAA-GFDL#961)

* Make changes to lines with more than 120 columns

Co-authored-by: Ryan Mulhall <35538242+rem1776@users.noreply.github.com>
Co-authored-by: Eric Stofferahn <Eric.Stofferahn@noaa.gov>
Co-authored-by: rem1776 <Ryan.Mulhall@lscamd50-d.gfdl.noaa.gov>
Co-authored-by: Tom Robinson <33458882+thomas-robinson@users.noreply.github.com>
Co-authored-by: Rusty Benson <6594772+bensonr@users.noreply.github.com>
Co-authored-by: uramirez8707 <49168881+uramirez8707@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@outlook.com>
Co-authored-by: Seth Underwood <Seth.Underwood@noaa.gov>
Co-authored-by: Uriel Ramirez <uriel.ramirez@noaa.gov>
Co-authored-by: Matthew Thompson <matthew.thompson@nasa.gov>
Co-authored-by: Eric <7784797+GFDL-Eric@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Approved
Development

Successfully merging this pull request may close these issues.

Move empty and non-existent data_table handling to data_override
4 participants