add cpld_control_wave with 1deg wave grid #385

aliabdolali · 2021-01-19T19:36:08Z

Description

add cpld_control_wave with 1deg wave grid.

No expected changes to the old tests. But the baseline should be created to include new test.

DeniseWorthen · 2021-01-19T19:47:41Z

tests/tests/cpld_control_wave

+export atm_petlist_bounds=$APB_cpl_wwav
+export ocn_petlist_bounds=$OPB_cpl_wwav
+export ice_petlist_bounds=$IPB_cpl_wwav
+export wav_petlist_bounds=$WPB_cpl_wwav


This will set the tasking to the tasking when using waves, which is appropriate for the 1/4 deg model but not for 1deg. You'll need to create a new set of PE tasking for 'cpl_dflt' which is for the 1deg case, but which includes waves, maybe call it 'cpl_dflt_wwav'.

DeniseWorthen · 2021-01-19T19:55:55Z

tests/tests/cpld_control_wave

+
+export CPLWAV='.T.'
+export CPLWAV2ATM='.T.'
+export RT1DEG='.T.'


In default_vars, we set the default resolution for each component using "OCNRES" etc. I would suggest to create a default WW3 res which is 1deg. You wouldn't need the RT1DEG variable because the default tests would be 1 deg ww3, 1deg ocean, 1deg ice and c96 fv3.

The current ww3 tests which are all for 1/4 deg would then need to set a non-default resolution. This would be consistent w/ how we vary the resolution across the different tests.

@DeniseWorthen Thanks for the suggestions. I modified the test but still get a failure.
Here are my changes:

I added cpl_dflt_wwav. Do you have any suggestions for the numbers? I assumed the 1deg is lighter than 1/4, and heavier than dflt, so I set the total to 288. What each of these variables stand for? I mean where can I see the definitions?

I removed export RT1DEG='.T.'

I added WAVRES='1.00'

DeniseWorthen · 2021-01-19T20:45:02Z

tests/default_vars.sh

@@ -22,9 +22,13 @@ elif [[ $MACHINE_ID = wcoss_dell_p3 || $MACHINE_ID = wcoss2 ]]; then
  TASKS_strnest=96 ; TPN_strnest=28 ; INPES_strnest=2 ; JNPES_strnest=4

  TASKS_cpl_dflt=192; TPN_cpl_dflt=28; INPES_cpl_dflt=3; JNPES_cpl_dflt=8
-  THRD_cpl_dflt=1; WPG_cpl_dflt=6;  MPB_cpl_dflt="0 143"; APB_cpl_dflt="0 149"
+  THRD_cpl_dflt=1; WPG_cpl_dflt=6;  ePB_cpl_dflt="0 143"; APB_cpl_dflt="0 149"


You have a typo here and elsewhere: ePB_cpl_dflt.

The WPG is the atm write component tasks, the MPB is the mediator, the APB is the atmosphere, the OPB is the ocean and the IPB is the ice.

DeniseWorthen · 2021-01-19T21:11:31Z

You can add the words "skip-ci" (without the quotes) to a commit message while you're making changes. That will prevent the CI tests from running. Then after all the logs are added, at that point we do the CI tests.

aliabdolali · 2021-01-19T22:37:14Z

You can add the words "skip-ci" (without the quotes) to a commit message while you're making changes. That will prevent the CI tests from running. Then after all the logs are added, at that point we do the CI tests.

Thanks @DeniseWorthen Good idea. BTW, my test fails at the initiation step and I do not know what is the reason.

DeniseWorthen · 2021-01-19T23:23:09Z

Point me to your run directory and I'll take a look.

aliabdolali · 2021-01-20T00:54:29Z

@DeniseWorthen Here is the path
/scratch1/NCEPDEV/stmp2/Ali.Abdolali/FV3_RT/rt_75766/cpld_control_wave_prod
Thanks a million for your kind help.

DeniseWorthen · 2021-01-20T13:05:52Z

How are you compiling and then running this test? The error I see (in err) is an odd one (/scratch1/NCEPDEV/stmp2/Ali.Abdolali/FV3_RT/rt_75766/cpld_control_wave_prod/./fv3.exe': corrupted size vs. prev_size: 0x0000000012dd46b0).

I have your branch checked out here: /scratch2/NCEPDEV/climate/Denise.Worthen/WORK/ufs_1degw3/tests. I've added an rt.test which is a temporary file containing only the compile line for ufs-cpld+waves and this single new test. I am trying that now.

I also noticed that the test is still using some of the the 1/4deg configuration (DT_ATMOS='450') settings. You'll want to remove these since you want to use the default c96mx100 settings (these are set in default_vars). See what I've done in tests/cpld_control_wave.

DeniseWorthen · 2021-01-20T17:58:57Z

I was able to get the test to run (see /scratch1/NCEPDEV/stmp2/Denise.Worthen/FV3_RT/rt_23024/cpld_control_wave_prod) but there were two issues.

One, the test is trying to move the file out_grd.glo_1deg into the baseline but it can't find it. Is this supposed to be the ww3 restart file? There is a file in the run directory named 20161004.000000.restart.glo_1deg. If that is the restart, then it needs to be used as the file name in the test.

Two, I had to adjust the PE tasking for the 1deg wave test. You had created the new TASKS_cpl_dflt_wwav but they were still appropriate for the 1/4 deg. For the 1deg resolution, I gave waves 12 PEs (the 1/4 uses 40). I only updated the hera setting so you'll need do the same for the others. See my changes here: /scratch2/NCEPDEV/climate/Denise.Worthen/WORK/ufs_1degw3/tests

aliabdolali · 2021-01-20T18:05:16Z

I was able to get the test to run (see /scratch1/NCEPDEV/stmp2/Denise.Worthen/FV3_RT/rt_23024/cpld_control_wave_prod) but there were two issues.

One, the test is trying to move the file out_grd.glo_1deg into the baseline but it can't find it. Is this supposed to be the ww3 restart file? There is a file in the run directory named 20161004.000000.restart.glo_1deg. If that is the restart, then it needs to be used as the file name in the test.

Two, I had to adjust the PE tasking for the 1deg wave test. You had created the new TASKS_cpl_dflt_wwav but they were still appropriate for the 1/4 deg. For the 1deg resolution, I gave waves 12 PEs (the 1/4 uses 40). I only updated the hera setting so you'll need do the same for the others. See my changes here: /scratch2/NCEPDEV/climate/Denise.Worthen/WORK/ufs_1degw3/tests

@DeniseWorthen You are awesome. I'll modify the PEs for all the platforms. The out_grd.glo_1deg is the gridded outputs which should be compared with the baseline. I will add 20161004.000000.restart.glo_1deg to the filename to be compared with the baseline.
Once I confirm the test is running at my end, I will ask you to review it.
AA

skip-ci

aliabdolali · 2021-01-20T19:28:00Z

@junwang-noaa @JessicaMeixner-NOAA @DeniseWorthen
With guidance and help from Denise, I prepared the cpld_control_wave test. I need to run the whole rt.conf and create the baseline. Could you tell me when is my turn? so I can run it on Hera and Orion (the machines I have access to).

JessicaMeixner-NOAA · 2021-01-21T16:21:32Z

tests/fv3_conf/cpld_control_run.IN

@@ -50,6 +50,12 @@ cp    @[INPUTDATA_ROOT]/CICE_FIX/@[OCNRES]/grid_cice_NEMS_mx@[OCNRES].nc .
 cp    @[INPUTDATA_ROOT]/CICE_FIX/@[OCNRES]/kmtu_cice_NEMS_mx@[OCNRES].nc .
 cp    @[INPUTDATA_ROOT]/CICE_FIX/@[OCNRES]/mesh.mx@[OCNRES].nc .

+# WW3 fix/input
+if [[ $CPLWAV == .T. && $CPLWAV2ATM == .T. ]]; then


We would need this only if CPLWAV is true, cplwave2atm=f would still need mod_defs and the input file

JessicaMeixner-NOAA · 2021-01-21T16:27:56Z

tests/rt.conf

@@ -128,8 +128,9 @@ RUN     | fv3_ccpp_esg_HAFS_v0_hwrf_thompson_debug
 # CPLD tests                                                                                                                                                                      #
 ###################################################################################################################################################################################

-COMPILE | SUITES=FV3_GFS_2017_coupled,FV3_GFS_2017_satmedmf_coupled,FV3_GFS_v15p2_coupled,FV3_GFS_v16beta_coupled S2S=Y           | - wcoss_cray gaea.intel jet.intel       | fv3 |
+COMPILE | SUITES=FV3_GFS_2017_coupled,FV3_GFS_2017_satmedmf_coupled,FV3_GFS_v15p2_coupled,FV3_GFS_v16beta_coupled S2S=Y WW3=Y     | - wcoss_cray gaea.intel jet.intel       | fv3 |


This makes the compile the same as line 162, which makes me wonder if we should either move the new test w/waves under that compile or just remove the second compile?

@JessicaMeixner-NOAA I fixed them. Thanks.

skip-ci

DeniseWorthen · 2021-01-21T17:08:32Z

tests/default_vars.sh

@@ -25,6 +25,10 @@ elif [[ $MACHINE_ID = wcoss_dell_p3 || $MACHINE_ID = wcoss2 ]]; then
  THRD_cpl_dflt=1; WPG_cpl_dflt=6;  MPB_cpl_dflt="0 143"; APB_cpl_dflt="0 149"
  OPB_cpl_dflt="150 179"; IPB_cpl_dflt="180 191"

+  TASKS_cpl_dflt_wwav=204; TPN_cpl_dflt_wwav=40; INPES_cpl_dflt_wwav=3; JNPES_cpl_dflt_wwav=8


This should be TPN_cpl_dfld_wwav=28 (same as cpl_dflt). This is the nodes/processor which varies across machines.

DeniseWorthen · 2021-01-21T17:08:59Z

tests/default_vars.sh

@@ -167,6 +179,10 @@ elif [[ $MACHINE_ID = cheyenne.* ]]; then
  THRD_cpl_dflt=1; WPG_cpl_dflt=6;  MPB_cpl_dflt="0 143"; APB_cpl_dflt="0 149"
  OPB_cpl_dflt="150 179"; IPB_cpl_dflt="180 191"

+  TASKS_cpl_dflt_wwav=204; TPN_cpl_dflt_wwav=40; INPES_cpl_dflt_wwav=3; JNPES_cpl_dflt_wwav=8


Same here, cheyenne is only 36/node.

DeniseWorthen · 2021-01-21T17:09:14Z

tests/default_vars.sh

@@ -205,6 +221,10 @@ elif [[ $MACHINE_ID = stampede.* ]]; then
  THRD_cpl_dflt=1; WPG_cpl_dflt=6;  MPB_cpl_dflt="0 143"; APB_cpl_dflt="0 149"
  OPB_cpl_dflt="150 179"; IPB_cpl_dflt="180 191"

+  TASKS_cpl_dflt_wwav=204; TPN_cpl_dflt_wwav=40; INPES_cpl_dflt_wwav=3; JNPES_cpl_dflt_wwav=8


and here (48)

@DeniseWorthen Done

skip-ci

DeniseWorthen · 2021-01-27T19:22:08Z

Closed.

add cpld_control)wave with 1deg wave grid

d0c08f6

DeniseWorthen reviewed Jan 19, 2021

View reviewed changes

modify setting

dec774a

DeniseWorthen reviewed Jan 19, 2021

View reviewed changes

Ali added 3 commits January 19, 2021 20:53

fix typo in tests/default_vars.sh

657b941

add WPB

7f29488

edit typo

a56e892

skip-ci

acb5573

finalize PE allocation

8b106a0

skip-ci

JessicaMeixner-NOAA reviewed Jan 21, 2021

View reviewed changes

some fixes

0cfb00a

skip-ci

DeniseWorthen reviewed Jan 21, 2021

View reviewed changes

edit TPN for different machines

39636ca

skip-ci

DeniseWorthen mentioned this pull request Jan 26, 2021

MOM6 bugfixes, GFDL update, update CDMBGWD settings; fix for restart reproducibility (without waves) when USE_LA_LI2016=True, sign error on fprec passed to ocean, GFDL update, resolution dependent cdmbgwd settings #379

Merged

DeniseWorthen closed this Jan 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add cpld_control_wave with 1deg wave grid #385

add cpld_control_wave with 1deg wave grid #385

aliabdolali commented Jan 19, 2021

DeniseWorthen Jan 19, 2021 •

edited

Loading

DeniseWorthen Jan 19, 2021

aliabdolali Jan 19, 2021 •

edited

Loading

DeniseWorthen Jan 19, 2021

DeniseWorthen commented Jan 19, 2021

aliabdolali commented Jan 19, 2021

DeniseWorthen commented Jan 19, 2021

aliabdolali commented Jan 20, 2021

DeniseWorthen commented Jan 20, 2021

DeniseWorthen commented Jan 20, 2021

aliabdolali commented Jan 20, 2021

aliabdolali commented Jan 20, 2021

JessicaMeixner-NOAA Jan 21, 2021

JessicaMeixner-NOAA Jan 21, 2021

aliabdolali Jan 21, 2021

DeniseWorthen Jan 21, 2021

DeniseWorthen Jan 21, 2021

DeniseWorthen Jan 21, 2021 •

edited

Loading

aliabdolali Jan 21, 2021

DeniseWorthen commented Jan 27, 2021

add cpld_control_wave with 1deg wave grid #385

add cpld_control_wave with 1deg wave grid #385

Conversation

aliabdolali commented Jan 19, 2021

Description

DeniseWorthen Jan 19, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aliabdolali Jan 19, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DeniseWorthen commented Jan 19, 2021

aliabdolali commented Jan 19, 2021

DeniseWorthen commented Jan 19, 2021

aliabdolali commented Jan 20, 2021

DeniseWorthen commented Jan 20, 2021

DeniseWorthen commented Jan 20, 2021

aliabdolali commented Jan 20, 2021

aliabdolali commented Jan 20, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DeniseWorthen Jan 21, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DeniseWorthen commented Jan 27, 2021

DeniseWorthen Jan 19, 2021 •

edited

Loading

aliabdolali Jan 19, 2021 •

edited

Loading

DeniseWorthen Jan 21, 2021 •

edited

Loading