Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recent update broke DOME #19

Closed
kshedstrom opened this issue Dec 1, 2021 · 6 comments · Fixed by #20
Closed

Recent update broke DOME #19

kshedstrom opened this issue Dec 1, 2021 · 6 comments · Fixed by #20

Comments

@kshedstrom
Copy link

DOME runs fine when compiled for debugging, but fails in repro mode:

At line 1285 of file //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM_continuity_PPM.F90
Fortran runtime error: Index '26' of dimension 3 of array 'por_face_areav' above upper bound of 25
chinook01.rcs.alaska.edu 303% which mpif90
/usr/local/pkg/mpi/OpenMPI/1.10.3-GCC-8.3.0-2.32/bin/mpif90

This is in dev/gfdl code du jour.

@kshedstrom
Copy link
Author

Oops, forgot to recompile for debug mode. I'm getting the same thing in Supercritical and some of my other OBC tests. Here's a full backtrace:

Error termination. Backtrace:
#0  0x16c0a9b in zonal_mass_flux
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM_continuity_PPM.F90:469
#1  0x16c5ae3 in __mom_continuity_ppm_MOD_continuity_ppm
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM_continuity_PPM.F90:155
#2  0x6ad011 in __mom_continuity_MOD_continuity
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM_continuity.F90:100
#3  0xe4a939 in __mom_dynamics_split_rk2_MOD_step_mom_dyn_split_rk2
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM_dynamics_split_RK2.F90:573
#4  0x1734a82 in step_mom_dynamics
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM.F90:1093
#5  0x173a836 in __mom_MOD_step_mom
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/src/core/MOM.F90:808
#6  0x13c5039 in mom_main
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/config_src/drivers/solo_driver/MOM_driver.F90:476
#7  0x13c6480 in main
	at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/MOM6/config_src/drivers/solo_driver/MOM_driver.F90:27

@kshedstrom
Copy link
Author

I see that it is a reference to index k, outside of any k do loop.

if (do_I(i)) FAvi(i) = GV%H_subroundoff*(G%dx_Cv(i,J)*por_face_areaV(i,J,k))

@marshallward
Copy link
Member

marshallward commented Dec 1, 2021

Does it work before the DOME patch?

@marshallward
Copy link
Member

Sorry, just saw the line, not related to the DOME patch. It seems to be due to the porous topography PR (#3).

@adcroft
Copy link
Member

adcroft commented Dec 1, 2021

I think this is from the new porous topography code merge in PR #3 . The failing line 469 looks inconsistent with the block L472-L477 immediately below. The lack of k-loop is obvious but the latter block is determining the area from transport/velocity and should already include the porous factor. I'm not sure L69 is needed. @Hallberg-NOAA , @sditkovsky ?

@sditkovsky
Copy link

I agree. I think we can remove the porous factor in L469. In general for the barotropic dynamics, I tried to let the porous modifications be communicated naturally via the transport/velocity. There will be a similar issue with L1285 for the meridional direction.

adcroft added a commit to adcroft/MOM6 that referenced this issue Dec 1, 2021
- An errant use of the porous face area led to an out-of-bounds k-index
  reported in NOAA-GFDL#19.
- Closes NOAA-GFDL#19
@Hallberg-NOAA Hallberg-NOAA linked a pull request Dec 1, 2021 that will close this issue
marshallward pushed a commit that referenced this issue Dec 1, 2021
- An errant use of the porous face area led to an out-of-bounds k-index
  reported in #19.
- Closes #19
marshallward pushed a commit that referenced this issue Jan 26, 2022
* additions for stochastic physics and ePBL perts

* cleanup of code and enhancement of ePBL perts

* Update MOM_diabatic_driver.F90

remove conflict with dev/emc

* Update MOM_diabatic_driver.F90

further resolve conflict

* Update MOM_diabatic_driver.F90

put id_sppt_wts, etc back.

* add stochy_restart writing to mom_cap

* additions for stochy restarts

* clean up debug statements

* clean up code

* fix non stochastic ePBL calculation

* re-write of stochastic code to remove CPP directives

* remove blank link in MOM_diagnostics

* clean up MOM_domains

* make stochastics optional

* correct coupled_driver/ocean_model_MOM.F90 and other cleanup

* clean up of code for MOM6 coding standards

* remove stochastics container

* revert MOM_domains.F90

* clean up of mom_ocean_model_nuopc.F90

* remove PE_here from mom_ocean_model_nuopc.F90

* remove debug statements

* stochastic physics re-write

* move stochastics to external directory

* doxygen cleanup

* add write_stoch_restart_ocn to MOM_stochastics

* add logic to remove incrments from restart if outside IAU window

* revert logic wrt increments

* add comments

* update to gfdl 20210806 (#74)

* remove white space and fix comment

* Update MOM_oda_incupd.F90

remove unused index bounds, and fix sum_h2 loop.

Co-authored-by: pjpegion <Philip.Pegion@noaa.gov>
Co-authored-by: Marshall Ward <marshall.ward@noaa.gov>

* Fussing with zotero.bib.

Getting a warning about a repeated bibliography entry for adcroft2004.
Rob thinks this is a hash failure.

* Still fussing with zotero.bib

- it was complaining about the (unused) Kasahara reference.

* Several little things, one is making sponge less verbose.

- Pointing to OBC wiki file from the lateral parameterizations doc.

- Using the MOM6 verbosity to control the time_interp verbosity.

- Making the check for negative water depths more informative.

* return a more accurate error message in MOM_stochasics

* Working on boundary layer docs.

* Done with EPBL docs?

* Undoing some patches from others

* Cleaning up too-new commits

* Adding in that SAL commit again.

* correction on type in directory name

* Added some to vertical viscisity doc.

* Cleaned up whitespace leftover from porous topomerge.

- Spacing within expressions was uneven and made multiplation look like
  POW functions. Leftover from merging #3.
- No answer changes.

* Fix out-of-bounds k index in PPM flux

- An errant use of the porous face area led to an out-of-bounds k-index
  reported in #19.
- Closes #19

* Adding Channel drag figure

* Take cite out of figure caption.

* Copyright year 2022
marshallward pushed a commit that referenced this issue Jul 2, 2022
* initial hooks for stochastic EOS modifications

* remove debug statements

* add documentation

* Change ampltiude from 0.39 to sqrt(.39)

* remove global_indexing logic from stoch_eos_init

* switch to using MOM_random and add restart capability

* update random sequence to update each each time-step

* remove tseed0 from MOM_random (leftover from debugging)

* Added necessary submodules and S^2, T^2 diagnostics to MOM_diagnostics

* Added diagnostics for outputting variables related to the stochastic parameterization.

* Diagnostics in MOM_PressureForce_FV updated for stochastic (rather than deterministic) Stanley SGS T variance parameterization.

* Added parentheses for reproducibility.

* Changed diagnostics to account for possible absence of stoch_eos_pattern in MOM_PressureForce_FV,
when deterministic parameterization is on.

* remove mom6_da_hooks and geoKdTree from pkg

* add stochastic compoment to MOM_thickness_diffuse

* fix array size declaration and post_data

* Corrected indexing of loops in MOM_calc_varT

* Changed how parameterization of SGS T variance (deterministic and stochastic) is switched on in PGF and thickness diffusion codes

* Corrected a few typos

* Cleaned up indices, redundant diagnostic, printing

* Fixed diagnostic IDs

* Fixed diagnostics typo

* Corrected indices in calculation of tv%varT

* Minor index fix

* Corrected bug in pressure in Stanley diagnostics

* Fixed whitespace error

* Stoch eos clock (#5)

*Added a clock for the Stanley parameterization

Co-authored-by: jkenigson <jkenigso@gmail.com>

* add halo update to random pattern

* Update MOM_stoch_eos.F90

Fix bug for looping over compute domain (is -> isc etc.)

* Avoid unnessary computations on halo (MOM_stoch_eos) and code clean-up (MOM_thickness_diffuse)

* Removed halo updates before determ param calc

* Update MOM_stoch_eos.F90

Removed unnecessary code

* Bug - indices are transposed

* Changed Stanley stochastic coefficient from exp(X) to exp(aX) (#9)

* Changed Stanley stochastic coefficient from exp(X) to exp(aX)

* Extra spaces removed

* Stoch eos init fix (#10)

* Don't bother calculating tv%varT if stanley_coeff<0

* Missing then added

* Merge Ian Grooms Tvar Discretization (#11)

* Update MOM_stoch_eos.F90

In progress updating stencil for$ | dx \times \nabla T|^2$ calculation

* New discretization of |dx\circ\nablaT|^2

Co-authored-by: Ian Grooms <ian.grooms@colorado.edu>

* Multiplied tvar%SGS by grid cell thickness ratio

* Added limiter for tv%varT

* Stoch eos ncar linear disc (#12)

* Update MOM_stoch_eos.F90

In progress updating stencil for$ | dx \times \nabla T|^2$ calculation

* New discretization of |dx\circ\nablaT|^2

* AR1 timescale land mask

Adds land mask to the computation of the AR1 decorrelation time

* Update dt in call to MOM_stoch_eos_run

The call to `MOM_stoch_eos_run` (which time steps the noise) is from within `step_MOM_dynamics`. `step_MOM_dynamics` advances on time step `dt` (per line 957), but the noise is updated using `dt_thermo`. It seems more appropriate to update the noise using `dt`, since it gets called from within `step_MOM_dynamics`.

* Fixed the units for r_sm_H

* Remove vestigial declarations

The variables `hl`, `Tl`, `mn_T`, `mn_T2`, and `r_sm_H` are no longer used, so I removed their declarations and an OMP private clause

Co-authored-by: Ian Grooms <ian.grooms@colorado.edu>

* Update MOM_thickness_diffuse.F90

Changed index for soft convention

* Update CVMix-src

* Ensure use_varT, etc., initialized

* Don't register stanley diagnostics if scheme is off

* Stanley density second derivs at h pts (#15)

* Change discretization of Stanley correction (drho_dT_dT at h points)

* Limit Stanley noise, shrink limiting value

* Revert t variance discretization

* Reverted variable declarations

* Stanley scheme in mixed_layer_restrat, vert_fill in stoch_eos, code cleanup (#19)

* Test Stanley EOS param in mixed_layer_restrat

* Fix size of TS cov, S var in Stanley calculate_density calls

* Test move stanley scheme initialization

* Added missing openMP directives

* Revert Stanley tvar discretization (#18)

* Perform vertical filling in calculation of T variance

* Variable declaration syntax error, remove scaling from get_param

* Fix call to vert_fill_TS

* Code cleanup, whitespace cleanup

Co-authored-by: Jessica Kenigson <jessicak@cheyenne1.cheyenne.ucar.edu>

* Use Stanley (2020) variance; scheme off at coast

* Comment clean-up

* Remove factor of 0.5 in Tvar

* Don't calculate Stanley diagnostics on halo

* Change start indices in stanley_density_1d

* Stanley param in MOM_isopycnal_slopes (#22)

Stanley param in MOM_isopycnal_slopes and thickness diffuse index fix

* Set eady flag to true if use_stored_slopes is true

* Cleanup, docs, whitespace

* Docs and whitespace

* Docs and whitespace

* Docs and whitespace

* Whitespace cleanup

* Whitespace cleanup

* Clean up whitespace

* Docs cleanup

* use_stanley

* Update MOM_lateral_mixing_coeffs.F90

* Adds link to another TEOS10 module

* Set Stanley off for testing

* Line continuation

Co-authored-by: Phil Pegion <38869668+pjpegion@users.noreply.github.com>
Co-authored-by: Philip Pegion <Philip.Pegion@noaa.gov>
Co-authored-by: Jessica Kenigson <jessicak@cheyenne6.cheyenne.ucar.edu>
Co-authored-by: Jessica Kenigson <jessicak@cheyenne3.cheyenne.ucar.edu>
Co-authored-by: jkenigson <jkenigso@gmail.com>
Co-authored-by: jskenigson <jessica.kenigson@colorado.edu>
Co-authored-by: Jessica Kenigson <jessicak@cheyenne1.cheyenne.ucar.edu>
Co-authored-by: Jessica Kenigson <jessicak@cheyenne5.cheyenne.ucar.edu>
Co-authored-by: Philip Pegion <ppegion@Philips-MacBook-Pro.local>
Co-authored-by: Jessica Kenigson <jessicak@cheyenne4.cheyenne.ucar.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants