history write error on TOT_WOODPROC with CLMFATES gnu

[rgknox]

Brief summary of bug

I'm generating an error during my first history write, on the variable TOT_WOODPRODC. This is a 1x1_brazil, cold start, FATES enabled simulation with the gnu compiler on ubuntu linux.

The specific error is a SIGFPE: floating point exception. from:

#3 0x5622a3091a44 in __ncdio_pio_MOD_ncd_io_1d_double
at /home/rgknox/Models/CTSM/src/main/ncdio_pio.F90.in:1813
#4 0x5622a316c2cb in hfields_write
at /home/rgknox/Models/CTSM/src/main/histFileMod.F90:3625

I've traced this through and worked some things out. First, if I force the data field to zero at the time of writing (inside the ncd_io_1d routine), the error does not trigger.

The field at the time of write 2.6725596077885196E+042. This is not only related to the initialization of TOT_WOODPRODC, it's components "prod10_grc" and "prod100_grc" are problematic. In CNProductsMod.F90:ComputeProductSummaryVars(), we build prod10_grc and prod100_grc from other diagnostics.

this%prod10_grc(g) is a results of this%dwt_prod10_gain_grc(g), dt, this%gru_prod10_gain_grc(g), this%hrv_deadstem_to_prod10_grc(g) and this%prod10_loss_grc(g)

this%dwt_prod10_gain_grc(g) 0.0000000000000000
dt = 1800.0000000000000
this%gru_prod10_gain_grc(g) = 1.0000000000000000E+036
this%hrv_deadstem_to_prod10_grc(g) = 0.0000000000000000
this%prod10_loss_grc(g) = 1.9099846540774809E+034

So, I'm following these variables to where they are set. Its interesting how one of them is zero'd, one of them is set invalid, and one of them is uninitialized. I did recently work on this code, so I'm self assigning and will try to resolve this soon.

Also, the same error happens for the nitrogen variant of these data structures.

General bug information

CTSM version you are using: [output of git describe]

CTSM: ctsm5.1.dev139-1-g1e2e2c35d
FATES: sci.1.67.1_api.27.0.0

Does this bug cause significantly incorrect results in the model's science? No. Its more of a break the model and prevent a run type of thing, than a bias your results type of thing.

Configurations affected:

CLMFATES, gnu, DEBUG ONLY

Details of bug

see above.

Important details of your setup / configuration so we can reproduce the bug

see above, I used the 1x1_brazil to trigger the error, but it shouldn't be location specific, I expect any and all grids should trigger

Important output or errors that show the problem

see above