Loop to kmt instead of km?

[mnlevy1981]

Many loops inside MARBL run from 1:km, meaning computation is often being done in levels below the sea floor. We should look into having loops run from 1:kmt instead (probably requires stricter initialization of arrays to avoid undefined values below the sea floor).

[mnlevy1981]

@klindsay28 ran into a floating point exception that was not being trapped consistently, and traced it back to sub-seafloor values of POC%remin(:) (and possibly also POC%sflux_out(:) and POC%hflux_out(:)). It turns out that capturing these exceptions every time can be done with the Intel compiler by changing the CESM DEBUG flags from -check pointers -fpe0 to -check pointers -init=snan,arrays -fpe0 (there are other compiler flags as well, but adding -init=snan,arrays is the important part). Unfortunately, running a POP test in this manner traps an error in grid.F90::calc_tpoints and it seems like working through all the POP code could potentially lead down a rabbit hole that isn't worth exploring.

Given the state of the CESM tests, it would be useful to test the MARBL stand-alone driver with these flags and fix any issues detected. This will require #156 to be completed, which we want done before addressing the kmt / km issue anyway.

[mnlevy1981]

In putting together a stand-alone driver test for surface_flux_compute() and interior_tendency_compute(), I noticed that the test was not providing reproducible results because diagnostics below kmt were being populated with uninitialized data. Using Fe_scavenge_rate as an example, though the issue cropped up in several different diagnostics: running with 55 levels, I would get

Fe_scavenge_rate =
  0.0386787107731557, 0.12802456639366, 0.216246733376949, 0.3020111843886,
    0.382519642871635, 0.450071131143231, 0.500563620357867,
    0.524511903869358, 0.53147992608976, 0.534566565393883,
    0.536120854175715, 0.536877312374914, 0.537146645219644,
    0.537122451961349, 0.536937588018157, 0.536678678981546,
    0.536415178243179, 0.53614672546118, 0.53586884163929, 0.535578196293952,
    0.535271607944892, 0.53494577154833, 0.534597097588535,
    0.534221581096575, 0.533814671034334, 0.533371131185432,
    0.532884880914057, 0.532348810294138, 0.531754560104855,
    0.531092268753384, 0.530350273026331, 0.52951476913811,
    0.528569436550718, 0.527495044975825, 0.52626908568835,
    0.524865494644331, 0.52325459532982, 0.521403446201045,
    0.519276852460151, 0.516839354131167, 0.514058469801271,
    0.510909257181649, 0.507379772185044, 0.50347628552068,
    0.499226445175204, 0.494678580427071, 0.489896436681847,
    0.484950573608275, 0.479909187952373, 0.474831135104922,
    0.469762512986904, 0.464736496707024, 0.459775190911553,
    0.454892230889046, 0.450095285158297, 0.445388078893254, 0, 0, 0, 0 ;

with the last 5 values occasionally changing (e.g. 0.445388078893254, NaN, NaN, NaN, NaN or 0.445388078893254, 4822.81633029173, 4892.11429149759, 4938.74044497848, 4972.06479598114) because the actual computed values are just

Fe_scavenge_rate =
  0.0386787107731557, 0.12802456639366, 0.216246733376949, 0.3020111843886,
    0.382519642871635, 0.450071131143231, 0.500563620357867,
    0.524511903869358, 0.53147992608976, 0.534566565393883,
    0.536120854175715, 0.536877312374914, 0.537146645219644,
    0.537122451961349, 0.536937588018157, 0.536678678981546,
    0.536415178243179, 0.53614672546118, 0.53586884163929, 0.535578196293952,
    0.535271607944892, 0.53494577154833, 0.534597097588535,
    0.534221581096575, 0.533814671034334, 0.533371131185432,
    0.532884880914057, 0.532348810294138, 0.531754560104855,
    0.531092268753384, 0.530350273026331, 0.52951476913811,
    0.528569436550718, 0.527495044975825, 0.52626908568835,
    0.524865494644331, 0.52325459532982, 0.521403446201045,
    0.519276852460151, 0.516839354131167, 0.514058469801271,
    0.510909257181649, 0.507379772185044, 0.50347628552068,
    0.499226445175204, 0.494678580427071, 0.489896436681847,
    0.484950573608275, 0.479909187952373, 0.474831135104922,
    0.469762512986904, 0.464736496707024, 0.459775190911553,
    0.454892230889046, 0.450095285158297, _, _, _, _, _, ;

For now I'll have the test only write values down to kmt and when this issue is addressed I'll be able to test it by writing the full column again.