Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling carbon isotopes changes answers for transient cases #675

Closed
ekluzek opened this issue Apr 5, 2019 · 15 comments · Fixed by #694
Closed

Enabling carbon isotopes changes answers for transient cases #675

ekluzek opened this issue Apr 5, 2019 · 15 comments · Fixed by #694
Assignees
Labels
investigation Needs to be verified and more investigation into what's going on. priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations
Milestone

Comments

@ekluzek
Copy link
Collaborator

ekluzek commented Apr 5, 2019

Brief summary of bug

Turning on C13 or C14 shouldn't change answers, and yet it does in certain cases.

General bug information

CTSM version you are using: release-clm5.0.20

Does this bug cause significantly incorrect results in the model's science? Maybe?

Configurations affected: C13 or C14 turned on

Details of bug

Adding use_c13=T or use_c14=T, to the SSPRCP126 compset

Important details of your setup / configuration so we can reproduce the bug

Saw this at first in the BSSP126 compset versus BSSP126cmip6. Can also see it in I compset cases.

My test case is: SMS_Ln20.f09_g17_gl4.ISSP126Clm50BgcCrop.cheyenne_intel.clm-crop

Where I've added the following:

./xmlchange GET_REFCASE=TRUE,RUN_REFCASE="b.e21.BHIST.f09_g17.CMIP6-historical.010_v2"
./xmlchange RUN_REFDATE=2015-01-01,RUN_STARTDATE=2015-01-01
./xmlchange CLM_NAMELIST_OPTS="use_init_interp=.true.  init_interp_method='use_finidat_areas'"
./xmlchange RUN_TYPE=hybrid

Set the hist_nhtfrq=1 and use_c13 = T

I know this is also a problem with use_c14=T. And the c13/c14 timeseries file might effect answers as well.

Important output or errors that show the problem

All fields are identical at step 00000, but at 01800 they start to diverge:

 grep RMS
 RMS ACTUAL_IMMOB                     1.9579E-11            NORMALIZED  1.1151E-04
 RMS AGNPP                            4.8354E-13            NORMALIZED  1.9943E-07
 RMS AR                               9.1612E-12            NORMALIZED  1.4726E-06
 RMS BGNPP                            3.7259E-14            NORMALIZED  1.6998E-08
 RMS CH4PROD                          6.7875E-12            NORMALIZED  1.1826E-04
 RMS CH4_SURF_AERE_SAT                4.3764E-13            NORMALIZED  1.3365E-05
 RMS CH4_SURF_AERE_UNSAT              6.5525E-14            NORMALIZED  3.6636E-05
 RMS CH4_SURF_DIFF_SAT                1.4474E-15            NORMALIZED  4.7384E-06
 RMS CH4_SURF_DIFF_UNSAT              1.5199E-16            NORMALIZED  5.5104E-07
 RMS COL_FIRE_CLOSS                   2.5753E-11            NORMALIZED  3.3462E-04
 RMS COL_FIRE_NLOSS                   2.5877E-13            NORMALIZED  1.7228E-04
 RMS CONC_O2_SAT                      5.3083E-08            NORMALIZED  8.9322E-07
 RMS CONC_O2_UNSAT                    3.1502E-08            NORMALIZED  1.8251E-08
 RMS COST_NACTIVE                     2.5476E-07            NORMALIZED  2.0829E-06
 RMS COST_NFIX                        1.0602E-17            NORMALIZED  1.7159E-15
 RMS CWDC                             2.4920E-02            NORMALIZED  4.2131E-05
 RMS CWDC_LOSS                        1.2255E-10            NORMALIZED  1.2563E-04
 RMS CWDC_vr                          3.4802E-02            NORMALIZED  4.4043E-05
 RMS CWDN                             1.1650E-11            NORMALIZED  5.0512E-12
 RMS CWDN_vr                          3.7327E-11            NORMALIZED  1.5132E-11
 RMS DEADCROOTC                       2.4056E-09            NORMALIZED  4.7772E-12
 RMS DEADCROOTN                       1.1872E-11            NORMALIZED  6.2222E-12
 RMS DEADSTEMC                        1.8613E-08            NORMALIZED  1.1603E-11
 RMS DEADSTEMN                        8.3583E-11            NORMALIZED  1.3778E-11
 RMS DENIT                            3.2733E-13            NORMALIZED  2.1733E-05
 RMS DISPVEGC                         2.4458E-08            NORMALIZED  1.1000E-11
 RMS DISPVEGN                         2.0998E-10            NORMALIZED  1.9248E-11
 RMS ER                               1.8383E-10            NORMALIZED  1.7014E-05
 RMS FAREA_BURNED                     4.2503E-14            NORMALIZED  1.4367E-04
 RMS FCH4                             7.2559E-16            NORMALIZED  3.0521E-05
 RMS FCH4TOCO2                        1.8423E-12            NORMALIZED  3.9622E-05
 RMS FCO2                             6.8548E-13            NORMALIZED  1.9570E-05
 RMS FPI                              1.2983E-06            NORMALIZED  1.3182E-06
 RMS FROOTC                           7.1096E-10            NORMALIZED  1.1243E-11
 RMS FROOTC_ALLOC                     3.7259E-14            NORMALIZED  2.0016E-08
 RMS FROOTC_LOSS                      3.9406E-13            NORMALIZED  4.3135E-09
 RMS FROOTN                           1.5078E-11            NORMALIZED  1.0160E-11
 RMS FUELC                            4.5855E-02            NORMALIZED  1.7013E-05
 RMS F_DENIT                          3.2733E-13            NORMALIZED  2.1733E-05
 RMS F_N2O_DENIT                      1.8623E-14            NORMALIZED  2.2776E-05
 RMS F_N2O_NIT                        5.1416E-19            NORMALIZED  3.3140E-08
 RMS F_NIT                            8.5693E-16            NORMALIZED  3.3140E-08
 RMS GR                               1.0077E-12            NORMALIZED  7.4924E-06
 RMS GRAINC                           7.8283E-10            NORMALIZED  1.1638E-09
 RMS GRAINN                           2.3373E-23            NORMALIZED  2.2658E-21
 RMS GROSS_NMIN                       2.1269E-14            NORMALIZED  8.3274E-08
 RMS HR                               1.8226E-10            NORMALIZED  3.9768E-05
 RMS HR_vr                            2.7296E-10            NORMALIZED  5.3141E-05
 RMS LEAFC                            2.7037E-09            NORMALIZED  6.4148E-11
 RMS LEAFC_ALLOC                      5.1420E-15            NORMALIZED  4.1487E-09
 RMS LEAFC_LOSS                       1.5023E-12            NORMALIZED  2.5368E-08
 RMS LEAFN                            9.0910E-11            NORMALIZED  8.2172E-11
 RMS LITFALL                          5.9088E-12            NORMALIZED  6.4753E-09
 RMS LITR1C                           7.5878E-04            NORMALIZED  4.0087E-05
 RMS LITR1C_vr                        1.1524E-03            NORMALIZED  4.7443E-05
 RMS LITR1N                           6.6841E-10            NORMALIZED  1.5085E-09
 RMS LITR1N_vr                        1.9552E-09            NORMALIZED  4.3164E-09
 RMS LITR2C                           1.5181E-03            NORMALIZED  1.1978E-05
 RMS LITR2C_vr                        2.3055E-03            NORMALIZED  1.4230E-05
 RMS LITR2N                           7.2931E-10            NORMALIZED  3.2359E-10
 RMS LITR2N_vr                        2.3147E-09            NORMALIZED  1.0558E-09
 RMS LITR3C                           7.5904E-04            NORMALIZED  1.3329E-05
 RMS LITR3C_vr                        1.1527E-03            NORMALIZED  1.6067E-05
 RMS LITR3N                           3.6127E-10            NORMALIZED  3.2781E-10
 RMS LITR3N_vr                        1.1417E-09            NORMALIZED  1.0694E-09
 RMS LITTERC_HR                       1.8226E-10            NORMALIZED  7.5884E-05
 RMS LITTERC_LOSS                     3.4562E-10            NORMALIZED  7.3307E-05
 RMS LIVECROOTC                       3.0458E-11            NORMALIZED  1.5922E-11
 RMS LIVECROOTN                       1.0166E-12            NORMALIZED  1.3749E-11
 RMS LIVESTEMC                        4.2963E-10            NORMALIZED  5.4985E-11
 RMS LIVESTEMN                        1.3357E-11            NORMALIZED  5.1345E-11
 RMS NACTIVE                          3.7154E-13            NORMALIZED  1.9364E-05
 RMS NACTIVE_NH4                      4.3657E-15            NORMALIZED  1.0432E-05
 RMS NACTIVE_NO3                      1.5999E-16            NORMALIZED  3.8186E-07
 RMS NAM                              3.5584E-15            NORMALIZED  4.5328E-06
 RMS NAM_NH4                          3.5944E-15            NORMALIZED  9.7271E-06
 RMS NAM_NO3                          1.5892E-16            NORMALIZED  3.8247E-07
 RMS NBP                              1.8695E-10            NORMALIZED  1.9570E-05
 RMS NDEPLOY                          3.4379E-13            NORMALIZED  1.0551E-05
 RMS NECM                             1.8348E-15            NORMALIZED  3.4987E-05
 RMS NECM_NH4                         1.8383E-15            NORMALIZED  3.7531E-05
 RMS NECM_NO3                         1.5804E-17            NORMALIZED  4.5646E-06
 RMS NEE                              1.8695E-10            NORMALIZED  2.0138E-05
 RMS NEM                              5.8137E-12            NORMALIZED  1.6923E-04
 RMS NEP                              1.8383E-10            NORMALIZED  1.9857E-05
 RMS NET_NMIN                         1.9583E-11            NORMALIZED  2.4179E-04
 RMS NFIRE                            9.4753E-15            NORMALIZED  9.9733E-05
 RMS NFIX                             3.1157E-14            NORMALIZED  1.4466E-05
 RMS NNONMYC                          3.6741E-13            NORMALIZED  2.0023E-05
 RMS NNONMYC_NH4                      3.6900E-13            NORMALIZED  2.2170E-05
 RMS NNONMYC_NO3                      3.2160E-15            NORMALIZED  1.8858E-06
 RMS NPOOL                            6.1882E-10            NORMALIZED  4.5652E-09
 RMS NPP                              9.1612E-12            NORMALIZED  1.3893E-06
 RMS NPP_GROWTH                       9.1612E-12            NORMALIZED  7.4924E-06
 RMS NPP_NACTIVE                      5.5012E-12            NORMALIZED  7.2711E-06
 RMS NPP_NACTIVE_NH4                  4.4209E-13            NORMALIZED  6.7338E-06
 RMS NPP_NACTIVE_NO3                  1.8696E-13            NORMALIZED  1.9377E-05
 RMS NPP_NFIX                         2.7712E-13            NORMALIZED  1.3403E-05
 RMS NPP_NNONMYC                      4.9367E-12            NORMALIZED  7.2460E-06
 RMS NPP_NNONMYC_NH4                  3.9491E-12            NORMALIZED  6.4514E-06
 RMS NPP_NNONMYC_NO3                  1.5371E-12            NORMALIZED  2.2225E-05
 RMS NPP_NUPTAKE                      1.0169E-11            NORMALIZED  1.1096E-05
 RMS NUPTAKE                          3.4379E-13            NORMALIZED  1.0551E-05
 RMS NUPTAKE_NPP_FRACTION             1.7209E-03            NORMALIZED  3.5615E-02
 RMS PFT_FIRE_CLOSS                   1.3028E-11            NORMALIZED  2.8993E-04
 RMS PFT_FIRE_NLOSS                   1.7545E-13            NORMALIZED  1.5848E-04
 RMS POTENTIAL_IMMOB                  1.9895E-11            NORMALIZED  1.0769E-04
 RMS POT_F_DENIT                      3.2735E-13            NORMALIZED  2.1044E-05
 RMS RETRANSN                         9.9678E-13            NORMALIZED  6.5183E-14
 RMS RR                               5.6945E-13            NORMALIZED  5.0942E-07
 RMS SLASH_HARVESTC                   8.8230E-15            NORMALIZED  1.0059E-11
 RMS SMINN                            3.5342E-08            NORMALIZED  1.5460E-09
 RMS SMINN_TO_NPOOL                   3.4379E-13            NORMALIZED  1.6110E-05
 RMS SMINN_TO_PLANT                   1.9679E-11            NORMALIZED  1.5495E-09
 RMS SMINN_TO_PLANT_FUN               3.4379E-13            NORMALIZED  1.6110E-05
 RMS SMINN_vr                         5.2653E-08            NORMALIZED  4.4009E-09
 RMS SMIN_NH4                         3.5157E-08            NORMALIZED  2.3163E-08
 RMS SMIN_NH4_vr                      5.2395E-08            NORMALIZED  2.7014E-08
 RMS SMIN_NO3                         5.9391E-10            NORMALIZED  2.7827E-11
 RMS SMIN_NO3_LEACHED                 1.3463E-17            NORMALIZED  7.0496E-09
 RMS SMIN_NO3_RUNOFF                  4.8427E-16            NORMALIZED  2.3254E-08
 RMS SMIN_NO3_vr                      1.3497E-09            NORMALIZED  1.3463E-10
 RMS SOIL1C                           2.4764E-07            NORMALIZED  2.9768E-09
 RMS SOIL1C_vr                        3.7053E-07            NORMALIZED  4.7971E-09
 RMS SOIL1N                           3.0955E-08            NORMALIZED  2.9768E-09
 RMS SOIL1N_vr                        4.1426E-08            NORMALIZED  5.3634E-09
 RMS SOIL2C                           4.5750E-08            NORMALIZED  1.8523E-11
 RMS SOIL2C_vr                        7.0063E-08            NORMALIZED  3.6995E-11
 RMS SOIL2N                           4.1591E-09            NORMALIZED  1.8523E-11
 RMS SOIL2N_vr                        5.6969E-09            NORMALIZED  4.1362E-11
 RMS SOILC_CHANGE                     1.0169E-11            NORMALIZED  1.1096E-05
 RMS SOILC_vr                         4.3996E-07            NORMALIZED  7.5212E-11
 RMS SOILN_vr                         5.2626E-08            NORMALIZED  9.8473E-11
 RMS SOM_C_LEACHED                    4.1610E-17            NORMALIZED  5.1216E-02
 RMS SR                               1.8218E-10            NORMALIZED  3.1956E-05
 RMS STORVEGC                         1.9889E-08            NORMALIZED  1.0640E-10
 RMS STORVEGN                         6.6016E-10            NORMALIZED  3.3721E-11
 RMS TOTCOLC                          2.7616E-02            NORMALIZED  1.1486E-06
 RMS TOTCOLCH4                        9.5756E-09            NORMALIZED  1.6811E-08
 RMS TOTCOLN                          7.5210E-10            NORMALIZED  3.8402E-13
 RMS TOTECOSYSC                       2.7616E-02            NORMALIZED  1.1473E-06
 RMS TOTECOSYSN                       7.5210E-10            NORMALIZED  3.8389E-13
 RMS TOTLITC                          3.0360E-03            NORMALIZED  1.4984E-05
 RMS TOTLITC_1m                       2.9757E-03            NORMALIZED  1.4804E-05
 RMS TOTLITN                          1.7478E-09            NORMALIZED  4.6007E-10
 RMS TOTLITN_1m                       1.7464E-09            NORMALIZED  4.6375E-10
 RMS TOTPFTC                          3.7336E-08            NORMALIZED  1.5666E-11
 RMS TOTPFTN                          7.4833E-10            NORMALIZED  2.4968E-11
 RMS TOTSOMC                          2.9332E-07            NORMALIZED  1.4057E-11
 RMS TOTSOMC_1m                       2.8788E-07            NORMALIZED  3.7422E-11
 RMS TOTSOMN                          3.5108E-08            NORMALIZED  1.8480E-11
 RMS TOTSOMN_1m                       3.4456E-08            NORMALIZED  4.9091E-11
 RMS TOTVEGC                          3.7336E-08            NORMALIZED  1.5490E-11
 RMS TOTVEGN                          7.4833E-10            NORMALIZED  2.4548E-11
 RMS WOODC                            2.1296E-08            NORMALIZED  1.0058E-11
 RMS WOODC_ALLOC                      6.7087E-14            NORMALIZED  4.6659E-08
 RMS WOODC_LOSS                       7.9003E-12            NORMALIZED  3.1116E-09
@ekluzek ekluzek added type: bug - impacts science priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations investigation Needs to be verified and more investigation into what's going on. labels Apr 5, 2019
@ekluzek ekluzek added this to the cesm2.1.1 milestone Apr 5, 2019
@ekluzek ekluzek self-assigned this Apr 5, 2019
@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 5, 2019

Note, that we don't see this in every case. The first month for BHIST, B1850, BW1850, BWHIST with and without cmip6 are identical (and cmip6 turns on c13 and c14 as well as both of the c13/c14 time-series files). So the cases that start at 1850, and have spunup Carbon isotopes from REFCASE files, all work as expected with identical answers (for at least the first month).

@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 6, 2019

Here's the list of fields different for the case when c14 is turned on rather than c13 (again after 1 time step).

 RMS ACTUAL_IMMOB                     1.8650E-21            NORMALIZED  1.0622E-14
 RMS AGNPP                            1.5705E-22            NORMALIZED  6.4772E-17
 RMS AR                               1.3577E-21            NORMALIZED  2.1823E-16
 RMS BGNPP                            6.0231E-24            NORMALIZED  2.7478E-18
 RMS CH4PROD                          6.4315E-22            NORMALIZED  1.1206E-14
 RMS CH4_SURF_AERE_SAT                4.3269E-23            NORMALIZED  1.3214E-15
 RMS CH4_SURF_AERE_UNSAT              6.8941E-24            NORMALIZED  3.8546E-15
 RMS CH4_SURF_DIFF_SAT                4.0561E-23            NORMALIZED  1.3279E-13
 RMS CH4_SURF_DIFF_UNSAT              1.6965E-23            NORMALIZED  6.1506E-14
 RMS COL_FIRE_CLOSS                   2.6848E-21            NORMALIZED  3.4885E-14
 RMS COL_FIRE_NLOSS                   2.6630E-23            NORMALIZED  1.7729E-14
 RMS CONC_O2_SAT                      5.7576E-18            NORMALIZED  9.6882E-17
 RMS CONC_O2_UNSAT                    1.0028E-17            NORMALIZED  5.8097E-18
 RMS COST_NACTIVE                     3.9928E-17            NORMALIZED  3.2645E-16
 RMS COST_NFIX                        3.4674E-18            NORMALIZED  5.6115E-16
 RMS CWDC                             2.5978E-12            NORMALIZED  4.3918E-15
 RMS CWDC_LOSS                        1.2783E-20            NORMALIZED  1.3105E-14
 RMS CWDC_vr                          3.6225E-12            NORMALIZED  4.5845E-15
 RMS CWDN_vr                          7.3041E-26            NORMALIZED  2.9611E-26
 RMS DENIT                            3.0936E-23            NORMALIZED  2.0540E-15
 RMS ER                               1.7616E-20            NORMALIZED  1.6304E-15
 RMS FAREA_BURNED                     4.3671E-24            NORMALIZED  1.4762E-14
 RMS FCH4                             1.5522E-25            NORMALIZED  6.5292E-15
 RMS FCH4TOCO2                        1.7723E-22            NORMALIZED  3.8117E-15
 RMS FCO2                             6.5999E-23            NORMALIZED  1.8843E-15
 RMS FPI                              1.2633E-16            NORMALIZED  1.2827E-16
 RMS FROOTC_ALLOC                     5.1129E-24            NORMALIZED  2.7467E-18
 RMS FROOTC_LOSS                      5.6395E-23            NORMALIZED  6.1731E-19
 RMS FUELC                            4.7138E-12            NORMALIZED  1.7489E-15
 RMS F_DENIT                          3.0936E-23            NORMALIZED  2.0540E-15
 RMS F_N2O_DENIT                      1.8376E-24            NORMALIZED  2.2473E-15
 RMS F_N2O_NIT                        1.3498E-28            NORMALIZED  8.7002E-18
 RMS F_NIT                            7.6759E-26            NORMALIZED  2.9685E-18
 RMS GR                               1.4109E-22            NORMALIZED  1.0490E-15
 RMS GROSS_NMIN                       3.4553E-24            NORMALIZED  1.3529E-17
 RMS HR                               1.7389E-20            NORMALIZED  3.7941E-15
 RMS HR_vr                            2.6131E-20            NORMALIZED  5.0873E-15
 RMS LEAFC_LOSS                       1.6604E-22            NORMALIZED  2.8039E-18
 RMS LITFALL                          5.7498E-22            NORMALIZED  6.3011E-19
 RMS LITR1C                           7.2286E-14            NORMALIZED  3.8190E-15
 RMS LITR1C_vr                        1.0978E-13            NORMALIZED  4.5198E-15
 RMS LITR1N                           1.2148E-18            NORMALIZED  2.7416E-18
 RMS LITR1N_vr                        4.9128E-18            NORMALIZED  1.0846E-17
 RMS LITR2C                           1.4514E-13            NORMALIZED  1.1452E-15
 RMS LITR2C_vr                        2.2111E-13            NORMALIZED  1.3647E-15
 RMS LITR2N                           6.1271E-18            NORMALIZED  2.7185E-18
 RMS LITR2N_vr                        6.1279E-19            NORMALIZED  2.7951E-19
 RMS LITR3C                           7.2497E-14            NORMALIZED  1.2731E-15
 RMS LITR3C_vr                        1.1039E-13            NORMALIZED  1.5387E-15
 RMS LITR3N_vr                        9.5736E-21            NORMALIZED  8.9675E-21
 RMS LITTERC_HR                       1.7374E-20            NORMALIZED  7.2336E-15
 RMS LITTERC_LOSS                     3.2925E-20            NORMALIZED  6.9836E-15
 RMS NACTIVE                          3.8695E-23            NORMALIZED  2.0167E-15
 RMS NACTIVE_NH4                      4.5682E-25            NORMALIZED  1.0915E-15
 RMS NACTIVE_NO3                      1.2929E-25            NORMALIZED  3.0859E-16
 RMS NAM                              4.5777E-25            NORMALIZED  5.8313E-16
 RMS NAM_NH4                          3.9100E-25            NORMALIZED  1.0581E-15
 RMS NAM_NO3                          1.3710E-25            NORMALIZED  3.2996E-16
 RMS NBP                              1.7973E-20            NORMALIZED  1.8815E-15
 RMS NDEPLOY                          3.5743E-23            NORMALIZED  1.0970E-15
 RMS NECM                             1.4006E-25            NORMALIZED  2.6708E-15
 RMS NECM_NH4                         1.3932E-25            NORMALIZED  2.8445E-15
 RMS NECM_NO3                         1.4200E-27            NORMALIZED  4.1016E-16
 RMS NEE                              1.7969E-20            NORMALIZED  1.9356E-15
 RMS NEM                              5.6211E-22            NORMALIZED  1.6362E-14
 RMS NEP                              1.7637E-20            NORMALIZED  1.9050E-15
 RMS NET_NMIN                         1.8651E-21            NORMALIZED  2.3028E-14
 RMS NFIRE                            9.8251E-25            NORMALIZED  1.0341E-14
 RMS NFIX                             3.4441E-24            NORMALIZED  1.5991E-15
 RMS NNONMYC                          3.8232E-23            NORMALIZED  2.0836E-15
 RMS NNONMYC_NH4                      3.8423E-23            NORMALIZED  2.3085E-15
 RMS NNONMYC_NO3                      4.9968E-25            NORMALIZED  2.9301E-16
 RMS NPOOL                            7.3475E-20            NORMALIZED  5.4204E-19
 RMS NPP                              1.5035E-21            NORMALIZED  2.2801E-16
 RMS NPP_GROWTH                       1.1829E-21            NORMALIZED  9.6743E-16
 RMS NPP_NACTIVE                      8.2221E-22            NORMALIZED  1.0867E-15
 RMS NPP_NACTIVE_NH4                  6.1941E-23            NORMALIZED  9.4348E-16
 RMS NPP_NACTIVE_NO3                  2.1982E-23            NORMALIZED  2.2782E-15
 RMS NPP_NFIX                         3.5745E-23            NORMALIZED  1.7289E-15
 RMS NPP_NNONMYC                      7.3148E-22            NORMALIZED  1.0737E-15
 RMS NPP_NNONMYC_NH4                  6.3313E-22            NORMALIZED  1.0343E-15
 RMS NPP_NNONMYC_NO3                  1.3895E-22            NORMALIZED  2.0091E-15
 RMS NPP_NUPTAKE                      1.2732E-21            NORMALIZED  1.3893E-15
 RMS NUPTAKE                          3.5743E-23            NORMALIZED  1.0970E-15
 RMS NUPTAKE_NPP_FRACTION             1.8121E-07            NORMALIZED  3.7495E-06
 RMS PFT_FIRE_CLOSS                   1.3721E-21            NORMALIZED  3.0534E-14
 RMS PFT_FIRE_NLOSS                   1.8084E-23            NORMALIZED  1.6335E-14
 RMS POTENTIAL_IMMOB                  1.8953E-21            NORMALIZED  1.0259E-14
 RMS POT_F_DENIT                      3.0980E-23            NORMALIZED  1.9915E-15
 RMS RR                               9.5682E-23            NORMALIZED  8.5597E-17
 RMS SMINN                            8.8969E-18            NORMALIZED  3.8917E-19
 RMS SMINN_TO_NPOOL                   3.6992E-23            NORMALIZED  1.7334E-15
 RMS SMINN_TO_PLANT                   5.5729E-21            NORMALIZED  4.3880E-19
 RMS SMINN_TO_PLANT_FUN               3.6992E-23            NORMALIZED  1.7334E-15
 RMS SMINN_vr                         1.2809E-17            NORMALIZED  1.0706E-18
 RMS SMIN_NH4                         5.4936E-18            NORMALIZED  3.6195E-18
 RMS SMIN_NH4_vr                      1.1129E-17            NORMALIZED  5.7378E-18
 RMS SMIN_NO3                         2.8761E-19            NORMALIZED  1.3476E-20
 RMS SMIN_NO3_LEACHED                 5.2281E-29            NORMALIZED  2.7376E-20
 RMS SMIN_NO3_RUNOFF                  7.1937E-28            NORMALIZED  3.4544E-20
 RMS SMIN_NO3_vr                      4.4249E-19            NORMALIZED  4.4140E-20
 RMS SOIL1C                           3.6311E-16            NORMALIZED  4.3648E-18
 RMS SOIL1C_vr                        3.9809E-16            NORMALIZED  5.1540E-18
 RMS SOIL1N                           1.3786E-17            NORMALIZED  1.3258E-18
 RMS SOIL1N_vr                        4.0253E-17            NORMALIZED  5.2115E-18
 RMS SOIL2C                           3.1371E-15            NORMALIZED  1.2701E-18
 RMS SOIL2C_vr                        2.8277E-15            NORMALIZED  1.4931E-18
 RMS SOIL2N_vr                        5.9835E-22            NORMALIZED  4.3442E-24
 RMS SOILC_CHANGE                     1.2732E-21            NORMALIZED  1.3893E-15
 RMS SOILC_vr                         5.6556E-15            NORMALIZED  9.6684E-19
 RMS SOILN_vr                         5.4803E-18            NORMALIZED  1.0255E-20
 RMS SOM_C_LEACHED                    1.7540E-18            NORMALIZED  2.1590E-03
 RMS SR                               1.7388E-20            NORMALIZED  3.0501E-15
 RMS TOTCOLC                          2.9509E-12            NORMALIZED  1.2274E-16
 RMS TOTCOLCH4                        1.4328E-18            NORMALIZED  2.5154E-18
 RMS TOTECOSYSC                       2.9242E-12            NORMALIZED  1.2148E-16
 RMS TOTLITC                          2.8925E-13            NORMALIZED  1.4276E-15
 RMS TOTLITC_1m                       2.8434E-13            NORMALIZED  1.4145E-15
 RMS TOTLITN                          3.0636E-18            NORMALIZED  8.0641E-19
 RMS TOTLITN_1m                       3.0636E-18            NORMALIZED  8.1353E-19
 RMS TOTSOMC_1m                       7.8427E-16            NORMALIZED  1.0195E-19
 RMS WOODC_ALLOC                      1.4920E-23            NORMALIZED  1.0377E-17
 RMS WOODC_LOSS                       8.2299E-22            NORMALIZED  3.2415E-19

@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 6, 2019

The fields that are ignored are for C14 (and a similar list for C13):

Could not find match for file1 variable C14_AR in file2
Could not find match for file1 variable C14_GPP in file2
Could not find match for file1 variable C14_HR in file2
Could not find match for file1 variable C14_NBP in file2
Could not find match for file1 variable C14_SOILC_vr in file2
Could not find match for file1 variable C14_TOTECOSYSC in file2
Could not find match for file1 variable C14_TOTLITC in file2
Could not find match for file1 variable C14_TOTSOMC in file2
Could not find match for file1 variable C14_TOTVEGC in file2

@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 6, 2019

The difference between the C13 and C14 case is limited to two fields (after 20 time-steps).

RMS time_bounds 1.3258E-01 NORMALIZED 8.7779E-01
RMS FSDS 9.2472E+01 NORMALIZED 4.7918E-01

@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 6, 2019

Strangely the case with BOTH c13 and c14 on is different from all the previous cases: no-ciso, c13 (by 117 fields), and c14 (by 157 fields). That's just at the first time-step.

@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 6, 2019

Turning on c13 timeseries, and comparing to c13 alone, gives answer changes, but it looks like it may largely be roundoff. 248 fields differ after 36000 seconds (20 time-steps), but the RMS difference is between E-11 and E-30, with the median around E-19, only 16 greater than E-14.

@klindsay28
Copy link

subroutine DynamicAreaConservation of biogeochem/CNVegetationFacade.F90 has the following lines of code:

    call CStateUpdateDynPatch(bounds, num_soilc_with_inactive, filter_soilc_with_inactive, &
         this%cnveg_carbonflux_inst, this%cnveg_carbonstate_inst, &
         soilbiogeochem_carbonstate_inst)
    if (use_c13) then
       call CStateUpdateDynPatch(bounds, num_soilc_with_inactive, filter_soilc_with_inactive, &
            this%c13_cnveg_carbonflux_inst, this%c13_cnveg_carbonstate_inst, &
            soilbiogeochem_carbonstate_inst)
    end if
    if (use_c14) then
       call CStateUpdateDynPatch(bounds, num_soilc_with_inactive, filter_soilc_with_inactive, &
            this%c14_cnveg_carbonflux_inst, this%c14_cnveg_carbonstate_inst, &
            soilbiogeochem_carbonstate_inst)
    end if

Passing soilbiogeochem_carbonstate_inst to CStateUpdateDynPatch in the use_c13 and use_c14 blocks, instead of c13_soilbiogeochem_carbonstate_inst and c14_soilbiogeochem_carbonstate_inst looks suspiscious to me. It looks like perhaps soilbiogeochem_carbonstate_inst is getting updated multiple times when carbon isotopes are turned on.

@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 17, 2019

Ohh, nice, thanks for finding that @klindsay28!

I think you are right, this looks wrong. It looks to me that the code should be...

diff --git a/src/biogeochem/CNVegetationFacade.F90 b/src/biogeochem/CNVegetationFacade.F90
index 859d92b..b3b900d 100644
--- a/src/biogeochem/CNVegetationFacade.F90
+++ b/src/biogeochem/CNVegetationFacade.F90
@@ -703,12 +703,12 @@ subroutine DynamicAreaConservation(this, bounds, clump_index, &
     if (use_c13) then
        call CStateUpdateDynPatch(bounds, num_soilc_with_inactive, filter_soilc_with_inactive, &
             this%c13_cnveg_carbonflux_inst, this%c13_cnveg_carbonstate_inst, &
-            soilbiogeochem_carbonstate_inst)
+            c13_soilbiogeochem_carbonstate_inst)
     end if
     if (use_c14) then
        call CStateUpdateDynPatch(bounds, num_soilc_with_inactive, filter_soilc_with_inactive, &
             this%c14_cnveg_carbonflux_inst, this%c14_cnveg_carbonstate_inst, &
-            soilbiogeochem_carbonstate_inst)
+            c14_soilbiogeochem_carbonstate_inst)
     end if
     call NStateUpdateDynPatch(bounds, num_soilc_with_inactive, filter_soilc_with_inactive, &
          this%cnveg_nitrogenflux_inst, this%cnveg_nitrogenstate_inst,

@billsacks
Copy link
Member

Yes, good find indeed @klindsay28 ! Mea culpa! I agree with the fix.

@bishtgautam you may want to check whether the same bug exists in ELM.

@bishtgautam
Copy link

@billsacks Thanks for pointing this out

@klindsay28
Copy link

I've tested those changes in 2 5-day long coupled ssp585 experiments, with and without carbon isotopes. CLM output, written every timestep, in these experiments is identical, and agrees with output from a non-isotope experiment done without the changes.

@dlawrenncar
Copy link
Contributor

dlawrenncar commented Apr 17, 2019 via email

ekluzek added a commit to ekluzek/CTSM that referenced this issue Apr 17, 2019
…ified when only isotopic soil carbon should have
@ekluzek
Copy link
Collaborator Author

ekluzek commented Apr 17, 2019

I have a branch on my fork "cisofix" with the fix put into place.

@billsacks
Copy link
Member

(Maybe this is already apparent to everyone following this issue, but in case not....)

I believe the impact of this bug is the following: When there are changes in patch areas (in the first timestep of each year in a run with transient vegetation), what's supposed to happen is: The bulk soil C pools get an addition of C equal to (some fraction of?) the bulk C in roots, and similarly for the C isotopes. What was actually happening was that the root C isotope pools were being added to the bulk soil C pools rather than the isotope soil C pools in this case. So my guess is that the impact on bulk C is relatively small (assuming that isotope ratios are relatively small) - though I can't judge whether this might still impact long-term soil C significantly. But the impact on the isotopic concentrations would probably be larger.

@billsacks billsacks changed the title Turning Carbon isotopes will sometimes cause a change in answers... Enabling carbon isotopes changes answers for transient cases Apr 18, 2019
@dlawrenncar
Copy link
Contributor

Diagnostics from a run with the bug fix compared to control run are here:

[http://webext.cgd.ucar.edu/I20TR/clm50_release-clm5.0.20_1deg_GSWP3V1_isofix2_hist/lnd/clm50_release-clm5.0.20_1deg_GSWP3V1_isofix2_hist.1995_2014-clm50_release-clm5.0.20_1deg_GSWP3V1_isofix_hist.1995_2014/setsIndex.html]

Main message is that bulk carbon is essentially not affected, fortunately. Significant impacts on 13C and 14C SOM, litter C, and HR are seen that effectively invalidate that data in regions where there has been land use change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
investigation Needs to be verified and more investigation into what's going on. priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants