-
Notifications
You must be signed in to change notification settings - Fork 312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New CN matrix fails with single point sites with the new ctsm5.3 datasets. #2780
Comments
This is the only test we have for mimics_matrixcn. It's also possible that the tests that passed would fail if run out far enough. |
Here's the note about this test when it was added. I'm also doing some longer and different tests in ctsm5.2.028 to see the test just happened to pass since it was too short. As well as making sure the same test works without MIMCS. |
Longer tests and tests at f10 in ctsm5.2.028 seem to be fine. SMS_D.1x1_brazil.I1850Clm60BgcCrop.derecho_intel.clm-mimics_matrixcn So maybe there is something specific about this with ctsm5.3.0 datasets. We'll mark this as an expected fail for now though. |
The other test that fails in the same way is: SMS_Ld10_D_Mmpi-serial.CLM_USRDAT.I1PtClm60Bgc.derecho_gnu.clm-default--clm-NEON-HARV--clm-matrixcnOn |
...and SMS_Ld10_D_Mmpi-serial.CLM_USRDAT.I1PtClm60Bgc.izumi_nag.clm-default--clm-NEON-HARV--clm-matrixcnOn |
My gut feeling is that these tests need new finidat files, based on past experiences where CNmatrix has crashed with one finidat and not with another (#2592). E.g. the nearest neighbor from the finidat may not contain the right pft combinations needed for these single-point simulations. |
In one of the failing tests, I changed finidat from Next I want to try setting finidat to the interpolated file saved in |
A broader question we wonder here (@slevis-lmwg and I) for the group to assess: (discussed at CTSM SE Oct/10th/2024)
|
maybe matrix tests always need to start from a cold start? if you're running matrix, then by definition you're doing a spinup. |
I updated the questions above, from the mornings discussion. |
Troubleshooting suggests that my gut feeling was wrong.
1x1 matrix tests that pass:
|
Trying a Clm6 version and Clm6 DEBUG version of the first in the above list of already passing tests:
and non-DEBUG versions of the failing tests:
So DEBUG must be uncovering a problem in these two. I will think about what I want to try next... |
I added diagnostic write-statements just before the error gets triggered in SparseMatrixMultiplyMod.F90 line 1246: |
@ekluzek I will run this by you before I open a PR with this code change. My branch is in this directory: |
@slevis-lmwg that's correct the inequality should be >= rather than just >. One point there is to just make sure the array size isn't too small. The array must've been larger all the time previously. I'd have to think about why that's the case... I'm glad you were able to figure that out. |
Since ctsm5.2.dev175 to ctsm5.3.0 we've been running tests with MIMICS with above ground CN matrtix that have been passing. The test is SMS_D.1x1_brazil.I1850Clm60BgcCrop.derecho_gnu.clm-mimics_matrixcn. This has the soil CN matrix off (because MIMICS is non-linear), but above ground CN matrix on (use_soil_matrixcn = .false. use_matrixcn = .true.).
There are two reasons for doing this test:
The hope for "1" was especially there as we weren't finding methods to speed up the spinup of MIMICS. The test did pass for 30 tags, and just started failing in ctsm5.3.0 with the following type of error in the log files:
lnd.log:
cesm.log:
The line it fails on from above is the SHR_ASSERT_FL in this section of code in SparseMatrixMultiplyMod.F90:
The call in CNVegMatrixMod.F90 is here:
Definition of done:
The text was updated successfully, but these errors were encountered: