This is a small changeset, which only adds the passing of a text based flag to FATES, that defines which soil decomposition method is active. This setting is needed by FATES, so that it can pass specific boundary conditions to CLM (and potentially ELM in the future). This change was necessitated because MIMICS requires a Nitrogen to lignin-carbon ratio in the litter flux, yet other decomposition schemes do not. Moreover, the generation of this boundary condition is not trivial, so it is worth it to avoid its computation if mimics is not on.  This change should have NO impact on any tests.

slevis resolved conflicts:
cime_config/testdefs/ExpectedTestFails.xml
cime_config/testdefs/testlist_clm.xml
src/biogeochem/CNPhenologyMod.F90

Refactor NutrientCompetition / CNAllocation to provide hooks for AgSys

Major refactor of NutrientCompetition / CNAllocation to provide hooks
for AgSys crop model: separates the NutrientCompetition modules into
pieces based on (1) consolidating duplicate code between the Clm45 and
FlexibleCN versions, and (2) separating pieces that will vs. won't be
used for crop patches when running with the upcoming AgSys crop model.

I have restored the old CNAllocationMod, with some of the
responsibilities that it used to have. (I'm not sure it's appropriate to
have the calculation of gpp and maint resp in CNAllocationMod, but I
left it there because it has always been combined with the allocation
code, including back when we had a separate CNAllocationMod.)

slevis resolved conflicts:
src/biogeochem/CNDriverMod.F90
src/biogeochem/NutrientCompetitionFlexibleCNMod.F90
src/biogeochem/NutrientCompetitionCLM45defaultMod.F90

slevis important note:
To resolve conflicts in the third (ie, last) of the above files, I
checked with Will Wieder, and we agreed to not maintain a clm45
version of matrixcn.

Get FATES and carbon_only test-suite tests to pass with MIMICS active

To pass with MIMICS active, FATES and carbon_only tests need calculations of:
ligninNratioAvg and annsum_npp_col, both currently declared and calculated in CNVegCarbonFluxType.F90.

ligninNratioAvg is a function of leafc_to_litter_patch, frootc_to_litter_patch, soilbiogeochem_cwdc_col, soilbiogeochem_cwdn_col, soilbiogeochem_decomp_cascade_ctransfer_col.

I got both FATES and carbon_only tests to pass by declaring local copies (in MIMICS) of ligninNratioAvg and annsum_npp_col.

slevis resolved conflicts:
src/biogeochem/CNVegCarbonFluxType.F90
src/soilbiogeochem/SoilBiogeochemCarbonFluxType.F90

First pass at getting fates-sp mode to run without soil BGC

Bring in externals updates that include adding AWS as a supported machine.

Add a FatesSP compset and user-mod directory.

Turn soil-BGC off when running FatesSP. Replace PGI tests on izumi with nvhpc tests on cheyenne.
Fix an issue with a particular gnu compiler version in ch4Mod.F90 in DEBUG mode. Add some more
threading tests for Fates. Start adding a prescribed-LAI tests for FatesSp

Add a file to setup a python environment in conda for the python testing. Set line length for black
to same as pylint (100). Add some extra excludes for black to the pylint checking.

Paths for CIME python code was changed, which required changes to some CTSM python code.

slevis resolved conflicts:
cime_config/testdefs/ExpectedTestFails.xml
cime_config/testdefs/testlist_clm.xml
src/biogeochem/CNCStateUpdate1Mod.F90

Update cime to fix DAE issue
modify_neon issue
When FatesSp is on, make soil_decomp None by default

slevis resolved conflicts:
cime_config/testdefs/ExpectedTestFails.xml

This set of changes 1) allows FATES to dictate the number of patches on the naturaly vegetated land unit and 2) is compatible with a version of FATES that updates the parameter file format (changes in parameter naming and parameter set).  Efforts were made to adhere to the system of how patches are looped and loop bounds are defined. There is interaction with the reading of the surface file, which previously dictated all patch counts based on the number of PFTs in the file.  FATES-SP and biogeography preserves the ability to use surface file weighting factors associated with PFTs and CFTs to drive area fractions in FATES.  This change is motivated by FATES-side updates to the parameter file, which includes new settings where the user can control the number of primary "fates_maxpatch_primary" and secondary "fates_maxpatch_secondary" patches. Users should also be aware of a new namelist setting "use_fates_tree_damage" which defaults to false (inoperative).

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl

First Commit of CN Matrix Solution:
Start bringing in the use_matrixcn and use_soil_matrixcn control options for the Carbon/Nitrogen BGC model using
the matrix solution methodology developed by Dr. Yiqi Luo and his EcoLab members: Drs. Xingjie Lu,
Yuanyuan Huang and Zhengguang Du, at Northern Arizona University

Work on Anomaly forcing for SSEP scenarios:
Do some work on anomoly forcing for SSP scenarios. Update the script in tools/contrib to work with the latest
data and replicate results of Keith Oleson and Sean Swenson. Add some test cases for it.

Python Environment Management using Conda:
More work on the manage_python_env script to setup the "ctsm_py" conda environment. This now works for UCAR machines
(such as cheyenne and casper) as well as CGD machines (like izumi). This script can also be called for tools, and
the test tools mechanism is now using it rather than ncar_pylib.

./manage_python_env

SCAM Fix:
Add checkimport method for CMEPS so that single-column case can work with SCAM, from Jim Edwards.

Black Formatter for Python Code:
Run "black" python code reformatter on all code under the "python" directory. Add the commits
to .git-blame-ignore-revs so they can be ignored when "git blame" is used, if you add this
to each clone where you want this to apply:

   git config blame.ignoreRevsFile .git-blame-ignore-revs

You can do it for your worktrees, but since the file only applies to CTSM for ctsm5.1.dev100 and
forward, you'll need to do it for each CTSM clone seperately. Also there is now a github action
that checks that python code under the "python" subdirectory is black clean. If it's not the action
will fail and send you an email about it.

Make Map Data Fix:
Simple fix from Sam Rabin.

Fix for DISPLA on History Output:
Fix from Keith Oleson to set history output for DISPLA (displa_patch) to zero when vegetation is buried by snow.

slevis resolved conflicts:
bld/CLMBuildNamelist.pm
bld/namelist_files/namelist_defaults_ctsm.xml
bld/unit_testers/build-namelist_test.pl
cime_config/testdefs/testlist_clm.xml
src/biogeochem/CNAllocationMod.F90
src/biogeochem/CNCStateUpdate1Mod.F90
src/biogeochem/CNDriverMod.F90
src/biogeochem/CNNStateUpdate1Mod.F90
src/biogeochem/CNPhenologyMod.F90
src/biogeochem/CNVegCarbonFluxType.F90
src/biogeochem/NutrientCompetitionCLM45defaultMod.F90
src/biogeochem/NutrientCompetitionFlexibleCNMod.F90
src/soilbiogeochem/SoilBiogeochemCarbonFluxType.F90

slevis had to restore many other files for which git replaced the
matrixcn mods with dev100's matrixcn stubs.

slevis is testing the code with
SMS_Ln9.ne30pg2_ne30pg2_mg17.I2000Clm50BgcCrop.cheyenne_intel.clm-clm50cam6LndTuningMode
and getting a column cbalance error at the moment.

Use conda environment rather than ncar_pylib with the fsurdat_modifier system test

Reason: ncar_pylib is going away soon.

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl
cime_config/testdefs/testlist_clm.xml
src/soilbiogeochem/SoilBiogeochemCarbonStateType.F90

 Get gross unrepresented land use transitions working in CLM5.1. This is additional optional
 data added to the landuse.timeseries files for transient simulations. The current landuse.timseries
 files have this data, but it's set to zero. This data will be part of the CTSM5.2 surface dataset
 (that is upcoming) and be on by default for clm5_3 physics.

slevis resolved conflict:
bld/unit_testers/build-namelist_test.pl

FATES API update to facilitate fates refactor

This updates a number of FATES type names and module use statements
which correspond with a refactoring effort that moves FATES
patches and cohorts into their own respective modules.

With the FATES update is a minor science update, so there are
changes to answers for FATES.

This also incorporates a minor update to a more recent version
of the ccs config external.

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl
cime_config/buildnml
src/biogeochem/CNVegCarbonFluxType.F90

Migration of FATES to share normal soil BGC call sequence and functionality

This set of changes enables the normal soil biogeochemistry that is used for CN, to be used
for FATES as well. FATES had been using a simplified subset of soil biogeochemistry in its
own module. This change required coordination of litter flux and methane boundary conditions
from FATES to CLM.  CNVEG datastructures were given trivial allocation (of size one on index
zero) to prevent inappropriate use of CNVEG datastructures while FATES is active. Note that
now the carbon balance checking for the soil is now active when FATES is active. Various
accomodations have also been put in place to enable nitrogen cycling between the two models.

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl
src/biogeochem/CNCStateUpdate1Mod.F90
src/biogeochem/CNDriverMod.F90
src/biogeochem/CNVegCarbonStateType.F90
src/biogeochem/CNVegNitrogenStateType.F90
src/biogeochem/CNVegetationFacade.F90
src/soilbiogeochem/SoilBiogeochemCarbonFluxType.F90
src/soilbiogeochem/SoilBiogeochemLittVertTranspMod.F90
src/soilbiogeochem/SoilBiogeochemNStateUpdate1Mod.F90
src/soilbiogeochem/SoilBiogeochemNitrogenFluxType.F90
src/soilbiogeochem/SoilBiogeochemNitrogenStateType.F90

because I updated 2 local copies of cn-matrix_v3 at the same time

Used this one for testing:
SMS_D.1x1_brazil.I1850Clm51BgcCrop.cheyenne_gnu.clm-mimics_matrixcn

Various BFB fixes and updates

Purpose/description of changes
------------------------------
the default comes in a later tag (slevis)

Regular and python testing passed.

Does not change answers relative to dev158.

slevis resolved conflicts UNTESTED:
bld/unit_testers/build-namelist_test.pl
cime_config/testdefs/testlist_clm.xml
src/biogeochem/CNPhenologyMod.F90
src/biogeochem/CNVegNitrogenFluxType.F90
src/biogeochem/NutrientCompetitionFlexibleCNMod.F90
AND I will still look for potential additional inconsistencies that I
may need to resolve

because I updated 2 parallel local copies of cn-matrix_v3

Improvements to processing of crop calendar files

In python/ctsm/crop_calendars/, process_ggcmi_shdates.py and regrid_ggcmi_shdates.py are used to convert the raw GGCMI crop calendar files into CTSM-compatible versions. This update fixes some bugs, removes dependencies on the nco utilities, enables the use of surface datasets as template files (in addition to the existing ability to use CTSM output files), and standardizes things for consistency with other CTSM Python tools.

Adds capability for cropland soil tillage and post-harvest residue removal.

Tillage: This PR brings in the tillage code written by Sam Levis and Michael Graham and used in Graham et al. (2021, ERL, doi:10.1088/1748-9326/abe6c6). Low- and high-intensity tillage here work by increasing the decomposition rate of different soil carbon pools. These "decomposition multipliers" vary based on soil pool and how long it's been since the crop was planted; they are set with new paramfile variables till_decompk_multipliers and mimics_till_decompk_multipliers. Note that tillage is off by default.

Residue removal: Adds a parameter hat represents what fraction of post-harvest crop residues (stems and leaves) should be removed to the crop product pool rather than being transferred to litter.

slevis resolved conflicts:
src/soilbiogeochem/SoilBiogeochemDecompCascadeBGCMod.F90

Changes to improve vegetation health at high latitudes

Details in PR ESCOMP#2348

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl

…_diag

Ran aux_clm on derecho and got expected results

Merge b4bdev 20240321

Merge b4b-dev

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl

Merge b4b-dev

Some fixes for run_neon that came up in ctsm5.1.dev172.
Small bug fix for units of vector format hillslope hydrology.
Start adding PLUMBER2 users-mods (NOT functional)

slevis resolved conflicts:
.git-blame-ignore-revs
bld/namelist_files/namelist_defaults_ctsm.xml
bld/unit_testers/build-namelist_test.pl
cime_config/config_compsets.xml
cime_config/testdefs/testlist_clm.xml

ix clm6_0 defaults and CESM testing issues, add tests to detect these problems

Fix the clm6_0 defaults that were incorrect in ctsm5.2.0. The use-cases needed to be changed to handle clm6_0.
And clm6_0 wasn't handled in the merge of ctsm5.1.dev174 for the two new settings. Simplified the use-cases which
should help prevent these problems in the future. use_init_interp will be set for ctsm5.1 finidat files.

Fix some testing. mksurfdata_esmf and the MKSUIRFDATA test didn't work in a CESM checkout. build-namelist unit tests
weren't going over the use-cases. Also started adding some tests to go over finidat files, but didn't yet capture all of them.
Add some scripts to compare namelists in order to detect namelist issues for physics
version updates, and more easily see namelist changes in one place (bld/unit_testers/cmp_baseline_lnd_in_files,
bld/unit_testers/compare_namelists).

Add some tests to detect issues in CESM and CAM testing (add more tests for cam6.0 forcing, add new IHistClm60BgcCropG
compset). Add a NoAnthro compset to the ctsm_sci testing. Add I1850Clm60Sp test for f09 and f19 to ctsm_sci.

Change NoAnthro compsets to use RTM rather than MOSART. Add science support to some clm6_0 compsets, as was obvious.

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl
cime_config/testdefs/ExpectedTestFails.xml
cime_config/testdefs/testlist_clm.xml

Rm manage_externals and update documentation accordingly

Fixes ESCOMP#2537 Remove manage_externals stuff from run_sys_tests
Fixes ESCOMP#2536 Update documentation for git-fleximod
Fixes ESCOMP#2467 Remove references to cheyenne from the testlist

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl
cime_config/testdefs/ExpectedTestFails.xml
cime_config/testdefs/testlist_clm.xml

Merge b4b-dev

slevis resolved conflicts:
.git-blame-ignore-revs
bld/unit_testers/build-namelist_test.pl
src/biogeochem/CNDriverMod.F90
src/main/controlMod.F90

Relax tolerance for truncating small snocan values in CanopyFluxes

See the PR ESCOMP#2457 for details.

slevis resolved conflicts:
bld/unit_testers/build-namelist_test.pl