CESM System testing workflow

CESM supports a wide variety of compsets and model configurations on several different hardware platforms and compilers. We strive for testing to cover a majority of the compsets and configurations used by the community. In order to do this we distribute tests over the available hardware. The tests are described in XML documents which are distributed with the CIME source for allactive, coupler test and data compsets and with the component models for component specific compsets. The location of these XML files is given in cime/config/cesm/config_files.xml in variable TESTS_SPEC_FILE. Each of these files provides a list of tests, compilers, machines and test categories. The CSEG integration team at NCAR regularly performs prealpha, and prebeta category tests and the component support teams regularly perform auxiliary tests associated with that team's component. Tests are performed using the cime tool create_test with xml arguments like:

./create_test --xml-category prealpha --xml-machine yellowstone --xml-compiler intel 

will conduct the prealpha test suite for the intel compiler on yellowstone.

The CSEG team regularly conducts prealpha and prebeta tests on model snapshots and tags these snapshots with names of the form CESM2.0.alpha01a or CESM2_0_beta01, each CESM tag represents a collection of component tags along with a CIME tag.

Putting this all together, a typical run of the prealpha test suite on yellowstone could look like:

./create_test --xml-category prealpha --xml-machine yellowstone --xml-compiler intel --compare CESM2.0.alpha01a --generate CESM2.0.alpha01b
./create_test --xml-category prealpha --xml-machine yellowstone --xml-compiler pgi --compare CESM2.0.alpha01a --generate CESM2.0.alpha01b
./create_test --xml-category prealpha --xml-machine yellowstone --xml-compiler gnu --compare CESM2.0.alpha01a --generate CESM2.0.alpha01b

Notice that we use tags heavily in this workflow: In general, each baseline directory is associated with a given, frozen tag. This means that we have many old baseline directories on disk. Once a tag is made, the baseline directory for that tag is kept around for some period, to facilitate testing of feature branches that may have been made off of the given tag.

Also note that, on any given machine, there is a single baseline directory for each tag, containing tests spanning all compilers used on that machine. (If there are identical tests with different compilers, they will be distinguished by their test names - e.g., XXXX.yellowstone_intel vs. XXXX.yellowstone_pgi.)