Skip to content

Commit

Permalink
Add note on need to set MPI_TYPE_DEPTH for LILAC
Browse files Browse the repository at this point in the history
  • Loading branch information
billsacks committed Nov 4, 2022
1 parent fbfa05f commit 979c71f
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 0 deletions.
1 change: 1 addition & 0 deletions doc/source/lilac/obtaining-building-and-running/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@

obtaining-and-building-ctsm.rst
setting-ctsm-runtime-options.rst
notes-on-running-ctsm.rst
restarting.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
.. highlight:: shell

.. _notes-on-running-ctsm:

=======================
Notes on running CTSM
=======================

.. _runtime-environment-variables:

Environment variables that may need to be set at runtime
========================================================

With the MPT MPI library (which is the default MPI library on NCAR's cheyenne machine), it is important to set the environment variable ``MPI_TYPE_DEPTH`` to 16 when running CTSM (this setting is required by the Parallel IO library). Typically you should set this variable in your job submission script, using either:

.. code-block:: Bash
export MPI_TYPE_DEPTH=16
or:

.. code-block:: Tcsh
setenv MPI_TYPE_DEPTH 16
prior to running the model.
3 changes: 3 additions & 0 deletions doc/source/lilac/specific-atm-models/wrf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -390,8 +390,11 @@ A simple PBS script to run WRF-CTSM on ``Cheyenne`` looks like this:
#PBS -l select=2:ncpus=36:mpiprocs=36
### Run the executable
setenv MPI_TYPE_DEPTH 16
mpiexec_mpt ./wrf.exe
(See :numref:`runtime-environment-variables` for a description of the need to set ``MPI_TYPE_DEPTH`` on ``Cheyenne``.)

To submit a batch job to the ``Cheyenne`` queues, use ``qsub`` command followed
by the PBS script name.
For example, if you named this script ``run_wrf_ctsm.csh``, submit the job like this::
Expand Down

0 comments on commit 979c71f

Please sign in to comment.