Skip to content

CAM Development Workflow in GitHub

Jump to bottom
Jesse Nusbaumer edited this page Nov 21, 2019 · 48 revisions

This page contains the standard workflow for development and/or bug fixes for the NCAR Community Atmosphere Model (CAM) using Github.

This page is a work in progress, and will likely be edited frequently over the next several weeks/months.

Please use any recommendations here with caution until the workflow is finalized and becomes "official".

How to notify NCAR of a problem or bug in CAM

  1. First, search the DiscussCESM forums to see if your problem has already been mentioned, and if a useful answer has been provided.

  2. If the problem or bug hasn't been noted on the forums, then create a new topic and describe the problem or bug there. If the problem has already been described in the forums, but hasn't received a useful answer, then please add a post to that topic indicating that you also ran into this particular problem. Both of these options will require you to either log-in to the forums, or to register if you don't already have an account.

  3. If the problem or bug is found to be in need of fixing, then a GitHub issue will be created and added to CAM's issue list, and will remain open and searchable until the problem or bug has been fixed by an NCAR AMP scientist or software engineer. If you already have the fix or solution on your own personal GitHub fork, then please notify the scientist or engineer so that your fix can be implemented into CAM itself (see "How to make/store revisions to your personal CAM repository" below).

How to make/store revisions to your personal CAM repository (GitHub fork)

  1. You will work on your fork in a clone. If you do not already have a fork of ESCOMP/CAM, create one. See Working with clones for an example of the button used to create a fork.

  2. If you do not already have a clone, create one:

% git clone https://github.com/<GitHub userid>/CAM
% cd CAM
% git remote add ESCOMP https://github.com/ESCOMP/CAM
% git fetch --tags ESCOMP

For help setting up a new clone see Working with clones.

  1. Create and checkout a branch for your work. Normally, your branch should start from the ESCOMP development branch:
% git branch <new_branch_name> ESCOMP/cam_development
% git checkout <new_branch_name>

unless you are specifically working on a release bug fix:

% git branch <new_branch_name> ESCOMP/cam_cesm2_1_rel
% git checkout <new_branch_name>

or are starting a bug fix from a specific CAM tag:

% git branch <new_branch_name> cam6_2_000
% git checkout <new_branch_name>

where <new_branch_name> should describe the change you are going to make (the second argument is where that branch begins).

  1. Make modifications or changes to the branch as you see fit and run some relevant tests.

  2. Commit new changes to local git clone, e.g.:

% git add <new_file(s)>
% git commit -am "<description of changes>"

where <description of changes> should describe all the changes in this commit. To enter a longer commit message, omit the m and the "<description of changes>" and an editor window should open that allows you to enter a detailed message. Note that the first command is only needed if you added new files to your clone (but always take a moment to think about it as forgotten files are a common source of problems that occur during testing). If you aren't sure what files have been added or removed, then type the command:

% git status

which will list all added, removed, and modified files, including files which are in the git repo's directories, but which are not being tracked by git itself.

  1. Push changes back to personal CAM Fork:
% git push <origin> <new_branch_name>

where <origin> is the name of the source repository (the default name is 'origin', you might have chosen a different name when you created the clone).

  1. Run CAM tests You should run tests fairly frequently as you develop new code or make modifications to existing code. In particular, it is important that there is a test for your simulation -- something that tests the new feature. All tests are run from the <root>/cime/scripts directory.
  • Run an individual test: To run an individual test, you need a test type (<TEST>) compset (<COMPSET>), a model grid (<GRID>), a machine (<MACHINE>) and a compiler (<COMPILER>). The create_test command will look like this:
% ./create_test <TEST>.<GRID>.<COMPSET>.<MACHINE>_<COMPILER>

If you leave out the last part (<MACHINE>_<COMPILER>), CIME will choose a default compiler for the machine on which you are testing.

A CAM test example:

% ./create_test SMS_D_Ln9.f19_f19_mg17.FW4madSD.hobart_nag.cam-outfrq9s

where the last field is the particular configuration and run time modifications for this test. See the CIME manual section on testing for more details.

  • Run the aux_cam test suite: The aux_cam suite contains several tests which cover a variety of CAM simulation types. To run it, use this form of create_test:
% ./create_test --xml-category aux_cam --xml-compiler nag --test-id foo --output-root <scratch_dir> --test-root <scratch_dir>
  • To see the available tests:
% ./query_testlists

Use the --help option for details.

Finally, if running tests on Cheyenne, make sure to use the qcmd command to avoid running the test on the log-in nodes, like so:

% qcmd -- ./create_test <test_args>

where <test_args> are all the different create_test input arguments and flags described above.

  1. If you will submit your change as an update to ESCOMP/CAM, be sure an issue is open which describes the need for your change. If an issue does not exist, consult with an AMP scientist or software engineer so that an appropriate issue is opened. Update the issue and indicate that your code changes are ready to be added to CAM development. On the top of the far right box on github, is an assignee. Add the CAM SE in charge of scheduling (currently Cheryl Craig) to the issue. This step is very important as it will be the handoff of your issue to one of the CAM SEs for it to be added to the integration schedule. To see the process that happens after you issue the request you may folow your issue on the ESCOMP/CAM project board

How to remove old or unused branches

Once your modifications have been merged into the official CAM repo, you may have no more use for the local fork branch created to develop those modifications. In that case, you can remove the branch both from your local cloned repo and your GitHub CAM fork:

  1. First, make sure your local repo isn't checking out the old branch, by simply checking out a different branch:
    git checkout <some_other_branch>
  1. Then, remove branch from local repo:
    git branch -d <branch_name>
  1. Finally, remove branch from personal fork repo:
    git push --delete <origin> <branch_name>

You can also remove the branch via GitHub's user interface.

CAM wiki

CAM Documentation

CAM Model Development

Clone this wiki locally