Skip to content

Development Workflow

Jump to bottom Edit New page
Robert Jacob edited this page Jun 27, 2023 · 11 revisions

E3SM Development Workflow

(originally written by Jed Brown. Modified by Rob Jacob)

For complete documentation, see https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/1868455/Development+Getting+Started+Guide. The below is a brief summary.

E3SM development workflow uses Git with topic branches and pull requests. If you are coming from Subversion, the most important differences are:

  • Git is distributed: All repositories are equivalent up to conventions. Committing is a local operation and exchanged via explicit push and pull commands.
  • Complete DAG history: The history is a directed acyclic graph (DAG). No information is lost in merges and the tooling offers advanced query and manipulation. Git commands are basically commands for manipulating the DAG.

Setup

  • Set your name: git config --global user.name "Bobby Tables"
  • Set your email: git config --global user.email "me@example.com"
  • Do not push local branches nonexistent on upstream by default: git config --global push.default simple (older versions of git require git config --global push.default nothing)
  • Download and source git-prompt to add branch information to your bash prompt (optional)
  • Download and source git-completion.bash to enable tab completion of git commands (optional)

Using Forks

All development should be done on a github fork of the main E3SM repo. A github fork is just a git clone managed within github. Make a fork of the main E3SM repo and then make a local clone of your fork on your development platform. You can then make branches and push commits to your fork. When you are done, issue a pull request from your fork back to the main E3SM repo.

Keeping your fork in sync

The master branch of your fork should always be a copy of the master branch in E3SM. Github provides a button for this.

Adding other users to your fork.

Often you would like a collaborator to be able to make commits to your feature branch on your fork. Your fork is a repo you own and you can add collaborators with write permission just like any other repo you own independent of the permissions on the main E3SM repo. Your fork does not inherit the read/write collaborators of the main repo. See https://docs.github.com/en/github/setting-up-and-managing-your-github-user-account/inviting-collaborators-to-a-personal-repository.

Note: since E3SM is a public repo, you can not make a fork private.

Letting integrators make changes to your PR

When you make a PR from your fork, the integrator may have to make changes. You don't have to make them a collaborator on your fork. Instead do that from within the PR by selecting "Allow Edits from Maintainers". See https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork

Developers

Developers work in topic branches and issue pull requests to an appropriate integration branch (master or a maint branch). After cloning the repository, the typical workflow is to start a new branch from 'master' (or a component integration branch, such as 'atm/master'):

$ git checkout -b yourgitusername/area-of-E3SM/short-description-of-feature master

Develop in your branch, test locally, and commit in logical, reviewable units with good commit messages. When you believe your branch to be complete and tested, push your new branch

$ git push -u origin yourname/short-description-of-feature

and create a pull request to a suitable integration branch (e.g., 'master') github repository.

Integrators

E3SM uses gitworkflows as described here: https://git-scm.com/docs/gitworkflows but without the "seen" branch.

Integration into master or maint branches is done by designated E3SM staff and follows the guide at https://acme-climate.atlassian.net/wiki/spaces/ED/pages/3245434/Integrator+Guide

But developers may want to integrate feature branches to each other and should follow this guide.

Every branch has a purpose. Merging into branch branch-A is a declaration that the purpose of branch-A is better served by including those commits that were in branch-B. This is achieved with the command

(branch-A) $ git merge branch-B

Topic branches do not normally contain merge commits, but it is acceptable to merge from master or from other topic branches if your topic depends on a feature or bug fix introduced there. When making such a merge, use the commit message to state the reason for the merge. Never merge next into your branch. Never merge master just to "keep up" or make the PR easier.

For further philosophy on merges, see

Day to day git use

Inspecting history

  • Show logs: git log
  • Show logs for file or folder: git log file
  • Show changes for each log: git log -p (add file or folder name if required)
  • Show diff:
    • Current working tree: git diff path/to/file
    • To other commit: git diff <SHA1> path/to/file
    • Compare version of file in two commits: git diff <SHA1> <SHA1> path/to/file
  • Show changes that are in master, but not yet in my current branch:
    • At any path: git log ..master
    • Only affecting a path: git log ..master src/dm/impls/plex/
    • In my branch, but not yet in next: git log next.. src/dm/
    • Tabulated by author: git shortlog v3.3..master src/dm/impls/plex
  • Showing branches:
    • Not yet stable: git branch --all --no-merged master
    • Being tested by early users: git branch --all --merged next
    • Will be in the next release: git branch --all --merged master
    • Remove --all to the above to not include remote tracking branches (work you have not interacted with yet).
  • Find where to fix a bug:
    • Find the bad line (e.g., using a debugger)
    • Find the commit that introduced it: git blame path/to/file
    • Find the branch containing that commit: git branch --contains COMMIT (usually one topic branch, plus next)
    • Fix bug: git checkout topic-branch-name, fix bug, git commit, and merge to next, etc.

Miscellaneous git

  • Discard changes to a file which are not yet committed: git checkout path/to/file
  • Discard all changes to the current working tree: git checkout -f
  • Forward-port local commits to the updated upstream head on master: git rebase master (on feature branch)
  • Delete local branch: git branch -D <branchname>
  • Delete remote branch: git push origin :<branchname> (only after successful integration into master)

Write good commit messages

ComponentName: one-line explanation of commit

After a blank line, write a more detailed explanation of the commit.
Many tools do not auto-wrap this part, so wrap paragraph text at a
reasonable length. Commit messages are meant for other people to read,
possibly months or years later, so describe the rationale for the change
in a manner that will make sense later.

The message should prominently state its impact on users, such as a
different interface or a change of bit-for-bit results.

If this affects any known issues, include "fix #ISSUENUMBER" or
"see #ISSUENUM" in the message (without quotes). GitHub will create
a link to the issue as well as a link from the issue to this commit,
notifying anyone that was watching the issue. Feel free to link to
mailing list discussions.

* Commit messages may contain lists.

* Following these guidelines improves the ability of tools to quickly
  summarize changes according to various criteria.

If other people contributed significantly to a commit, perhaps by
reporting bugs or by writing an initial version of the patch,
acknowledge them using tags at the end of the commit message.

Reported-by: Helpful User <helpful@example.com>
Based-on-patch-by: Original Idea <original@example.com>
Thanks-to: Incremental Improver <improver@example.com>

Further reading.

Merges

Merge commits should explain why the merge occurs. For topic branches being integrated, this is a short summary of the feature or bug fix. If upstream (like 'master') is merged into a topic branch, the commit message should explain what is needed. If you don't know what to write, perhaps the merge should not take place.

External Resources

Add a custom footer
Add a custom sidebar
Clone this wiki locally