Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add xesmf to the standard python environment #2075

Merged
merged 4 commits into from
Dec 12, 2023

Conversation

glemieux
Copy link
Collaborator

@glemieux glemieux commented Jul 27, 2023

Description of changes

In preparation for adding the fates luh2 data tooling to the ctsm5.2 makefile, this PR adds the xesmf package to the standard ctsm_pylib environment.

Specific notes

See glemieux/fates#25 (comment) for details about compatibility issues with numba, which is a requirement of xesmf.

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):

Are answers expected to change (and if so in what way)? N/A

Any User Interface Changes (namelist or namelist defaults changes)?

Testing performed, if any: test that conda install works.

This also caps the numba environment to avoid
a numba/numpy compatability issue that is
currently unaddressed by numba
@glemieux
Copy link
Collaborator Author

glemieux commented Jul 27, 2023

@ekluzek I drafted this so that I could share my branch and the change easily. I wasn't sure if you'd want to merge this locally to your #2008 branch in preparation for adding the fates luh2 datatool call in NGEET/fates#1032 to the Makefile.data or make this an actual PR that I would test and we would bring to master first.

@glemieux
Copy link
Collaborator Author

I ran the python tests against the updated standard conda environment using make all in the python directory. Everything looks good:

(ctsm_pylib) ✔  ~/ctsm/python [python/xesmf-add {glemieux/python/xesmf-add}|⚑ 14] 
11:21 $ make all
python3 ./run_ctsm_py_tests  --unit
.....................................................................................................................................................................
----------------------------------------------------------------------
Ran 165 tests in 0.870s

OK
python3 ./run_ctsm_py_tests  --sys
...............Setting resource.RLIMIT_STACK to -1 from (307200000, -1)
Done converting /glade/scratch/glemieux/tmpwmp957eb/scrip.nc
.
----------------------------------------------------------------------
Ran 16 tests in 11.545s

OK
pylint -j 4 --rcfile=ctsm/.pylintrc ctsm

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

black --check --config pyproject.toml . ../cime_config/SystemTests ../cime_config/buildlib ../cime_config/buildnml
All done! ✨ 🍰 ✨
75 files would be left unchanged.

Still waiting for the latest conda environment build to solve before testing that out.

@glemieux
Copy link
Collaborator Author

glemieux commented Jul 27, 2023

Running tests with the "latest" conda environment has issues with making black:

(ctsm_pylib) ✔  ~/ctsm/python [python/xesmf-add {glemieux/python/xesmf-add}|⚑ 14] 
14:26 $ make all
python3 ./run_ctsm_py_tests  --unit
.....................................................................................................................................................................
----------------------------------------------------------------------
Ran 165 tests in 2.253s

OK
python3 ./run_ctsm_py_tests  --sys
...............copying /glade/u/home/glemieux/ctsm/ccs_config/machines/Depends.intel -> /glade/scratch/glemieux/tmpocrq60_5
Setting resource.RLIMIT_STACK to -1 from (307200000, -1)
Done converting /glade/scratch/glemieux/tmpocrq60_5/scrip.nc
.
----------------------------------------------------------------------
Ran 16 tests in 84.212s

OK
pylint -j 4 --rcfile=ctsm/.pylintrc ctsm

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

black --check --config pyproject.toml . ../cime_config/SystemTests ../cime_config/buildlib ../cime_config/buildnml
would reformat ../cime_config/buildlib
would reformat /glade/u/home/glemieux/ctsm/python/ctsm/test/test_unit_utils_add_tag.py
would reformat /glade/u/home/glemieux/ctsm/python/ctsm/modify_input_files/modify_mesh_mask.py
would reformat ../cime_config/buildnml
would reformat /glade/u/home/glemieux/ctsm/python/ctsm/toolchain/ctsm_case.py
would reformat /glade/u/home/glemieux/ctsm/python/ctsm/modify_input_files/modify_fsurdat.py
would reformat /glade/u/home/glemieux/ctsm/python/ctsm/test/test_unit_run_sys_tests.py
would reformat /glade/u/home/glemieux/ctsm/python/ctsm/site_and_regional/single_point_case.py
would reformat /glade/u/home/glemieux/ctsm/python/six.py

Oh no! 💥 💔 💥
9 files would be reformatted, 66 files would be left unchanged.
Makefile:53: recipe for target 'black' failed
make: *** [black] Error 1

@ekluzek ekluzek marked this pull request as ready for review August 4, 2023 20:00
FATES API update to facilitate fates refactor

This updates a number of FATES type names and module use statements
which correspond with a refactoring effort that moves FATES
patches and cohorts into their own respective modules.

With the FATES update is a minor science update, so there are
changes to answers for FATES.

This also incorporates a minor update to a more recent version
of the ccs config external.
@negin513
Copy link
Contributor

I am happy to xesmf this rolling in as a dependecy. xesmf is an outstanding package, and the entire regrid functionality (aka toolchain) could have been easily developed with it for a pure Python framework without any need to call the fortran binaries from inside the python! 😉
However, there was strong opposition against using xesmf!
Happy to see the progress!

@slevis-lmwg slevis-lmwg merged commit f335221 into ESCOMP:master Dec 12, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants