-
Notifications
You must be signed in to change notification settings - Fork 207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add flexible controls of GPU configuration #4396
Conversation
…/jedwards4b/cime/compare/28b7431..3f4b1ab modified: CIME/Tools/Makefile modified: CIME/XML/env_batch.py modified: CIME/XML/env_mach_specific.py modified: CIME/build.py modified: CIME/case/case.py modified: CIME/data/config/xml_schemas/config_machines.xsd modified: CIME/data/config/xml_schemas/env_mach_specific.xsd modified: CIME/scripts/create_newcase.py modified: CIME/test_scheduler.py modified: CIME/tests/test_unit_case.py
modified: CIME/XML/env_mach_pes.py modified: CIME/case/case.py modified: CIME/config.py modified: doc/source/users_guide/cime-customize.rst
modified: CIME/data/config/xml_schemas/config_machines.xsd
modified: CIME/data/config/xml_schemas/config_machines.xsd
modified: CIME/case/case.py
modified: CIME/case/case.py
modified: CIME/case/case.py
modified: CIME/case/case.py
modified: CIME/XML/env_mach_pes.py
modified: CIME/case/case.py
modified: CIME/XML/env_mach_pes.py modified: CIME/case/case.py
modified: CIME/case/case.py
modified: CIME/case/case.py
@jedwards4b will issue a separate PR to introduce these changes. |
I believe this works independently of the E3SM approach to define gpu-specific compilers? And I think we would handle a machine like derecho with separate machine entries for each node type. |
that's correct - we are departing from that approach. |
@sjsprecious It looks like you may need to add something to make sure this only happens when driver=nuopc |
Thanks @jedwards4b . Do you mean that we should only use these new GPU options for the NUOPC driver? Why can't we use it for MCT? |
If you want use them for MCT then you will need to make changes in |
Thanks @jedwards4b for your explanation. So what is the best way you think to handle the MCT case? Currently generating a case with MCT will fail due to the missing new XML variables as expected. Would you suggest bypassing those errors and continuing to build anyway (probably ignoring the given GPU options and always building a CPU case)? |
probably the easiest is to just add them to the mct config_component.xml file. |
Thanks @jedwards4b . I just opened a PR to introduce these new XML variables in CPL7 (ESCOMP/CESM_CPL7andDataComps#25). Could you please review and merge it if the PR looks good to you? |
@sjsprecious The workflow test for e3sm, mct above needs to pass. Remember that these variables are not defined for e3sm. |
Thanks @jedwards4b . Since I am not familiar with the E3SM build process, could you please let me know where I can add those new XML variables for the E3SM build workflow? |
This PR updates the configurations for the GPU offload based on the PRs in CMEPS (ESCOMP/CMEPS#363) and ccs_config_cesm (ESMCI/ccs_config_cesm#97).
An example to build a GPU case on Derecho with these new GPU options by using create_newcase looks:
./create_newcase --case /path_to_case_dir --mach derecho --compiler nvhpc --mpilib mpich --compset F2000dev --res f19_f19_mg17 --queue main --ngpus-per-node 4 --gpu-type a100 --gpu-offload openacc
An example to build a GPU case on Derecho with these new GPU options by using create_test looks:
./create_test ERP_Ln9_G4-a100-openacc.f19_f19_mg17.F2000dev.gust_nvhpc.cam-outfrq9s --test-root /path_to_case_dir --output-root /path_to_output -q main