-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/remove shell workflow #380
Feature/remove shell workflow #380
Conversation
…community#756) ## DESCRIPTION OF CHANGES: 1) Adjust y-direction size of write-component grid of `SUBCONUS_Ind_3km` predefined grid from 195 to 197 (this was just an oversight in PR ufs-community#725 ). 2) Redirect output of module load in launch script (`launch_FV3LAM_wflow.sh`) to `/dev/null` to avoid unwanted screen output (which was introduced in PR #[238](ufs-community/ufs-srweather-app#238) in ufs-srweather-app and is about how to load the `regional_workflow` environment and is not relevant in this context). ## TESTS CONDUCTED: 1) Plotted the `SUBCONUS_Ind_3km` grid to ensure it has correct size (it does). 2) Manually ran `launch_FV3LAM_wflow.sh` from the command line to verify that screen output is suppressed (it is).
@venitahagerty Christina suggested that we collaborate on this PR, in which I replace the shell workflow with python workflow generation code. In the process some shell scripts are removed. I have tried to make sure something used by the job scripts is not removed, but I may have missed something. Please let me know if you have any suggestions. @christinaholtNOAA The python unittest that I included in this PR seem to run fine here, so I guess the reason why it was not running on ufs-community repo is because it is a fork of fork of it, instead of a direct fork. |
@danielabdi-noaa Do you activate the conda environment regional_workflow before running generate_FV3LAM_wflow? The error I'm getting is that python modules jinja2, yaml, and f90nml are not available. |
Yes, that is correct. Those packages are only available under the |
@danielabdi-noaa I'm not seeing the command to activate regional_workflow in run_WE2E_tests.sh. In which shell is that environment activated? |
@venitahagerty When you load modules as follows the last step does the conda activation.
Then you can run the regression test. |
@danielabdi-noaa Sometimes the short range weather app will be run in an automated way. It would be useful if run_WE2E_tests.sh did the activation of the conda environment regional_workflow. Do you have any objections to adding that activation to that shell? |
@venitahagerty I think it is a good idea! The job launch script in |
3780563
to
ea76cc1
Compare
@venitahagerty Unfortunately activation of conda or other libraries needed for getting yaml,jinja etc is not uniform across platofrms. On cheyenne for example, once needs to activate ncarlib in a different way. Even for conda, the way it is activated on odin is different from other platforms, so it maybe best to leave it for the user. One potential solution is to print the exact commands needed to activate the needed environtment, when loading |
ea76cc1
to
3c4d048
Compare
…lues (ufs-community#759) * Modify RRFS North America 3- and 13-km domain configuration and WE2E test. * Modify default ISEED values for SPP * Fix grid in WE2E test
* update python scripts
…ity#758) * change output file name * change variable name * update python script * remove duplicates * add a check for empty variables * move variable to common area * clean up unnecessary comments * update scripts * remove duplicate * update python scripts * fix user-staged dir path issue in python script
3c4d048
to
0371505
Compare
…y#763) * Add new var to we2e tests for new grids * rename we2e tests for custom grid * remove unnecessary $
0371505
to
aab1006
Compare
I am going to turn this pull request from draft to ready since it looks like it is working reasonably well on three systems I tested so far. |
DESCRIPTION OF CHANGES:
This PR removes the shell workflow.
TESTS CONDUCTED:
I have run the tests successfully on HERA and ORION and JET. The successful tests are exactly the same a recent shell workflow generation run done in #744
Out of 56 tests
HERA
0 failures
ORION
1 failure
JET
5 failures (a lot of improvement since last time)
DEPENDENCIES:
None
DOCUMENTATION:
To be completed ...
ISSUE (optional):
None
CONTRIBUTORS (optional):
@christinaholtNOAA @venitahagerty