Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add GEFS C48 support on AWS #2818

Merged
merged 80 commits into from
Aug 26, 2024
Merged

Add GEFS C48 support on AWS #2818

merged 80 commits into from
Aug 26, 2024

Commits on Jun 18, 2024

  1. compiled OK now

    @weihuang-jedi
    weihuang-jedi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    103f2c4 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jun 19, 2024
    Configuration menu
    Copy the full SHA
    916ff6c View commit details
    Browse the repository at this point in the history

  2. re-test on aws with fewer changes

    @weihuang-jedi
    weihuang-jedi committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    b0ac406 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. make change in tasks.py to avoid error finding libiomp5.so problem

    @weihuang-jedi
    weihuang-jedi committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    3de972f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jun 21, 2024
    Configuration menu
    Copy the full SHA
    8308375 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. add comments so the reviewers know that these changes are for AWS, an… 

    …d hope can provide a better way
    @weihuang-jedi
    weihuang-jedi committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    bc4c4a8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'aws-forecast-only' of ssh://github.com/NOAA-EPIC/global… 

    …-workflow-cloud into aws-forecast-only
    @weihuang-jedi
    weihuang-jedi committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    924aede View commit details
    Browse the repository at this point in the history

  3. add comments so the reviewers know that these changes are for AWS, an… 

    …d hope can provide a better way
    @weihuang-jedi
    weihuang-jedi committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    b724937 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. reverse config.resource changes, and memory restriction on AWS

    @weihuang-jedi
    weihuang-jedi committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    12ab29f View commit details
    Browse the repository at this point in the history

  2. sync with emc repo

    @weihuang-jedi
    weihuang-jedi committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    adff250 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. move common data to a shared place

    @weihuang-jedi
    weihuang-jedi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    2290ea2 View commit details
    Browse the repository at this point in the history

  2. use ICs from s3-bucket

    @weihuang-jedi
    weihuang-jedi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    cd2c8e7 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    4e144e5 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. change as suggested by reviewer

    @weihuang-jedi
    weihuang-jedi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    46e3ef5 View commit details
    Browse the repository at this point in the history

  2. sync with develop

    @weihuang-jedi
    weihuang-jedi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    32f13eb View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. sync sorc/ufs_model.fd

    @weihuang-jedi
    weihuang-jedi committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a34a4c8 View commit details
    Browse the repository at this point in the history

  2. remove mpmd_opt from APRUN_UFS

    @weihuang-jedi
    weihuang-jedi committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    44011a3 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. mpmd_opt and switch off tracker/genesis default for AWS

    @weihuang-jedi
    weihuang-jedi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    965ec80 View commit details
    Browse the repository at this point in the history

  2. add TODO

    @weihuang-jedi
    weihuang-jedi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    3ce268e View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. remove ncl version on AWS

    @weihuang-jedi
    weihuang-jedi committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    f03ac78 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    007a56b View commit details
    Browse the repository at this point in the history

  3. sync ufs_model

    @weihuang-jedi
    weihuang-jedi committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    2f6ec6e View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 10, 2024
    Configuration menu
    Copy the full SHA
    dba83a7 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 12, 2024
    Configuration menu
    Copy the full SHA
    24fe804 View commit details
    Browse the repository at this point in the history

  2. sync and remove gempak from noaacloud

    @weihuang-jedi
    weihuang-jedi committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    e8a2e0f View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 15, 2024
    Configuration menu
    Copy the full SHA
    4013eb1 View commit details
    Browse the repository at this point in the history

  2. update modules hash

    @weihuang-jedi
    weihuang-jedi committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    a548c7f View commit details
    Browse the repository at this point in the history

  3. update module hash

    @weihuang-jedi
    weihuang-jedi committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    d37e646 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. use bucket

    @weihuang-jedi
    weihuang-jedi committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2a80162 View commit details
    Browse the repository at this point in the history

  2. remove /scratch1, but kept TODO

    @weihuang-jedi
    weihuang-jedi committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    fa44862 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 17, 2024
    Configuration menu
    Copy the full SHA
    55c7e7e View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. re-sync

    @weihuang-jedi
    weihuang-jedi committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    07851dc View commit details
    Browse the repository at this point in the history

  2. sync

    @weihuang-jedi
    weihuang-jedi committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    492808d View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. add is_exclusive to resource.AWSPW

    @weihuang-jedi
    weihuang-jedi committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    d7a262e View commit details
    Browse the repository at this point in the history

  2. sync hash with EMC repo

    @weihuang-jedi
    weihuang-jedi committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    af573af View commit details
    Browse the repository at this point in the history

  3. remove --export=ALL from native, when is_exclusive set true

    @weihuang-jedi
    weihuang-jedi committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    0929180 View commit details
    Browse the repository at this point in the history

  4. sync

    @weihuang-jedi
    weihuang-jedi committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    06fecca View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d8783ab View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d22bc6d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'aws-forecast-only' of ssh://github.com/NOAA-EPIC/global… 

    …-workflow-cloud into aws-forecast-only
    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    a5c441f View commit details
    Browse the repository at this point in the history

  4. Make AWS works similar to on-prem machine

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    77e8233 View commit details
    Browse the repository at this point in the history

  5. remove --export=ALL from 'native'

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    96f73ba View commit details
    Browse the repository at this point in the history

  6. remove --export=ALL from 'native'

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    a33a3be View commit details
    Browse the repository at this point in the history

  7. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 25, 2024
    Configuration menu
    Copy the full SHA
    80b294b View commit details
    Browse the repository at this point in the history

  8. add py-f90nml to noaacloud modulefile

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    01a8928 View commit details
    Browse the repository at this point in the history

  9. remove un-necessary added lines

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    b035947 View commit details
    Browse the repository at this point in the history

  10. remove un-necessary added lines

    @weihuang-jedi
    weihuang-jedi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    bf3b460 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. remove added lines which was originally for AWS, but should be define… 

    …d in hosts/awspw.yaml
    @weihuang-jedi
    weihuang-jedi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    47627ff View commit details
    Browse the repository at this point in the history

  2. restore as develop

    @weihuang-jedi
    weihuang-jedi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    7bf8900 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. try to fix pynorms error

    @weihuang-jedi
    weihuang-jedi committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    0685a8f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 29, 2024
    Configuration menu
    Copy the full SHA
    381403d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'aws-forecast-only' of ssh://github.com/NOAA-EPIC/global… 

    …-workflow-cloud into aws-forecast-only
    @weihuang-jedi
    weihuang-jedi committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    0e71f7d View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    2024835 View commit details
    Browse the repository at this point in the history

  2. sync with EMC repo

    @weihuang-jedi
    weihuang-jedi committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    2c52016 View commit details
    Browse the repository at this point in the history

  3. sync Gaea link with EMC repo, and only include blocks/packs that run … 

    …on AWS
    @weihuang-jedi
    weihuang-jedi committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    cd6c541 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'aws-forecast-only' of github.com:NOAA-EPIC/global-workf… 

    …low-cloud into aws-forecast-only
    @weihuang-jedi
    weihuang-jedi committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    1f60ed0 View commit details
    Browse the repository at this point in the history

  5. merge fro develop

    @weihuang-jedi
    weihuang-jedi committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    e1a57b4 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Remove ACCOUNT_SERVICE

    @weihuang-jedi
    weihuang-jedi committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    fe9a457 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-forecast-only

    @weihuang-jedi
    weihuang-jedi authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    5c6e052 View commit details
    Browse the repository at this point in the history

  2. run gefs

    @weihuang-jedi
    weihuang-jedi committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    6a4bada View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    e802ba0 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi authored Aug 8, 2024
    Configuration menu
    Copy the full SHA
    31e3b2b View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Merge remote-tracking branch 'origin/develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    868b671 View commit details
    Browse the repository at this point in the history

  2. sync after AWS code merged

    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    de55cc3 View commit details
    Browse the repository at this point in the history

  3. make gefs run on AWS

    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    f5a03d4 View commit details
    Browse the repository at this point in the history

  4. test in gefs C48 case

    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    e606fc7 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    1aff5c3 View commit details
    Browse the repository at this point in the history

  6. fix pynorms error

    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    8b3ad61 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'aws-gefs-2' of github.com:NOAA-EPIC/global-workflow-clo… 

    …ud into aws-gefs-2
    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    1c7e45a View commit details
    Browse the repository at this point in the history

  8. fix pynorms error

    @weihuang-jedi
    weihuang-jedi committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    73050a8 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. use 'unset memory' in resource.AWSPW, instead of change it in rocoto.py

    @weihuang-jedi
    weihuang-jedi committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    10d4104 View commit details
    Browse the repository at this point in the history

  2. revert hash

    @weihuang-jedi
    weihuang-jedi committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b3de7c1 View commit details
    Browse the repository at this point in the history

  3. remove export APRUN="${APRUN}"

    @weihuang-jedi
    weihuang-jedi committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    16770f8 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. revert sed separator change for now

    @weihuang-jedi
    weihuang-jedi committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    b9416fa View commit details
    Browse the repository at this point in the history

  2. revert sed separator change for now, more revert needed

    @weihuang-jedi
    weihuang-jedi committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    456fa0c View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi authored Aug 20, 2024
    Configuration menu
    Copy the full SHA
    a8576c2 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. turn off wave on AWS

    @weihuang-jedi
    weihuang-jedi committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    35ae83e View commit details
    Browse the repository at this point in the history

  2. sync with develop

    @weihuang-jedi
    weihuang-jedi committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    d059c70 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Merge branch 'NOAA-EMC:develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi authored Aug 23, 2024
    Configuration menu
    Copy the full SHA
    26ccc47 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'NOAA-EMC:develop' into aws-gefs-2

    @weihuang-jedi
    weihuang-jedi authored Aug 23, 2024
    Configuration menu
    Copy the full SHA
    b84c486 View commit details
    Browse the repository at this point in the history