Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove gldas from everywhere in the global-workflow #1590

Merged
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions Externals.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,6 @@ repo_url = https://github.com/NOAA-EMC/GDASApp.git
protocol = git
required = False

[GLDAS]
tag = fd8ba62
local_path = sorc/gldas.fd
repo_url = https://github.com/NOAA-EMC/GLDAS.git
protocol = git
required = False

[EMC-gfs_wafs]
hash = 014a0b8
local_path = sorc/gfs_wafs.fd
Expand Down
15 changes: 7 additions & 8 deletions docs/source/components.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
###########################
Global Workflow Components
Global Workflow Components
###########################

The global-workflow is a combination of several components working together to prepare, analyze, produce, and post-process forecast data.
Expand All @@ -13,20 +13,19 @@ The major components of the system are:
* Post-processing
* Verification

The Global Workflow repository contains the workflow and script layers. After running the checkout script, the code and additional offline scripts for the analysis, forecast, and post-processing components will be present. Any non-workflow component is known as a sub-module. All of the sub-modules of the system reside in their respective repositories on GitHub. The global-workflow sub-modules are obtained by running the checkout script found under the /sorc folder.
The Global Workflow repository contains the workflow and script layers. After running the checkout script, the code and additional offline scripts for the analysis, forecast, and post-processing components will be present. Any non-workflow component is known as a sub-module. All of the sub-modules of the system reside in their respective repositories on GitHub. The global-workflow sub-modules are obtained by running the checkout script found under the /sorc folder.

======================
Component repositories
======================

Components checked out via sorc/checkout.sh:

* **GFS UTILS** (https://github.com/ufs-community/gfs_utils): Utility codes needed by Global Workflow to run the GFS configuration
* **UFS-Weather-Model** (https://github.com/ufs-community/ufs-weather-model): This is the core model used by the Global-Workflow to provide forecasts. The UFS-weather-model repository is an umbrella repository consisting of cooupled component earth systeme that are all checked out when we check out the code at the top level of the repoitory
* **GFS UTILS** (https://github.com/ufs-community/gfs_utils): Utility codes needed by Global Workflow to run the GFS configuration
* **UFS-Weather-Model** (https://github.com/ufs-community/ufs-weather-model): This is the core model used by the Global-Workflow to provide forecasts. The UFS-weather-model repository is an umbrella repository consisting of cooupled component earth systeme that are all checked out when we check out the code at the top level of the repoitory
* **GSI** (https://github.com/NOAA-EMC/GSI): This is the core code base for atmospheric Data Assimilation
* **GSI UTILS** (https://github.com/NOAA-EMC/GSI-Utils): Utility codes needed by GSI to create analysis
* **GSI Monitor** (https://github.com/NOAA-EMC/GSI-Monitor): These tools monitor the GSI package's data assimilation, detecting and reporting missing data sources, low observation counts, and high penalty values
* **GLDAS** (https://github.com/NOAA-EMC/GLDAS): Code base for Land Data Assimiation
* **GSI UTILS** (https://github.com/NOAA-EMC/GSI-Utils): Utility codes needed by GSI to create analysis
* **GSI Monitor** (https://github.com/NOAA-EMC/GSI-Monitor): These tools monitor the GSI package's data assimilation, detecting and reporting missing data sources, low observation counts, and high penalty values
* **GDAS** (https://github.com/NOAA-EMC/GDASApp): Jedi based Data Assimilation system. This system is currently being developed for marine Data Assimilation and in time will replace GSI for atmospheric data assimilation as well
* **UFS UTILS** (https://github.com/ufs-community/UFS_UTILS): Utility codes needed for UFS-weather-model
* **Verif global** (https://github.com/NOAA-EMC/EMC_verif-global): Verification package to evaluate GFS parallels. It uses MET and METplus. At this moment the verification package is limited to providing atmospheric metrics only
Expand All @@ -43,7 +42,7 @@ External dependencies
Libraries
^^^^^^^^^

All the libraries that are needed to run the end to end Global Workflow are built using a package manager. Currently these are served via HPC-STACK but will soon be available via SPACK-STACK. These libraries are already available on supported NOAA HPC platforms
All the libraries that are needed to run the end to end Global Workflow are built using a package manager. Currently these are served via HPC-STACK but will soon be available via SPACK-STACK. These libraries are already available on supported NOAA HPC platforms

Find information on official installations of HPC-STACK here:

Expand Down
5 changes: 1 addition & 4 deletions docs/source/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The global-workflow configs contain switches that change how the system runs. Ma
| APP | Model application | ATM | YES | See case block in config.base for options |
+----------------+------------------------------+---------------+-------------+---------------------------------------------------+
| DOIAU | Enable 4DIAU for control | YES | NO | Turned off for cold-start first half cycle |
| | with 3 increments | | | |
| | with 3 increments | | | |
+----------------+------------------------------+---------------+-------------+---------------------------------------------------+
| DOHYBVAR | Run EnKF | YES | YES | Don't recommend turning off |
+----------------+------------------------------+---------------+-------------+---------------------------------------------------+
Expand All @@ -26,9 +26,6 @@ The global-workflow configs contain switches that change how the system runs. Ma
| DO_GEMPAK | Run job to produce GEMPAK | NO | YES | downstream processing, ops only |
| | products | | | |
+----------------+------------------------------+---------------+-------------+---------------------------------------------------+
| DO_GLDAS | Run GLDAS to spin up land | YES | YES | Spins up for 84hrs if sflux files not available |
| | ICs | | | |
+----------------+------------------------------+---------------+-------------+---------------------------------------------------+
| DO_VRFY | Run vrfy job | NO | YES | Whether to include vrfy job (GSI monitoring, |
| | | | | tracker, VSDB, fit2obs) |
+----------------+------------------------------+---------------+-------------+---------------------------------------------------+
Expand Down
8 changes: 3 additions & 5 deletions docs/source/jobs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ GFS Configuration
#################

.. figure:: _static/GFS_v16_flowchart.png

Schematic flow chart for GFS v16 in operations

The sequence of jobs that are run for an end-to-end (analysis+forecast+post processing+verification) GFS configuration using the Global Workflow is shown above. The system utilizes a collection of scripts that perform the tasks for each step.
Expand All @@ -12,7 +12,7 @@ For any cycle the system consists of two suites -- the "gdas" suite which provid

An experimental run is different from operations in the following ways:

* Workflow manager: operations utilizes `ecFlow <https://www.ecmwf.int/en/learning/training/introduction-ecmwf-job-scheduler-ecflow>`__, while development currently utilizes `ROCOTO <https://github.com/christopherwharrop/rocoto/wiki/documentation>`__. Note, experiments can also be run using ecFlow on platforms with ecFlow servers established.
* Workflow manager: operations utilizes `ecFlow <https://www.ecmwf.int/en/learning/training/introduction-ecmwf-job-scheduler-ecflow>`__, while development currently utilizes `ROCOTO <https://github.com/christopherwharrop/rocoto/wiki/documentation>`__. Note, experiments can also be run using ecFlow on platforms with ecFlow servers established.

* Dump step is not run as it has already been completed during the real-time production runs and dump data is available in the global dump archive on supported machines.

Expand All @@ -25,7 +25,7 @@ An experimental run is different from operations in the following ways:
Downstream jobs (e.g. awips, gempak, etc.) are not included in the diagram. Those jobs are not normally run in developmental tests.

=============================
Jobs in the GFS Configuration
Jobs in the GFS Configuration
=============================
+-------------------+-----------------------------------------------------------------------------------------------------------------------+
| JOB NAME | PURPOSE |
Expand Down Expand Up @@ -65,8 +65,6 @@ Jobs in the GFS Configuration
+-------------------+-----------------------------------------------------------------------------------------------------------------------+
| fcst | Runs the forecast (with or without one-way waves). |
+-------------------+-----------------------------------------------------------------------------------------------------------------------+
| gldas | Runs the Global Land Data Assimilation System (GLDAS). |
+-------------------+-----------------------------------------------------------------------------------------------------------------------+
| metpN | Runs MET/METplus verification via EMC_verif-global. |
+-------------------+-----------------------------------------------------------------------------------------------------------------------+
| prep | Runs the data preprocessing prior to the analysis (storm relocation if needed and generation of prepbufr file). |
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/gfs_00.def
Original file line number Diff line number Diff line change
Expand Up @@ -2224,10 +2224,6 @@
trigger /prod/primary/00/obsproc/v1.0/gdas/atmos/dump/jobsproc_gdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger /prod/primary/00/obsproc/v1.0/gdas/atmos/prep/jobsproc_gdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2354,7 +2350,7 @@
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit RUN 'gdas'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/gfs_06.def
Original file line number Diff line number Diff line change
Expand Up @@ -2224,10 +2224,6 @@
trigger /prod/primary/06/obsproc/v1.0/gdas/atmos/dump/jobsproc_gdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger /prod/primary/06/obsproc/v1.0/gdas/atmos/prep/jobsproc_gdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2354,7 +2350,7 @@
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit RUN 'gdas'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/gfs_12.def
Original file line number Diff line number Diff line change
Expand Up @@ -2225,10 +2225,6 @@
trigger /prod/primary/12/obsproc/v1.0/gdas/atmos/dump/jobsproc_gdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger /prod/primary/12/obsproc/v1.0/gdas/atmos/prep/jobsproc_gdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2355,7 +2351,7 @@
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit RUN 'gdas'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/gfs_18.def
Original file line number Diff line number Diff line change
Expand Up @@ -2224,10 +2224,6 @@
trigger /prod/primary/18/obsproc/v1.0/gdas/atmos/dump/jobsproc_gdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger /prod/primary/18/obsproc/v1.0/gdas/atmos/prep/jobsproc_gdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2354,7 +2350,7 @@
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit RUN 'gdas'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/prod00.def
Original file line number Diff line number Diff line change
Expand Up @@ -2259,10 +2259,6 @@ suite prod00
trigger ../dump/jgdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger ../obsproc/prep/jgdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2389,7 +2385,7 @@ suite prod00
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit CYC '00'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/prod06.def
Original file line number Diff line number Diff line change
Expand Up @@ -2262,10 +2262,6 @@ suite prod06
trigger ../dump/jgdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger ../obsproc/prep/jgdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2392,7 +2388,7 @@ suite prod06
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit CYC '06'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/prod12.def
Original file line number Diff line number Diff line change
Expand Up @@ -2259,10 +2259,6 @@ suite prod12
trigger ../dump/jgdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger ../obsproc/prep/jgdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2389,7 +2385,7 @@ suite prod12
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit CYC '12'
Expand Down
6 changes: 1 addition & 5 deletions ecf/defs/prod18.def
Original file line number Diff line number Diff line change
Expand Up @@ -2259,10 +2259,6 @@ suite prod18
trigger ../dump/jgdas_atmos_dump:release_sfcprep
endfamily
endfamily
family init
task jgdas_atmos_gldas
trigger ../analysis/jgdas_atmos_analysis == complete
endfamily
family analysis
task jgdas_atmos_analysis
trigger ../obsproc/prep/jgdas_atmos_prep == complete and ../obsproc/prep/jgdas_atmos_emcsfc_sfc_prep == complete
Expand Down Expand Up @@ -2389,7 +2385,7 @@ suite prod18
endfamily
endfamily
task jgdas_forecast
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete and ./atmos/init/jgdas_atmos_gldas == complete
trigger ./atmos/analysis/jgdas_atmos_analysis:release_fcst and ./wave/prep/jgdas_wave_prep == complete
endfamily
family enkfgdas
edit CYC '18'
Expand Down
69 changes: 0 additions & 69 deletions ecf/scripts/gdas/atmos/init/jgdas_atmos_gldas.ecf

This file was deleted.

Loading