Skip to content

Commit

Permalink
Adding a file (#359)
Browse files Browse the repository at this point in the history
  • Loading branch information
CPKalb authored Feb 13, 2024
1 parent 00b69d2 commit 563bcb4
Show file tree
Hide file tree
Showing 3 changed files with 221 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/Users_Guide/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ National Center for Atmospheric Research (NCAR) is sponsored by NSF.
vertical_interpolation
difficulty_index
aggregation
write_mpr
release-notes

**Indices and tables**
Expand Down
98 changes: 98 additions & 0 deletions docs/Users_Guide/write_mpr.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
**********************
Write MPR
**********************

Description
===========

This program writes data to an output file in MET’s Matched Pair (MPR) format. It
takes several inputs, which are described in the list below. The script will compute
the observation input and total number of observations. It will also check to see if
the output directory is present and will create that directory if it does not exist.

Example
=======

Examples for how to use this script can be found in the driver scripts of the use cases
listed below.

* `Stratosphere Polar <https://metplus.readthedocs.io/en/latest/generated/model_applications/s2s/UserScript_fcstGFS_obsERA_StratospherePolar.html#sphx-glr-generated-model-applications-s2s-userscript-fcstgfs-obsera-stratospherepolar-py>`_
* `Blocking <https://metplus.readthedocs.io/en/latest/generated/model_applications/s2s_mid_lat/UserScript_fcstGFS_obsERA_Blocking.html#sphx-glr-generated-model-applications-s2s-mid-lat-userscript-fcstgfs-obsera-blocking-py>`_
* `Weather Regime <https://metplus.readthedocs.io/en/latest/generated/model_applications/s2s_mid_lat/UserScript_fcstGFS_obsERA_WeatherRegime.html#sphx-glr-generated-model-applications-s2s-mid-lat-userscript-fcstgfs-obsera-weatherregime-py>`_

Information about Input Data
============================

At this time, all input arrays have to be one dimensional only and should be the same size.
The script does not make an attempt to check if input arrays are the same size. If any of
your input arrays are larger than the observation input array, the data will be chopped at
the length of the observation input. If an array is shorter than the observation input, the
program will error.

Currently, the the following variables cannot be set and will be output as NA: FCST_THRESH,
OBS_THRESH, COV_THRESH, ALPHA, OBS_QC, CLIMO_MEAN, CLIMO_STDEV, CLIMO_CDF. Additionally the
following variables also cannot be set and have default values: INTERP_MTHD = NEAREST,
INTERP_PNTS = 1, and OBTYPE = ADPUPA.

data_fcst: 1D array float
forecast data to write to MPR file
data_obs: 1D array float
observation data to write to MPR file
lats_in: 1D array float
data latitudes
lons_in: 1D array float
data longitudes
fcst_lead: 1D array string of format HHMMSS
forecast lead time
fcst_valid: 1D array string of format YYYYmmdd_HHMMSS
forecast valid time
obs_lead: 1D array string of format HHMMSS
observation lead time
obs_valid: 1D array string of format YYYYmmdd_HHMMSS
observation valid time
mod_name: string
output model name (the MODEL column in MET)
desc: string
output description (the DESC column in MET)
fcst_var: 1D array string
forecast variable name
fcst_unit: 1D array string
forecast variable units
fcst_lev: 1D array string
forecast variable level
obs_var: 1D array string
observation variable name
obs_unit: 1D array string
observation variable units
obs_lev: 1D array string
observation variable level
maskname: string
name of the verification masking region
obsslev: 1D array string
Pressure level of the observation in hPA or accumulation
interval in hours
outdir: string
Full path including where the output data should go
outfile_prefix: string
Prefix to use for the output filename. The time stamp will
be added in MET's format based off the first forecast time


Run from a python script
=========================

* Make sure you have these required Python packages:

* Python 3.7

* metcalcpy

* numpy

* os

.. code-block:: ini
write_mpr_file(data_fcst,data_obs,lats_in,lons_in,fcst_lead,fcst_valid,obs_lead,obs_valid,mod_name,desc,fcst_var,fcst_unit,fcst_lev,obs_var,obs_unit,obs_lev,maskname,obsslev,outdir,outfile_prefix)
The output fill be a .stat file located in outdir with data in `MET's Matched Pair Format <https://met.readthedocs.io/en/latest/Users_Guide/point-stat.html#id24>`_. The file will be labeled with outfile_prefix and then have lead time, valid YYYYMMDD, and valid HHMMSS stamped onto the file name.
122 changes: 122 additions & 0 deletions metcalcpy/util/write_mpr.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# ============================*
# ** Copyright UCAR (c) 2020
# ** University Corporation for Atmospheric Research (UCAR)
# ** National Center for Atmospheric Research (NCAR)
# ** Research Applications Lab (RAL)
# ** P.O.Box 3000, Boulder, Colorado, 80307-3000, USA
# ============================*
import os
import numpy as np


def write_mpr_file(data_fcst,data_obs,lats_in,lons_in,fcst_lead,fcst_valid,obs_lead,obs_valid,mod_name,desc,fcst_var,fcst_unit,fcst_lev,obs_var,obs_unit,obs_lev,maskname,obsslev,outdir,outfile_prefix):

"""
Function to write an output mpr file given a 1d array of observation and forecast data
Parameters:
----------
data_fcst: 1D array float
forecast data to write to MPR file
data_obs: 1D array float
observation data to write to MPR file
lats_in: 1D array float
data latitudes
lons_in: 1D array float
data longitudes
fcst_lead: 1D array string of format HHMMSS
forecast lead time
fcst_valid: 1D array string of format YYYYmmdd_HHMMSS
forecast valid time
obs_lead: 1D array string of format HHMMSS
observation lead time
obs_valid: 1D array string of format YYYYmmdd_HHMMSS
observation valid time
mod_name: string
output model name (the MODEL column in MET)
desc: string
output description (the DESC column in MET)
fcst_var: 1D array string
forecast variable name
fcst_unit: 1D array string
forecast variable units
fcst_lev: 1D array string
forecast variable level
obs_var: 1D array string
observation variable name
obs_unit: 1D array string
observation variable units
obs_lev: 1D array string
observation variable level
maskname: string
name of the verification masking region
obsslev: 1D array string
Pressure level of the observation in hPA or accumulation
interval in hours
outdir: string
Full path including where the output data should go
outfile_prefix: string
Prefix to use for the output filename. The time stamp will
be added in MET's format based off the first forecast time
"""

"""
Get the data length to create the INDEX and TOTAL variables in the MPR line
"""
dlength = len(data_obs)
index_num = np.arange(0,dlength,1)+1

"""
Get the length of the model, FCST_VAR, FCST_LEV, OBS_VAR, OBS_LEV, VX_MASK, etc for formatting
"""
mname_len = str(max([5,len(mod_name)])+3)
desc_len = str(max([4,len(desc)])+3)
mask_len = str(max([7,len(maskname)])+3)
fvar_len = str(max([8,max([len(l) for l in fcst_var])])+3)
funit_len = str(max([8,max([len(l) for l in fcst_unit])])+3)
flev_len = str(max([8,max([len(l) for l in fcst_lev])])+3)
ovar_len = str(max([7,max([len(l) for l in obs_var])])+3)
ounit_len = str(max([8,max([len(l) for l in obs_unit])])+3)
olev_len = str(max([7,max([len(l) for l in obs_lev])])+3)

"""
Set up format strings for the header (format_string) and data (format_string2)
"""
format_string = '%-7s %-'+mname_len+'s %-'+desc_len+'s %-12s %-18s %-18s %-12s %-17s %-17s %-'+fvar_len+'s ' \
'%-'+funit_len+'s %-'+flev_len+'s %-'+ovar_len+'s %-'+ounit_len+'s %-'+olev_len+'s %-10s %-'+mask_len+'s ' \
'%-13s %-13s %-13s %-13s %-13s %-13s %-9s\n'
format_string2 = '%-7s %-'+mname_len+'s %-'+desc_len+'s %-12s %-18s %-18s %-12s %-17s %-17s %-'+fvar_len+'s ' \
'%-'+funit_len+'s %-'+flev_len+'s %-'+ovar_len+'s %-'+ounit_len+'s %-'+olev_len+'s %-10s %-'+mask_len+'s ' \
'%-13s %-13s %-13s %-13s %-13s %-13s %-9s %-10s %-10s %-10s %-12.4f %-12.4f %-10s %-10s %-12.4f %-12.4f ' \
'%-10s %-10s %-10s %-10s\n'

"""
Create the output directory if it doesn't exist
"""
if not os.path.exists(outdir):
os.makedirs(outdir)

"""
Put the timestamp on the output file
"""
fcst_valid_str = fcst_valid[0]
ft_stamp = fcst_lead[0]+'L_'+fcst_valid_str[0:8]+'_'+fcst_valid_str[9:15]+'V'
full_outfile = os.path.join(outdir,outfile_prefix+'_'+ft_stamp+'.stat')

"""
Write the file
"""
#print('Writing output MPR file: '+full_outfile)
with open(full_outfile, 'w') as mf:
# Write the header
mf.write(format_string % ('VERSION', 'MODEL', 'DESC', 'FCST_LEAD', 'FCST_VALID_BEG', 'FCST_VALID_END',
'OBS_LEAD', 'OBS_VALID_BEG', 'OBS_VALID_END', 'FCST_VAR', 'FCST_UNITS', 'FCST_LEV', 'OBS_VAR',
'OBS_UNITS', 'OBS_LEV', 'OBTYPE', 'VX_MASK', 'INTERP_MTHD', 'INTERP_PNTS', 'FCST_THRESH',
'OBS_THRESH', 'COV_THRESH', 'ALPHA', 'LINE_TYPE'))
for dpt in range(dlength):
# Write the data
mf.write(format_string2 % ('V9.1',mod_name,desc,fcst_lead[dpt],fcst_valid[dpt],fcst_valid[dpt],
obs_lead[dpt],obs_valid[dpt],obs_valid[dpt],fcst_var[dpt],fcst_unit[dpt],fcst_lev[dpt],
obs_var[dpt],obs_unit[dpt],obs_lev[dpt],'ADPUPA',maskname,'NEAREST','1','NA','NA','NA','NA','MPR',
str(dlength),str(index_num[dpt]),'NA',lats_in[dpt],lons_in[dpt],obsslev[dpt],'NA',data_fcst[dpt],
data_obs[dpt],'NA','NA','NA','NA'))

0 comments on commit 563bcb4

Please sign in to comment.