Nudging script

[water-e]

Workflow for nudging

SAL_nu.nc
This format is recently changed and not well documented, although I think it is simple in structure. There is a file called gen_nudge_from_hycom.f90 in the schism repo that is based on the latest format. It mentions a file called include.gr3 that allows you to specify a subset of the domain to include/exlude from the nc file for nudging. This should match the computational grid as far as nodes, and has a value ("depth" in docs) of zero where nudging is to be skipped. I would recommend we set this to one everywhere, because we have to deal with the ocean nudging as described below. Zeroing out the rest of the ocean and bay can be done with weights.

Steps:

1. Download ec and/or temp for stations shown in list (enhancement: coordinate with stations_utm.csv in Modeling_Data/station_inventories)
2. Run data through error detection, median_outliers in error_detect2.py in pyschism should be good enough -- I haven't used this, but based on what Shaurya showed me it is fairly effective. The process just scrubs bad data so the product of QA/QC will be a gappy series. We won't deal with the gaps initially -- we will just skip them in the interpolation. If there are few missing data, it would be reasonable to low pass them to smooth out noise (hourly average). The full output of this stage is a matrix of size NT x NX, where NT is the number of hourly values and NX is the number of observation stations.
3. Read the simulation mesh from its gr3 (read_mesh) and output the locaitons of nodes (mesh.nodes). Write to a file. These are the places TO which we will interpolate.
4. In R, run bay_delta_interpolate.r using bay_delta_coarse_v1.gr3 as the interpolation mesh and the fine simulation nodes as the interpolation points. This script involves these steps:
   a. Read and set up the FEM basis once and for all which is based on the coarse interpolation mesh. This takes time so don't put it in the time loop.
   b. Loop through the hours of the data described in (2), possibly filter for NA and interpolate from the x,y points where we have data to the centroids. The result of this is an NT x NP block, where NP is big (250000) and NT is 24x7 for one week
5. Output the interpolated data in _nu.nc format.

*** SAL_nu.gr3
6. In a separate step, create a weighting file (SAL_nu.gr3) where the weight exponentially decays with the distance to the nearest station (5-10km is a good decay rate ... we do want dead end sloughs to get information). Note that cdist in scipy is good for calculating distances from NX0 to NX1 points. The simulation mesh can be read with read_mesh and written (with new values and filename) with write_mesh. Locations are in mesh.nodes. The ocean (except for 7 below) bay and deep part of Suisun/Grizzly Bay should be skipped.

7. We need to reconcile the SAL_nu.gr3 with the one we currently generate in yaml which nudges in a band around the boundary. See file snippets below. The temperature nudging is more complicated than salinity because it comes from the ROMS near-coast model. Overall, I would say skip that and start with salt. Kijin and Joseph know more about the details

=========== TEM_nudge.yaml


default: 0.
polygons:
  - name: ocean
    type: none
    attribute: max(0., min(1. / 2. / 86400., 1. / 2. / 86400. * (((x - 542699.) ** 2 / 32.3e3 / 32.3e3 + (y - 4183642.) **2 / 32.3e3 / 32.3e3) - 1.) / ((41.e3 / 32.3e3) ** 2 - 1.)))
    vertices:
       - [497085, 4213519]
       - [531756, 4214317]
       - [550885, 4170246]
       - [552280, 4136745]
       - [496487, 4137742]




=========== SAL_nudge.yaml

default: 0.
polygons:
  - name: ocean
    type: none
    attribute: max(0., min(1. / 2. / 86400., 1. / 2. / 86400. * (((x - 542699.) ** 2 / 32.3e3 / 32.3e3 + (y - 4183642.) **2 / 32.3e3 / 32.3e3) - 1.) / ((41.e3 / 32.3e3) ** 2 - 1.)))
    vertices:
       - [497085, 4213519]
       - [531756, 4214317]
       - [550885, 4170246]
       - [552280, 4136745]
       - [496487, 4137742]
  - name: sugar_cut
    attribute: 0.000138888
    vertices:
       - [638917.549493, 4184801.771746]
       - [639171.479853, 4185291.948535]
       - [639695.723177, 4185447.171184]
       - [640727.827220, 4185357.305440]
       - [641342.174865, 4185455.340798]
       - [641874.609491, 4185651.411513]
       - [644561.356525, 4184997.842462]
       - [646568.225498, 4183404.767898]
       - [646478.121177, 4180815.000532]
       - [639212.436363, 4180790.491692]
       - [638483.410491, 4183086.152986]