Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error - max column in satinfo file for IR #174

Open
rtodling opened this issue Sep 25, 2024 · 9 comments
Open

Error - max column in satinfo file for IR #174

rtodling opened this issue Sep 25, 2024 · 9 comments
Assignees

Comments

@rtodling
Copy link
Contributor

rtodling commented Sep 25, 2024

I am puzzled about the error-max column in the sat-info file in GSI's recent versions; this is the 6th column in the file. Is it correct to set the max value to a value that is smaller than the ops-error itself (4th column)?

The corresponding JEDI yaml files have the same settings - consistently so (which is good) - but I wonder if GSI and JEDI do different things w/ these numbers (in case it is for some odd reason meaningful to see the error-max to a value larger than the error itself).

Can someone please appease my mind in that this is not an error - or a possible issue in:

a) GSI itself
b) JEDI doing things different than GSI

?

Thank you.

@gmao-wgu
Copy link
Contributor

gmao-wgu commented Sep 25, 2024

This column is used to provide the upper bound in the gross-check, that is, if abs(omf) > minium(3*obs error, err_max), the observation will be tossed away.

@gmao-wgu
Copy link
Contributor

Yes, JEDI is consistent with GSI regarding err_max, except for the q observations from the rawinsonde. There is no gross check in the YAML, which I believe Akira knows more about.

@rtodling
Copy link
Contributor Author

Yes, JEDI is consistent with GSI regarding err_max, except for the q observations from the rawinsonde. There is no gross check in the YAML, which I believe Akira knows more about.

pls, let's leave the handling of conventional obs for another discussion - I have seen the issue in swell and that's fine.

@rtodling
Copy link
Contributor Author

Yes, JEDI is consistent with GSI regarding err_max, except for the q observations from the rawinsonde. There is no gross check in the YAML, which I believe Akira knows more about.

This column is used to provide the upper bound in the gross-check, that is, if abs(omf) > minium(3*obs error, err_max), the observation will be tossed away.

right - which means that for all cases where err-max has changed recently, the criterium above is always going to be:

abs(omf) > err_max

and because err_max in the latest changes is typically smaller than 3*obs_err we are effectively letting more data into the analysis which have larger abs(omf) but still giving those data the same obs-err ... is this really what we want to do? It seems to me that if we are letting data with more scatter in (larger omf) we should down weight the data no?

@gmao-wgu
Copy link
Contributor

I am not sure what you mean. My understanding was that we only have a small number of high-peaking channels from IR(AIRS, IASI and CrIS-FSR) with decreased err_max values, which results in fewer observations being assimilated.

@rtodling
Copy link
Contributor Author

BTW: I just want to say that it looks to me like - for IR - the error bound check done in JEDI is not as simple as what you state above:

   out[ichan][iloc] = std::fmin(3.0 * obserr[ichan] 
                           * (1.0 / pow(errflat[0][iloc], 2))
                           * (1.0 / pow(errftaotop[ichan][iloc], 2)), obserr_bound_max[ichan]);

in routine:

filters/obsfunctions/ObsErrorBoundIR.cc

of UFO ... perhaps someone understand clearly the relationship between the JED and GSI tests ...

@gmao-wgu
Copy link
Contributor

That is because the final upper bound used in the gross-check is not a constant value provided in the satinfo file, and it also depends on other things.

@rtodling
Copy link
Contributor Author

rtodling commented Oct 2, 2024

That is because the final upper bound used in the gross-check is not a constant value provided in the satinfo file, and it also depends on other things.

I understand that, but I don't see how the JEDI logics map onto what GSI does! Do you?

@gmao-wgu
Copy link
Contributor

gmao-wgu commented Oct 2, 2024

Yes, the gross-check for IR in JEDI is identical to the one in GSI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants