id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
132	Rview/change valid ranges	Dave Offiler	Dave Offiler	"Some of the 'valid ranges' tested in ''ropp_io_rangecheck()'' may be too strict for some parameters for which there is no clear-cut threshold for valid/not valid. The philosophy of the range check should be to set as 'missing' physically unrealistic values, not to knock out merely bad/noisy data. 

a) Review all parameter default valid ranges as defined in ''ropp_io_types.f90'' for meeting 'physically unrealistic' criteria
and change as appropriate.

b) Change the values for Excess Phase (+/- 10,000m) & BA (-0.01 to 1.0) as suggested by Stig (see emails below) for v1.2.

c) Include an explicit review of the ranges in Josep's beta testing 'shopping list' for any updates in the v2.0 release version.

NB: Stig/Martin have been informed how to over-ride the built-in defaults, so they can apply whatever range limits they wish for the current v1.1. The changes here can be applied to the defaults in the next & subsequent release versions.

From Stig:
{{{
Dave,

I had a look at some more BA data, and talked to Martin about what
we think the range check in ROPP 1.2 should/could be. We suggest 
to have wide margins for a couple of reasons:

1) we don't think the ROPP range check should be like a QC check. 
A range check is okay to avoid e.g., format overflow - but does not 
necessarily have to screen out bad data. Data can anyways be 
pretty bad, but still within physically expected ranges, so strict
limits won't catch all the bad data anyways.

2) If data are indeed bad, it is nice to have the full information
in the NetCDF files and not have the data masked by -99999.9 
values.

3) We don't really know how large BA (and phases) can become once
the open loop data starts flowing - theoretically BA can become 
infinite, although that can't happen in practice - 
but how big then?

So here is the suggestion:

 -0.01 < BA < 1.0
 
Martin also noted that the excess phases have a range between 0m 
and 1500m. That also seems way too narrow. L1 and L2 excess phases
can become negative even if they by definition starts at zero for
a setting occ because the ionosphere can make them dip below 
zero before they start increasing in the neutral atmosphere. 
For rising occs, I could imagine EUM to set the first data point 
to zero (they don't do that yet, In fact most EUM phases are way 
out of range right now - but COSMIC do it), in which case rising
phases would decrease to become negative in the neutral atmosphere. 

Anyways, 1500m is too little - it can easily reach 4000m (for 
COSMIC open loop data that happens most of the time).

So to give some margin, a suggestion there could be:

 -10000m < excess phase < 10000m.

I'm a bit concerned that there may be more range checks around in
the ROPP that are too strict. Maybe we should consider review all
range checks in connection with the release of ROPP 1.2. Perhaps
specifically ask beta-testers to look at these. What do you think?

Best regards,
Stig
}}}

And in reply:
{{{
Hi Stig,

I agree that the purpose of the ROPP rangecheck is to throw out
'physically invalid' values, not merely 'bad' ones. The obvious
things are like latitudes, which have to lie between +/- 90 degr,
but others like BA are not so clearcut.

The range values for the various parameters have a long history -
back to the CLIMAP project of the early 2000's, and they haven't
been systematically reviewed since - Chris just took the values
from the CLIMAP format document (forerunner of the ROPP test format)
when he wrote the v1.0 ropp_io_types.f90 code, and so these are 
the (default) values tested with the new ropp_io_rangecheck()
routine in v1.1

We can change the BA & Phase limits for v1.2 as you suggest, and ask
Josep review all the values as part of his beta testing 'contract'.
Meanwhile, if anyone has suggestions to revise the current default
values, we'll be pleased to take them into account before the beta
testing starts.

Dave
}}}

"	defect	closed	normal	2.0	ropp_io	1.1	fixed	valid range check	Huw Lewis