id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
696	Problem with occultations being cut short when dtime > 240s	Stig Syndergaard	Stig Syndergaard	"The call to ropp_io_rangecheck in Section 6.1a of ropp_pp_occ_tool.f90 cuts occultations short if dtime is larger than 240s. The purpose of the call that was put in during the merge of occ_tool and invert_tool in this change set: https://trac.romsaf.org/ropp/changeset/4638/ropp_src/branches/dev/Share/dmi_trunk_8.0 was to insure the processing of gap-free data similar to the call in Section 6.1b, which was carried over from ropp_pp_invert_tool.f90, and originally introduced here: https://trac.romsaf.org/ropp/changeset/3639/ropp_src/branches/dev/Share/ROPP70_prototype

A key sentence from an email correspondence with Ian from May 13 2013 is this: ""Call range-checking.  This has the effect of zapping all the lev1b profiles where the impacts are missing.  Doesn't have any other effect on the data because they've already been through range-checking when they were read in.""

This was with level 1b. With level 1a the situation is different. The valid_range of dtime goes to 240s. The range-checking set's values larger than 240s to ropp_MDFV and then in a later loop removes the samples because the independent variable dtime is now missing for those samples.

The problem is that dtime > 240s can be valid, and is valid in particular during the ionospheric experiments on Metop-A that started on 2020-06-09 and ended on 2020-09-11. For those data:
{{{
- dtime for setting occs starts much later than around 0s
- for the first period until 2020-07-15 it was maybe 1-2 minutes later
- in later period until 2020-08-26 it was more
- in the period after 2020-08-26 it was less again
- in all cases time_offset increased accordingly as it should
- the effect is that many profiles are cut short, and there is no data at all when the first dtime > 240 s.
- side effect: many pcd values are set to 241 in the DMI processing, because the processing crashes with segmentation fault before the offline bit is set in the call to ropp_io_set_attrb.
}}}
These parts of reading in data with potentially missing values in them, should be revised. It may not be sufficient to remove the call to ropp_pp_rangecheck, that alone could create other problems. Better perhaps to increase the range for dtime, or simple not have it range-checked. This should be investigated. Also the excess phase data are checked in the range-checking, and set to ropp_MDFV if outside some ranges. It is unclear if these ranges are wide enough as phase data can be offset be a very large number of cycles, but still be perfectly fine. This should also be investigated. Potentially, range-checking should never be done on level 1a data. The occ_tool should not crash with segmentation fault no matter how odd the input data are.
"	task	closed	critical	DMI ROPP developments	ropp_pp	10.0	fixed