Santi Oliveras (IEEC) noticed that occasionally ropp_pp_occ_tool stops running. He supplied the attached 3 files (and their paired navigation bit files), which I ran through the attached test1.sh.
ropp_pp_occ_tool runs for me, in the sense that it completes (in 15-20 secs), although the output is pants, eg:
---------------------------------------------------------------------- ROPP Occultation Pre-processor Tool ---------------------------------------------------------------------- INFO: Reading configuration file stalled_processes/cosmic_pp.cf. INFO: Processing profile 1 of 1 INFO: Reading input data file stalled_processes/ropp_atmPhs_C001.2011.192.01.14.G09_2010.2640_nc. INFO: (OC_20110711011445_C001_G009_UCAR) ...: Occultation point: Lat= 55.67, Lon= 83.43 INFO (from ropp_pp_preprocess): COSMIC data preprocessing INFO (from ropp_pp_preprocess): Reading lost carrier flag data from input file INFO (from ropp_pp_preprocess): Reading external NDM data from file stalled_processes/gpsBit_C001.2011.192.01.14.G09_2010.2640_txt INFO (from ropp_pp_preprocess): COSMIC data: openloop preprocessing ... (from ropp_pp_preprocess): Navbit correllation area: 62.4 66.4 ... (from ropp_pp_preprocess): Max. navbit correlation: 0.983 at 0 ... (from ropp_pp_cutoff): L1 aperture: 16388.670 FFT points: 1024 ... (from ropp_pp_cutoff): L1 aperture: 835.244 FFT points: 64 ... (from ropp_pp_cutoff): L2 aperture: 946.200 FFT points: 64 ... (from ropp_pp_cutoff): L2 badness between 15.0 and 50.0 km: -Infinity ... (from ropp_pp_cutoff): Click at time( 2) = -0.475 ... (from ropp_pp_cutoff): Click at time( 2) = -0.475 ... (from ropp_pp_cutoff): Click at time( 2) = -0.475 INFO: Retrieving bending angle profiles by GEOMETRIC OPTICS ...: Smoothed bending angle. ...: Pmin = 2000.000 Pmax = 132457.749 ...: ws_go_smooth 143 ...: Full resolution bending angle. ...: Pmin = Infinity Pmax = -Infinity ...: ws_go_full 0 WARNING: Cannot process profile (full): Pmin > Pmax INFO: Retrieving bending angle profiles by WAVE OPTICS ...: WM = 143 ...: W = 0 ...: WL = 0 WARNING: Cannot process profile (wo): Pmin > Pmax WARNING: Cannot process profile: Invalid bending angles computed INFO: Writing to output file ropp_atmPhs_C001.2011.192.01.14.G09_2010.2640_out.nc
I suspect this is associated with the presence of missing data (specifically, phase_L2=-999) in the source files. In which case, #287 refers.
There's also a mismatch between the size of the data in the nc file (dim_lev1a = 6199) and the size of the navbit file (which has 7500 elements). Could this be significant?
Attachment: | stalled_processes.txt added |
---|
Attachment: | stalled_processes.tar.gz added |
---|
Resolution: | → fixed |
---|---|
Status: | new → closed |
Asked Santi if he was happy with this:
Hi Santi, I think it is the missing data that is causing the problems. When I remove it from one of the profiles (attached), ropp_pp_occ_tool runs OK and gives sensible refractivities and bending angles. See https://trac.romsaf.org/ropp/ticket/375 for some discussion. If you agree that the missing data is the cause of your problems (ie the attached dataset works OK for you), then I'll close the ticket as the issue is being addressed in another one (#287). Regards, Ian.
No reply after 3 months so closing ticket.
Despite concerns over the navbit file compatibility, I tested the sensitivity to missing data by removing the missing phase_L2 points from the file and retrying (with the old navbit file). (ncks to find the missing data points and to extract the non-missing parts to two files, ncpdq to make dim_lev1a their record dimension, ncrcat to glue them together, ncpdq to make dim_unlim the record dimension).
It works:
And the bangle and refrac profiles look OK too.
So I think this missing data portions of the profile explain it. In which case, we can probably close this ticket as the issue is addressed in #287.