Reconcile results of IDL and rose versions of Test Folder on IT-PP-06

[Ian Culverwell]

There is a small difference between the results of the old IDL test folder and the new rose-based one when they run test IT-PP-06, which tests the new WOPT code on fours standard profiles. The difference is small, but it is big enough that the rose-based system exceeds the thresholds for 'CASE_12' (a 'Challenging, Cat 3 case: 0.1 < max (−dN/dr) ≤ 0.157). E.g. gfortran at ROPP9.1:

IDL

   Found absolute mean difference, mean absolute difference and max difference between input and output refrac for CASE_12_ref1d:   5 -  19km:   0.925 /   1.152 /   3.208 %

ROSE

   Found absolute mean difference, mean absolute difference and max difference between input and output refrac for CASE_12_ref1d:   5 -  19km:   0.021 /   1.553 /   6.459 %

IDL it_pp_06 Error: (8) Mean absolute deviations between input and output refrac exceed 1.5% above 5km.

IDL it_pp_06 Error: (9) Max absolute deviations between input and output refrac exceed 4.0% above 5km.

The picture is confused because the rose test folder doesn't (yet) process the results of the runs or generate its own graphics, so the 'ROSE' data above have been generated by passing the results of the rose test folder through the later, processing steps of the IDL code.

Note, however, that if the rose-generated executables used in this test (namely, ropp_pp_wopt_tool, ropp2ropp and ropp_pp_invert_tool) are used in the full IDL code, then the results match the IDL results. So there must be some subtlety of the rose code that means its results are not the same as you get when you (think you) run the same code via IDL. I suspect some sort of ancillary file difference. We could compare the results step-by-step.

By the way, both are better than the ROPP9.0 results, especially at lower levels, for example:

Found absolute mean difference, mean absolute difference and max difference between input and output refrac for CASE_12_ref1d: 5 - 19km: 0.925 / 1.152 / 3.208 %

[Ian Culverwell]

IT-PP-06_gfortran_linux_IDL.png

[Ian Culverwell]

IT-PP-06_gfortran_linux_ROSE.png

[Ian Culverwell]

IT-PP-06_gfortran_linux_ROPP90.png

[Ian Culverwell]

Oops - that last one should read

Found absolute mean difference, mean absolute difference and max difference between input and output refrac for CASE_12_ref1d: 5 - 19km: 0.214 / 1.142 / 3.672 %

[Ian Culverwell]

(The problem became clear when testing the ROPP9.1 results from the HPC compilers. For some reason the netCDF files were getting scrunched - putting multiple copies of profiles in the output multifile, in an irreproducible fashion - so I passed the results of the rose suite through the IDL test folder as described above. This made the ifort and ftn results exceed the threshold, as above. But if I pass (e.g.) the gfortran rose test results through the IDL code in the same way, then they too exceed the threshold to the same extent. So it is a test folder issue, not a compiler issue.)

[Ian Culverwell]

Curses! The rose suite output of ropp_pp_wopt_tool reveals

WARNING (from ropp_pp_wopt_tool):  Cannot open GEOPOT_COEF =  ... will return missing HMSL
 
WARNING (from ropp_pp_wopt_tool):  Cannot open GEOPOT_CORR =  ... will return missing HMSL
 
WARNING (from ropp_pp_wopt_tool):  Undulation is incalculable ... consider settings of $GEOPOT_COEF and $GEOPOT_CORR ... continuing with undulation set to zero

which the IDL version doesn't. (I had forgotten that the WOPT tool needs undulations.) When I rerun the gfortran tests without defining $GEOPOT_CORR and $GEOPOT_CORR, I reproduce the (bad) rose result. When I make them available, I reproduce the (good) rose result.

Update the rose suite accordingly, and rerun the ifort anf ftn tests, so that they can be passed through the IDL folder.

[Ian Culverwell]

I meant, of course, '... the (good) IDL result'.

[Ian Culverwell]

test1.sh

[Ian Culverwell]

Result - OK!

The ifort and ftn runs now find GEOPOT_COEF and GEOPOT_CORR, so the output of IT-PP-06 now reads

INFO (from ropp_pp_wopt_tool):  Reading config namelist from  default_wopt.nml
INFO (from ropp_pp_wopt_tool):  Calculating occultation point
INFO (from ropp_pp_wopt_tool):  Reading refractivity profile from CASE_05_ref1d.nc
INFO (from ropp_pp_wopt_tool):  Calculating signal on first screen
INFO (from ropp_pp_wopt_tool):  Apodizing signal on first screen
...

And they now pass the test folder, as the 'pure' IDL runs do: ifort

Found absolute mean difference, mean absolute difference and max difference between input and output refrac for CASE_12_ref1d: 5 - 19km: 0.925 / 1.152 / 3.208 %

ftn

Found absolute mean difference, mean absolute difference and max difference between input and output refrac for CASE_12_ref1d: 5 - 19km: 0.925 / 1.152 / 3.208 %

Job done!

[Ian Culverwell]

IT-PP-06_ifort_linux.png

[Ian Culverwell]

IT-PP-06_ftn_linux.png

[Ian Culverwell]

Reverting ticket to ROPP9.1, and closing.