Incorrect 1DVAR BA errors?

[Dave Offiler]

Message via Helpdesk & Kent:

> -----Original Message-----
> From: helpdesk@grassaf.org [mailto:helpdesk@grassaf.org]
> Sent: Wednesday, March 03, 2010 2:42 PM
> To: undisclosed-recipients:
> Subject: GRAS SAF Helpdesk Request w.r.t. ROPP Software
> 
> The GRAS SAF Helpdesk has received a question from Pietro Sofi
> 
> The helpdesk subject is:
> 
> ending angle standard error deviations
> 
> The helpdesk description is:
> 
> Dear ROPP team, I would be very grateful for your help on the 
> following problem.
> 
> Wen I give to ropp_1dvar_bangle a COSMIC atmProfile (from
> CDAAC) processed with ucar2ropp colocated with a ECMWF in the 
> ropp-test data (VSDC for obs errors) I get:
> 
> ERROR (from ropp_1dvar_covar_bangle):  Bending angle standard error 
> deviations requested on a per profile basis, but observation data file 
> did not contain sigmas.
> 
> Truly observation sigmas (~10-4) are present and in range ([0 0.01]).
> 
> I substituted, in my oc_20090430211921_c002_g008_ucar.nc
> file, bendings and sigmas from a colocated ropp-test observation file 
> and ropp_1dvar_bangle successfully finished.
> 
> I substituted sigmas only (~10-3) and I got:
> 
> ERROR (from ropp_1dvar_solve):  Initial cost function too high. 
> 
> (Note that i had to fill first the L1 fields in ucar2ropp output, as 
> they were set to missing values, only generic bendings and sigmas were 
> filled).
> 
> What could be wrong?
> 
> Best regards,
> Pietro Sofi
> 

[Huw Lewis]

Response to original user enquiry by Huw:

Dear Pietro,

You sent an enquiry to the GRAS SAF helpdesk concerning the use of ROPP 1dVar using COSMIC 
observation data. I hope the response you received gave you some more background and you have 
been able to make progress with your work.

In particular, note that the 1dVar requires realistic a-priori errors for both the observations 
and the model. The bending angle sigmas provided by data providers for each occultation are not 
necessarily appropriate for defining the statistical observation error as required in 1dVar 
(which should also include an estimate of the forward model error for example). The bangle_sigma
 values as provided by COSMIC are likely to serve as a lower bound on the observation errors. I 
would encourage you to investigate the influence of the observation error values on your 1dVar 
results in any case. The GRAS SAF will shortly be releasing a report discussing these issues in 
more detail.

From a technical point of view, I can now provide you with some more information relating to 
your initial enquiry. I have checked the input file you mentioned (generated from COSMIC file 
atmPrf_C002.2009.120.21.19.G08_2009.2650_nc). You will notice that some of the provided 
'Bend_ang_stdv' variable values are missing in the original file. When you convert this to ROPP
 format using the ucar2ropp tool, the missing values are preserved, and so the 1dvar_bangle tool
 gave you an error to let you know that not all the 'bangle_sigma' values were valid. This is as 
you would require - using invalid error estimate values in your 1dVar analysis would give you 
problems! If you were to overwrite the missing values with something in the range 0-0.01 then 
the 1dVar tool would continue without errors.

To define more representative errors, you may wish to use the tools provided in the 
ropp_1dvar-4.0/errors/ directory. These are described in more detail in the errors/README file.
In particular, the IDL routine ropp_generate_Osigma can be used to overwrite the 'bangle_sigma'
variable in an observation file with a profile of representative bending angle errors. 

Can I also advise you to use the ucar2ropp tool with a '-p' option to thin the observation data 
before you use the ROPP 1dVar tools. The example COSMIC file contains the full resolution 4004 
observation levels. It would be more appropriate to use an observation profile with about 300 
points. To do this, run the ucar2ropp tool as:

ucar2ropp <inputfilename> -p 300   

to sample the data to 300 points. There are several other types of data thinning you may wish to
investigate. These are detailed in the ROPP User Guide (Part I, Section 2.7). In fact, if you do
 this thinning on your sample data the missing sigma values are avoided (since they are not 
considered in the thinning), and the ROPP 1dVar would also work directly. 

[Huw Lewis]

The user was therefore implementing the 1dVar processing incorrectly, and the warning messages were therefore appropriate since the original COSMIC data included missing sigma values. No response to the helpdesk answer has been received from the user.

These issues will be clarified in the ROPP-4.1 user guide documentation, along with new tools for generating ob and background errors. See [2453].

Ticket closed as invalid.