1dVar

[Huw Lewis]

Email from a new ROPP user

Hi Huw - My name is Fabrizio Pelliccia, I am a PhD student from 
Italy visiting UCAR at present. Christian Rocken (COSMIC CDAAC) 
said me that I could ask you about problems relative to ROPP. 
I need to retrieve profiles of pressure, temperature and humidity 
using the ROPP 1DVAR module (ropp_1dvar) starting from refractivity
 profiles contained in atmPrf files provided by UCAR COSMIC.
I have to use ropp_1dvar_refrac tool. This tool requires as input 
the text file specifying configuration options, the ROPP format 
observation file, the file with observation error covariance 
parameters, the ROPP format background data file and the file with
 background error covariance parameters (relative to ECMWF-type 
pressure-based model levels). About the ROPP format observation 
file I think I can use ucar2ropp tool, tool contained in ropp_io 
module, to convert UCAR 'atmPrf' netcdf files to ropp-standard 
netcdf (Is it correct? I've tried with an atmPrf file and it 
works). 
The problem is that I don't know where I can find or how I can 
obtain the file with observation error covariance parameters, the 
ROPP format background data file (colocated with the occultation)
 and the file with background error covariance parameters (using 
ECMWF data). Can you explain me this procedure? Have I missed 
something?
Correct me if I wrote something wrong, I started to study the ROPP
 package only ten days ago :):)

My response (shows how complex it is!)

Fabrizio,

Thanks for your email regarding using the ROPP 1dVar routines in your 
work. 

For more details about the 1dVar processing and the forward models in
ROPP, please refer to the User Guide documents (see Chapter 5 in
particular).
http://garf.grassaf.org/user-documents/grassaf_ropp_ug_1dvar.pdf

You are correct in saying that the ucar2ropp tool is designed to convert
the UCAR 'atmPrf' netcdf file to the ROPP-format netCDF. I'm pleased to
hear you have this working ok.

You are correct in your summary of the possible inputs to the ROPP 1dVar
tools. I would also refer you to the test example of running the ROPP
1dVar tool which you can see in file ropp_1dvar-2.0/tests/test_1dvar.sh
This shows the call

../tools/ropp_1dvar_refrac -y  $OBS -b $BACK --bg-corr $COV -o $OUT

Examples of each type of file (IT-1DVAR-01_*.nc and IT-1DVAR-04_*.nc)
are provided in the ropp_1dvar-2.0/data directory. Perhaps running the
1dVar tool with these example data files will help make sense of the
various inputs/outputs.

More specifically, 

OBSERVATIONS:
This is the converted UCAR atmPrf file

BACKGROUND DATA:
You can find co-located background profiles from the NCEP GFS and ECMWF
analyses (for example), via the COSMIC data center
e.g. http://cosmic-io.cosmic.ucar.edu/cdaac/login/cosmic/
I'm not sure where you are getting your atmPrf profiles from, but you
should be able to find corresponding background data for each profile
provided by UCAR in the same place.
There is currently no tool provided with ROPP to convert these files to
ROPP format netcdf (this is something we will consider for future ROPP
releases), but I hope you might be able to use the template provided
with the ucar2ropp tool to do this. Please ask if you have further
questions doing this. Alternatively, you may choose to edit the 1dVar
tools provided with ROPP to read the UCAR format directly? 

I would also like to draw your attention to the GRAS data and co-located
ECMWF background data provided by the GRAS SAF. Login to
http://garf.grassaf.org and you will find the 'GRAS Products Structure'
under the Data header. These data files are already provided in the ROPP
netCDF format. The 'bgr' files provided will at least show the structure
of data you are aiming to use.


OBSERVATION ERROR COVARIANCE:

File ropp_1dvar-2.0/data/IT-1DVAR-04_o.nc provides an example of the
refractivity error correlations assumed at the Met Office. This is used
with the 'VSFC' configuration option (section 5.3.2 in the User Guide).
Note this option requires the user to specify the magnitude of the
refractivity standard deviation by giving values to the 'refrac_sigma'
variable in the input observation file. We assume refractivity errors as
a fraction of the observed refractivity (so these are uniquely defined
for each input profile). I have attached an IDL routine
ropp_sigma_metoffice_obs.pro which shows how these errors are defined.
You may find that other centres choose to define the error structure
differently - this is something to investigate, and I would not want to
suggest that the method used at the Met Office is unique. 
In general, if using bending angle observations it is assumed that the
data are uncorrelated (VSDC option), and no observation correlation file
needs to be specified when using the tool ropp_1dvar_bangle.

BACKGROUND ERROR COVARIANCE:

I have attached a netcdf file for the 91 level ECMWF background data
which contains the correlations (ecmwf_error_corr_L91_rev). If you use
NCEP data, you would need to convert the errors for that model to an
appropriate form. Note the data in this file is held in 'packed' form -
only the upper triangle of the matrix is stored. Again, we define the
errors for each background variable in the background file (see
ropp_1dvar-2.0/data/IT-1DVAR-01_b.nc for example). I attach some data
files (provided to me by Christian Marquardt, EUMETSAT) which shows some
possible standard deviations for temperature and humidity you may wish
to use.

This highlights the need for more information to new users about the various inputs required to the 1dVar tools, particularly when many users will not have access to ROPP format data in the first place.

1) Provide examples of ECMWF 91 level correlations with sigma values. 2) Provide tools for defining observation sigmas within the package. 3) Consider expanding the ucar2ropp tool to cover observation and background data files provided by UCAR.

[Huw Lewis]

1) Added sub-directory errors/ to ropp_1dvar module containing example bg and obs error files, together with scripts to combine error information with input ROPP netcdf files. See ropp_1dvar/errors/README.

2) Expanded functionality of ucar2ropp tool to handle atmPhs (phase, amplitude), atmPrf (bending angle, refractivity) or ecmPrf, wetPrf (pressure, temperature etc).

Provided this functionality to user at time of query. Ticket closed as fixed.