Include the new (SES) 91L and 137L ECMWF B matrices in the ROPP distribution.

[Ian Culverwell]

Joe Nielsen (DMI) kindly supplied these, so we should supply them with ROPP. Don't forget to update the documentation.

[Ian Culverwell]

Joe says:

I am using a newer 91 level B-matrix from ECMWF, which is not listed above. I interpolate this matrix to 137 levels.

I attach it anyway, with the diagonals also.

[Johannes K. Nielsen]

Some documentation on these matrices an error profiles. The B-matrices are constructed from one dimensional profiles of error estimates combined with an error correlation matrix C: B_{kl} = \sigma_k \sigma_l C_{kl}

The error estimates (sigmas) for temperature and humidity are based on the so called “Scaled Ensemble Standard Deviation” (“ses”) variable1 [Holm, E. V. and Kral, T., Flow-dependent, geographically varying background error covariances for 1D-VAR applications in MTG-IRS L2 Processing, ECMWF Technical Memorandum No. 680, 2012.]. Specifically, the data type “ses” (stream=enda, class=od) disseminated by ECMWF, representing the estimated error of the forecast at 0900 UT and 2100 UT. An averaging procedure is applied; the errors actually used are averaged over a series “ses” retrievals. joe

The specific humidity background error profile and the surface pressure error are calculated individually for each occultation, assuming a global constant relative error of specific humidity and surface pressure, while a constant temperature background error profile is used. A detailed description of the background error covariance construction can be found in [ROM SAF, 1D-Var Algorithm Theoretical Baseline Document, SAF/ROM/DMI/ALG/1DV/002, RU.].

The correlation matrix (C) is constructed from two constant 91 level constant correlation matrices provided by ECMWF. The procedure for creating 60, 137 etc. levels is as follows:

1) Interpolate each part of the C matrix simply as a 2-D function. 2) Force diagonal to equal 1. 3) Set negative eigenvalues to same value as smallest positive eigenvalue.

These matrices are still to be subject to review (OR4 of the ROM SAF).

[Ian Culverwell]

I used the attached make_B.sh to generate the (equally attached) ropp_bg_ses_ecmwf_error_corr_L60.nc, ropp_bg_ses_ecmwf_error_corr_L91.nc and ropp_bg_ses_ecmwf_error_corr_L137.nc.

The first has the CDL dump:

netcdf ropp_bg_ses_ecmwf_error_corr_L60 {
dimensions:
	nbins = 1 ;
	n = 7381 ;
	n_state = 121 ;
variables:
	double lat_min(nbins) ;
		lat_min:long_name = "Minimum latitude valid for correlation matrix" ;
	double lat_max(nbins) ;
		lat_max:long_name = "Maximum latitude valid for correlation matrix" ;
	double corr(nbins, n) ;
		corr:long_name = "Packed correlation matrix" ;
	double sigma(nbins, n_state) ;
		sigma:long_name = "Background standard deviation" ;
		sigma:units = "K, 1, 1" ;

// global attributes:
		:author = "J K Nielsen, DMI and I D Culverwell, Met Office" ;
		:title = "B-matrix generated by the Scaled Ensemble Standard Deviation method of Holm et al, ECMWF Tech Memo 680, 2012" ;
		:notes = "Requires use_logp = use_logq = .TRUE. to be used in ropp_1dvar" ;

These files are now included in the distribution.

Also updated the documentation to say:

ropp bg ses ecmwf error corr L60.nc - Background error correlation matrix in packed 
form for 60L ECMWF model, generated by the Scaled Ensemble Standard Deviation method 
of Holm and Kral (2012). This matrix requires use logp = use logq = .TRUE. to be 
used in ropp 1dvar.

[Ian Culverwell]

Closing ticket.

[Ian Culverwell]

Closed wrong ticket! Reopening.

[Ian Culverwell]

No chance of code appearing for ROPP8.0, so moving ticket to ROPP9.0.

[Ian Culverwell]

Joe's code for implementing these changes is at r4331. Note that this code needs slightly different versions of the cov files, with separate variables temp_sigma, shum_rel_sigma and press_sfc_rel_sigma instead of the composite variable sigma. (Why?)

Joe says:

Hi Ian, the tar file contains 4 examples of such files 
(with relative sigmas). That is all I have. They are 
produced ad hoc for different purposes with at matlab 
script. The "ei" (ERA I) file is based on ef-fields 
rather than ses by the way. But the application is the 
same.
joe

These files are attached to this ticket.

[Ian Culverwell]

relpq.tar.gz

[Ian Culverwell]

Implementing in ROPP90_prototype. Difference in (bangle-)retrieved temps using bg_covar_method = RSFC (and the new SES cov file) and VSFC: Effect on shum:

(Diffs in press_sfc ~ 0.7 hPa.)

Effect on temps of bg_covar_method = RSFC with logp&q = .TRUE. minus logp&q = .FALSE.:. For shum:

(Diffs in press_sfc ~ 0.1 hPa.)

Similarly for refrac-retrieved T, q, and p*.

All these impacts seem to be of a sensible/expected size.

[Ian Culverwell]

RSFC-VSFC_temp.png

[Ian Culverwell]

RSFC-VSFC_shum.png

[Ian Culverwell]

RSFC1-RSFC2_temp.png

[Ian Culverwell]

RSFC1-RSFC2_shum.png

[Ian Culverwell]

Incorporated in ROPP90_prototype at r4812. (This includes a largely inconsequential change to ropp_io/ropp/ropp_io_free.f90 which is missing from r4331.)

[Ian Culverwell]

ROPP9.0 1D-Var User Guide updated at r4815.

[Ian Culverwell]

Joe has had a look at this and is happy:

I have looked at it.
No further comments.
joe

Closing ticket.