Missing data on decoding test TerraSAR-X BUFR

[Dave Offiler]

Torsten Schmidt at GFZ has used the ROPP-2 (and also ROPP-3) gfz2ropp and ropp2bufr tools to encode some test RO data from TerraSAR-X. Reports back from NCAR are that the BUFR contains unexpected 'missing data'. Torsten has provided a day's worth of original .dat/.dsc files, the converted ROPP netCDF and encoded BUFR files, plus some background emails with NCEP (not attached to this ticket).

Torsten's direct emails quoted here:

Hi Dave,



in addition to the mail in the morning I send you for your analysis some  more data for day 08.206. DAT and DSC contain our ascii files which are the input for the gfz2ropp program. NC contains the resulting netcdf files whereas BUFR are the same data as I already send you in the first mail.



Some comments to the gfz2ropp program. I had to make some small changes because the output from Georg Beyerle was not exactly compatible with the GRACE data. But this is not the reason for the missing values NOAA sees.



The changes are in

line 493: comment out this line

line 499-502: my output goes to 'OUTPUT/', therefor "datfile(8:9)"

instead of datfile(1:2)

line 502: change of TX to TS

line 521/522: comment out. The program was interrupted at this line. I don't know why.

line 526/527: inserted, because Georg didn't wrote the azimuth angle in the file header, but in the dsc file.



I've attached the version I've changed.



Just to remember, I used ROPP Vers.3 for the results attached.



Tomorrow I'm out of office for the whole day and back on Monday. So I hope you have all informations you need.



Maybe you get in contact directly with Greg (Greg.Krasowski@noaa.gov) if you have already results tomorrow?



Thank you very much, Dave.



Cheers,   Torsten



Offiler, Dave wrote:

> Hi Torsten,

> 

> Could you just try adding -p0 (0=zero) to the ropp2bufr command?

> This forces no thinning. Also -d to output more diagnostics. 

> Do you get the same problem with GRACE-A or is it just TSRX?

> 

> I have a meeting in a few minutes, but I'll have a look at your files 

> this afternoon.

> 

> Dave

> --

> David Offiler   Head Satellite Active Sensing Group

> Met Office   FitzRoy Road   Exeter   EX1 3PB   United Kingdom

> Tel: +44 (0)1392 88 6298    Fax: +44 (0)1392 88 5681

> E-mail: dave.offiler@metoffice.gov.uk   http://www.metoffice.gov.uk

> 

> -----Original Message-----

> From: Torsten Schmidt [mailto:tschmidt@gfz-potsdam.de]

> Sent: 10 December 2009 08:57

> To: Offiler, Dave

> Cc: Greg Krasowski; Lidia Cucurull; Jens Wickert; Georg Beyerle

> Subject: [Fwd: Re: TerraSAR-X RO data in BUFR format]

> 

> Hi Dave,

> 

> as you know we are working on the TerraSAR-X (open-loop) processing 

> and hope to have an operational software version soon.

> 

> As a preliminary dataset Georg Beyerle had prepared data from a period 

> covering some months in 2008. This dataset I've converted to bufr 

> (same procedure as with the operational GRACE data) using the ROPP software.

> 

> The problem is that NOAA sees a lot of 'strange things'. I don't know 

> what's about Sean. He got the same dataset but I've no response from 

> him until now.

> 

> The 'strange things' are related to, e.g., missing values in the 

> decoded data.

> 

> What I've done so far was the application of the ROPP Vers.2 software 

> just in the way I use it for the GRACE data (since February this year).

> 

> Because the 'strange things' happened with Vers.2 I installed 

> yesterday ROPP Vers.3 (ropp_utils-3.0, ropp_io-3.0) and also the new 

> bufrpack-18.1.

> 

> Then I converted one day of TSX data (08.206) by application of the 

> following programs: (1) gfz2ropp for the conversion of the dat and dsc 

> files to netcdf and (2) ropp2bufr ('ropp2bufr ncfile -c 78'), i.e., 

> without thinning of the data. There were now error message as well as 

> any warning. The results you find attached in the TSX_08.206.tar.gz 

> archive.

> 

> What I've also done for validation purposes is the conversion back to 

> netcdf with the bufr2ropp program. I can't find any missing values in 

> the back-converted files, but Greg from NOAA does. His error output is 

> also attached.

> 

> Dave, could you please give a comment to that. I'm not familiar with 

> bufr data so I think you are the person that could (hopefully) solve 

> the problem.

> 

> Thank you very much.

> 

> Bets regards,

> Torsten

Of note is that it seems that TSRX data in the DAT/DSC is written a bit differently from how CHAMP/GRACE-A is written, which means some modification to gfz2ropp anyway.

Not critically urgent, as TSRX data is not yet being processed in NRT, but it needs to be fully supported in the next few months, and ideally the problem reported here investigated and (if the problem is found to be in one of the ROPP tools) fixed asap to support ongoing testing.

First thing to do is to rebuild ROPP_IO tools using (ideally) the same F90/C compilers as used at GFZ (awaiting this info from Torsten...)