Changes between Initial version and Version 1 of Ticket #193


Ignore:
Timestamp:
2009-12-17T12:06:11Z (15 years ago)
Author:
Dave Offiler
Comment:

After a full analysis of sample DAT/DSC, netCDF and BUFR files provided by Torsten (GFZ), plus an example BUFR file from Greg (NCEP), the conclusion is that there is absolutely nothing wrong with the output from the ROPP BUFR encoder. A gfz2ropp-->ropp2bufr-->bufr2ropp cycle, and ncdump'ing the netCDF files to cdl shows identical output (apart from the usual non-significant file name, processing time and a few last-digit rounding errors).

Almost all of the alleged 'missing' data is indeed missing - and is expected to be missing, as these parameters (e.g. BA/N/T error, SH) are not in the GFZ files to start with! The exception to 'almost' is that profile lat/long (per level) is still claimed to be missing when decoded by NCEP, but are present and valid in the BUFR decoded with the MetDB kernel library (whether using the generic decbufr tool or ROPP bufr2ropp tool).

The problem must therefore lie in NCEP's decoder. TBC, but it seems they are attempting to decode 3 sets of BA per level - the GFZ DAT files only contain the iono-corrected BA, not L1 or L2, so there is only one 'frequncy' replication for GFZ data.

It appears that NCEP have never before decoded GFZ BUFR (CHAMP or GRACE-A), only CHAMP from UCAR, which (a) contains the parameters that are missing from GFZ and (b) contain the L1 and L2 as well as corrected BA, so they may well be making false assumptions in their decoding sequence.

While we will continue to provide advice to Greg in solving his problem, there is no evidence at all that there is any problem with the ROPP-generated BUFR, so this ticket is hereby closed as 'no problem here'.

The issue of the gfz2ropp tool needing modification to correctly convert TerraSAR-X DAT/DSC files is separately dealt with under Ticket #194.

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #193

    • Property Resolution → invalid
    • Property Status new → closed

  • Ticket #193 – Description

    initial v1  
    22
    33Torsten's direct emails quoted here:
    4 
    54{{{
    65Hi Dave,
    … …  
    87
    98
    10 in addition to the mail in the morning I send you for your analysis some  more data for day 08.206. DAT and DSC contain our ascii files which are the input for the gfz2ropp program. NC contains the resulting netcdf files whereas BUFR are the same data as I already send you in the first mail.
     9in addition to the mail in the morning I send you for your analysis
     10some  more data for day 08.206. DAT and DSC contain our ascii files
     11which are the input for the gfz2ropp program. NC contains the
     12resulting netcdf files whereas BUFR are the same data as I already
     13send you in the first mail.
    1114
    1215
    1316
    14 Some comments to the gfz2ropp program. I had to make some small changes because the output from Georg Beyerle was not exactly compatible with the GRACE data. But this is not the reason for the missing values NOAA sees.
     17Some comments to the gfz2ropp program. I had to make some small
     18changes because the output from Georg Beyerle was not exactly
     19compatible with the GRACE data. But this is not the reason for the
     20missing values NOAA sees.
    1521
    1622
    … …  
    2632line 502: change of TX to TS
    2733
    28 line 521/522: comment out. The program was interrupted at this line. I don't know why.
     34line 521/522: comment out. The program was interrupted at this line.
     35I don't know why.
    2936
    30 line 526/527: inserted, because Georg didn't wrote the azimuth angle in the file header, but in the dsc file.
     37line 526/527: inserted, because Georg didn't wrote the azimuth
     38angle in the file header, but in the dsc file.
    3139
    3240
    … …  
    4048
    4149
    42 Tomorrow I'm out of office for the whole day and back on Monday. So I hope you have all informations you need.
     50Tomorrow I'm out of office for the whole day and back on Monday.
     51So I hope you have all informations you need.
    4352
    4453
    4554
    46 Maybe you get in contact directly with Greg (Greg.Krasowski@noaa.gov) if you have already results tomorrow?
     55Maybe you get in contact directly with Greg Greg.Krasowski@noaa.gov)
     56if you have already results tomorrow?
    4757
    4858
    … …  
    5969
    6070> Hi Torsten,
    61 
    6271>
    63 
    6472> Could you just try adding -p0 (0=zero) to the ropp2bufr command?
    65 
    6673> This forces no thinning. Also -d to output more diagnostics.
    67 
    6874> Do you get the same problem with GRACE-A or is it just TSRX?
    69 
    7075>
    71 
    7276> I have a meeting in a few minutes, but I'll have a look at your files
    73 
    7477> this afternoon.
    75 
    7678>
    77 
    7879> Dave
    79 
    8080> --
    81 
    8281> David Offiler   Head Satellite Active Sensing Group
    83 
    8482> Met Office   FitzRoy Road   Exeter   EX1 3PB   United Kingdom
    85 
    8683> Tel: +44 (0)1392 88 6298    Fax: +44 (0)1392 88 5681
    87 
    8884> E-mail: dave.offiler@metoffice.gov.uk   http://www.metoffice.gov.uk
    89 
    9085>
    91 
    9286> -----Original Message-----
    93 
    9487> From: Torsten Schmidt [mailto:tschmidt@gfz-potsdam.de]
    95 
    9688> Sent: 10 December 2009 08:57
    97 
    9889> To: Offiler, Dave
    99 
    10090> Cc: Greg Krasowski; Lidia Cucurull; Jens Wickert; Georg Beyerle
    101 
    10291> Subject: [Fwd: Re: TerraSAR-X RO data in BUFR format]
    103 
    10492>
    105 
    10693> Hi Dave,
    107 
    10894>
    109 
    11095> as you know we are working on the TerraSAR-X (open-loop) processing
    111 
    11296> and hope to have an operational software version soon.
    113 
    11497>
    115 
    11698> As a preliminary dataset Georg Beyerle had prepared data from a period
    117 
    11899> covering some months in 2008. This dataset I've converted to bufr
    119 
    120100> (same procedure as with the operational GRACE data) using the ROPP software.
    121 
    122101>
    123 
    124102> The problem is that NOAA sees a lot of 'strange things'. I don't know
    125 
    126103> what's about Sean. He got the same dataset but I've no response from
    127 
    128104> him until now.
    129 
    130105>
    131 
    132106> The 'strange things' are related to, e.g., missing values in the
    133 
    134107> decoded data.
    135 
    136108>
    137 
    138109> What I've done so far was the application of the ROPP Vers.2 software
    139 
    140110> just in the way I use it for the GRACE data (since February this year).
    141 
    142111>
    143 
    144112> Because the 'strange things' happened with Vers.2 I installed
    145 
    146113> yesterday ROPP Vers.3 (ropp_utils-3.0, ropp_io-3.0) and also the new
    147 
    148114> bufrpack-18.1.
    149 
    150115>
    151 
    152116> Then I converted one day of TSX data (08.206) by application of the
    153 
    154117> following programs: (1) gfz2ropp for the conversion of the dat and dsc
    155 
    156118> files to netcdf and (2) ropp2bufr ('ropp2bufr ncfile -c 78'), i.e.,
    157 
    158119> without thinning of the data. There were now error message as well as
    159 
    160120> any warning. The results you find attached in the TSX_08.206.tar.gz
    161 
    162121> archive.
    163 
    164122>
    165 
    166123> What I've also done for validation purposes is the conversion back to
    167 
    168124> netcdf with the bufr2ropp program. I can't find any missing values in
    169 
    170125> the back-converted files, but Greg from NOAA does. His error output is
    171 
    172126> also attached.
    173 
    174127>
    175 
    176128> Dave, could you please give a comment to that. I'm not familiar with
    177 
    178129> bufr data so I think you are the person that could (hopefully) solve
    179 
    180130> the problem.
    181 
    182131>
    183 
    184132> Thank you very much.
    185 
    186133>
    187 
    188134> Bets regards,
    189 
    190135> Torsten
    191 
    192 
    193 
    194136}}}
    195137