Cannot decode EUMETSAT GRAS BUFR with bufr2ropp

[Dave Offiler]

Dr. Prashant Kumar reports via the Helpdesk:

Subject: GRAS BUFR data in ROPP software
Question:
Dear Sir

I installed the ROPP-4.1 version with bufr and netcdf utility in
Linux machine with ifort compiler. All the process are installed
successfully. I did some test with available bufr data in
ROPP-io-4.1 using bufr2ropp and able to get netcdf file
successfully.

Now, i want to use GRAS RO data which is available in our centre
SAC, ISRO through EUMETCAST in bufr format. Now i got this error.

Reading: gras file

Error: Normal end of file reached before any RO message could be
decoded.

Kindly suggest me, how i can use gras data in bufr2ropp utility.

prashant
-- 
Dr. Prashant Kumar
Scientist
Atmospheric Sciences Division
Space Applications Centre
Indian Space Research Organisation
Ahmedabad-380015
Ph: +91-9375785190, 07926916052(O)

A 'holding' reply was sent:

Dear Prashant Kumar.

Thank you for your request.

Please notice that multiple GRAS products are disseminated via
EUMETCast. The GRAS SAF products are disseminated in the saf-europe
channel with filenames matching the following pattern:

S-GRM_-GRAS_RO_L12_*_001_METOPA_*_DMI.BUFR

EUMETSAT are disseminating GRAS level 1b data in the EUMETCast eps
channel. Two example filenames are shown below:

gras_20100929_120742_metopa_20469_eps_o_thn.l1_bufr
gras_20100929_120742_metopa_20469_eps_o.l1_bufr

The EUMETSAT GRAS bufr files disseminated in the eps channel will
not work with the GRAS SAF bufr2ropp tool.

We suspect that you are trying to convert the EUMETSAT level 1b
buffer, and this is why you see the error message. If this is not
the case, please try to contact us again.

We hope this solves your problem!

Kind regards
The GRAS SAF Team

User's response was:

Dear Sir

Thanks, currently we downloaded these GRAS RO files from EUMETCAST in real time
gras_20100929_120742_metopa_20469_eps_o_thn.l1_bufr
gras_20100929_120742_metopa_20469_eps_o.l1_bufr

can u give me some suggestion how to use this level 1 files in ROPP.
As you mailed it will not support by bufr2ropp now i will check with
S-GRM_-GRAS_RO_L12_*_001_METOPA_*_DMI.BUFR file.

Regards.
prashant

While the statement in our reply that bufr2ropp doesn't work with EUMETSAT BUFR files was technically true, unfortunately it didn't trigger a Ticket to investigate why this is the case.

This Ticket is now raised, since in principle bufr2ropp ought to deal with EUMETSAT-encoded GRAS data unless there is a non-conformance of the BUFR template (in which case EUMETSAT should correct it).

[Dave Offiler]

Stig has provided two BUFR files of thinned BAs encoded by EUMETSAT. The first, from 2009 and supplied directly by Axel, can be decoded by bufr2ropp. The second, from 2010 is one captured off EUMETCast, and cannot be decoded. The INFO/ERROR messages claim that there are no valid GPSRO BUFR messages in the file.

Both files can be correctly decoded with the generic decbufr tool. From the header dump, it was immediately obvious that the 2009 file was encoded with BUFR Edition 3, and the 2010 with Edition 4. Since bufr2ropp uses some information extracted from Section 0 to check for valid BUFR and further that it is valid for GPSRO, this might explain the problem as Ed.4 inserted new octets into Section 0.

Running bufr2ropp -d on the 2010 file reveals that the Data Type is claimed not to be 'satellite sounding' (code 3) as expected, so the message is rejected as not valid for GPSRO.

Reviewing the function notRO() in bufr2ropp.f90, the Data Type (or Category) code value (and Section 2 flag) is extracted from a specific octet offset in Section 0. Checking the specification for Ed.4, these offsets are two octets further on. The effect for Ed.4 messages is that the code is looking in the wrong place, and (in this case) finding a value of 1, clearly not the expected 3.

Added a test to check for Ed.3 or Ed.4 (the Edition value itself is already pre-checked) and extract Data Type (and Sec.2 flag) from the appropriate offset. Some additional diagnostics also output to display the found and expected values when these differ. Re-built executable and bufr2ropp now correctly decodes the Ed.4 message (and the Ed.3 one). Also ran the tests/t_roppbufr script successfully. Checked notRO() tests and modified diagnostic outputs by temporarily hard-coding erroneous values. GPSRO encoded in BUFR Edition 4 should now be fully supported. [Currently, only EUMETSAT are doing this; DMI/GRAS-SAF, GFZ and UCAR are all using the MetDB BUFR library which does not support encoding of Ed.4.]

Passed updated bufr2ropp.f90 to Stig for testing with more EUMETCast Ed.4 files. If Stig is happy, I'll copy it to Dr. Kumar to confirm that it solves his original problem.

[Dave Offiler]

No further response from Stig, so have sent the patched file to Dr Kumar to test anyway.

[Dave Offiler]

Change committed as #2669.

No reply from Dr Kumar. Assuming this patch is acceptable, so closing this ticket.