MetDB extract problem

[Dave Offiler]

MetDB extract apparantly failing to decode/return expected number of GPSRO records. This is happening for local Desktop RPC, Mike's private OPS and operational OPS (Adrian reports latter mainly for QU18z runs) since 10 Nov.

Raised urgent remedy INC am 13/11. Brian Barwell assigned and investgating. Looks like problem running out of array space decoding a particular profile (similar to early days when we had to restrict MetDB BUGT to no more than 375 samples per profile). Brian has dug out that the replication count for frequency (should be 1 or 3 for Corrected BA only or L1+L2+C) is a Very Large Number. Futher, an example offending profile was a GFZ GRACE one at 01:17utxc on 12/11.

Looking at the original GFZ ROPP-text file, this particular file has only the header present; all the Level NPoints values are zero and there is no data content to the file. From the log files, the BUFR encoder has attempted to encode this occultation and has output a BUFR file of some 150-off bytes and the generated BUFR is techncially entirely valid. This is too much of a coincidence for the MetDB problem; slthough the BUFR is correct, and MetDB stores it OK, MetDB extract is gagging on it. Hence such cases need to be suppressed.

[Dave Offiler]

ropp2bufr is reading the offending file(s) normally, and encoding the usual Sections 0,1,3 from the provided data in the ROPP file header. Section 4 is also being popsulated with the non-profile parameters (date/time, RoC...), with MISSING data flags where necessary. But with no profile data (replication counts zero). A valid BUFR file is then produced and the cron script copies this onwards to MetDB.

ropp2bufr should be modified so that if there is no valid data in any of the sections to te be encoded (ie Levels 1b,2a,b,c,d) then no BUFR file is generated.

It is also noted that while most compilers deal with this file as expected, the NAG-compiled tool stops with a 'failed to read input file' message, so would not have generated a BUFR output file...

[Dave Offiler]

a) Last item (failing to read offending file with NAG-compiled version of ropp2bufr) traced to uninitalised 'ierr' in routine ropp_io_read_text_v10.f90. ierr is only being set in READ statements inputting the Levels (profile) data. There are no 'iostat=ierr' keys when reading header lines. Luckily other compilers seem to default to zero, but it is not good programming practice to rely on this.

Corrected by inserting an initial 'ierr=0' at rhe start of the routine and an 'iostat=ierr' on the READ statement for the no. of points (last header line). Re-built ropp_io library & tools with nag f95 and behaviour is now identical to other compiled versions (re-compiled ifort & pgf behave as before).

b) Added a block of code to ropp2bufr.f90 so that a profile is skipped from being encoded if all of Npoints for Levels 1b,2a,b,c,d are zero. Existing code will then detect if no profiles have been encoded this session, and remove any (empty) BUFR file that would have been pre-created ready to ship out a successfully-encoded BUFR message. Rebuilt with ifort, pgf & nag and (with (a) in place) all now reject GFZ files with header-only blocks.

c) Unrelated to the MetDB problem, but DMI have provided a first test BUFR file. In Section 1, Originating Centre is correctly encoded as 'Copenhagen' but the originating Sub-centre claimed to be the Royal Belgian Observatory! Traced to an incorrect look-up table entry (32 for ROB?) in roppbufrcodes.nl now changed to code zero (no sub-centre, implying DMI). Corrected version returned to Martin.

Commited as [990]