Updates to the EUM data reader and writer

[Ian Culverwell]

Two points have emerged from discussions with Axel.

1. Allow single level lev1b r_leo etc

by replacing

    IF (getbufr) THEN

in Sec 9.6 of SUBROUTINE ropp_io_read_eumdata(DATA, file, resolution, getlevel1a, getbufr) in ropp_io_read_ncdf_get.f90 with

   IF (.NOT. getlevel1a) THEN

See attached, and #429.

2. Add time_offset (wrt start time) to time in Sec 1.7.1 of ropp_io_read_ncdf_put.f90 by changing

     CALL ncdf_putvar('time', time, rec = irec)
   

to

  CALL ncdf_putvar('time', time+data%georef%time_offset, rec = irec)

Also attached.

[Ian Culverwell]

ropp_io_read_ncdf_get.f90.17062015

[Ian Culverwell]

ropp_io_write_ncdf_put.f90.17062015

[Ian Culverwell]

Safer version of ropp_io_write_ncdf_put.f90, provided by Axel, is attached.

[Ian Culverwell]

ropp_io_write_ncdf_put.f90.18062015

[Ian Culverwell]

Latest version includes an update, which, according to AvE,

first, yet another update to the above routine, I am 
now also checking the environment variable to set the 
data as either offline or NRT. Came out of a document I 
had to write where we identify the BUFR to netCDF 
variable mapping.

We have the inclusion of a block

  ! NRT of offline data
  CALL ncdf_getatt('environment', readstr)
  IF ( readstr /= 'Operational' ) data%PCD = IBSET(data%PCD, PCD_offline)

in Sec 9.4 of subroutine ropp_io_read_eumdata.

[Ian Culverwell]

ropp_io_read_ncdf_get.f90.16072015

[Ian Culverwell]

Following discussions between Stig (see #433) and Axel, we have agreed on the following solution:

• The getlev1a option to be encoded as a character, rather than a logical as at present, which can take the following values:
  • cl, provided 'level1a/closed_loop/dtime' exists;
  • cl+rs, provided 'level1a/closed_loop/dtime' exists and '/quality/rs_data_available' = 1 and 'level1a/raw_sampling/dtime' exists;
  • cl+ol, provided 'level1a/closed_loop/dtime' exists and '/quality/ol_data_available' = 1 and 'level1a/open_loop/dtime' exists;
  • none, the default, which, like all the other options, puts the single reference r_leo etc into GEOtype (under ROprof)
• (Actually, it may be preferable to remove none, since it's the default. To be decided.)
• Ensure that eum2bufr and ropp2bufr take these nominal reference values from the GEOtype substructure.
• Update ropp2bufr to convert positions and velocities to ECF coordinates when they are given in ECI coords.
• Probably best to set data%georef%time_offset = 0, in anticipation of the adoption of the updated BUFR definition. (This implies that the BUFR doc also needs updating.)

[Ian Culverwell]

eum_io.pdf

[Ian Culverwell]

Suggested improvements to the EUM data readers for ROPP9.0 are in the attached file https://trac.romsaf.org/ropp/attachment/ticket/431/eum_io.pdf. For discussion between Axel and Stig.

[Ian Culverwell]

Point 5 of the above (setting PCD_offline if environment}} /= {{{Operational) done at r4793.

This increases pcd of the standard output file from 21 to 23 (correctly: PCD_offline is bit 1, with a value of 21=2), and therefore that reference file needs updating too.

[Ian Culverwell]

One further amendment is needed: to read the GNSS identifier from the satellite_prn attribute of the /data/transmitter/satellite group. This will, in future, discriminate between GPS, GLONASS, Galileo, Beidou, ... GNSSs. (Currently, eum2ropp assumes that all data come from GPS, so that all gns_ids are automatically 'G??'.)

[Ian Culverwell]

Code changes made at r4817.

[Ian Culverwell]

While we're at it, update the ROPP8.0 IO UG to reflect the pssible new GNSS constelltions and their satellite IDs. Commit these changes at r4818.

[Ian Culverwell]

(I mean ROPP9.0 IO UG, of course.)

[Ian Culverwell]

Do the same thing with UCAR input at r4819.

[Ian Culverwell]

And same again with GFZ input at r4826.

[Ian Culverwell]

The introduction of a new locations (namely, rodata%{leo,gns}_pod%{pos,vel} for the 'reference' POD data, as well as the removal of the now-redundant -b option, have been made at r4857. (See also #452.)

[Ian Culverwell]

Stig has also requested that the PCD_open_loop bit be set according to the value of /quality/ol_data_used, rather than the request and existence of ol or rs data. This has been done at r4951. Checks out OK on GRAS_1B_M02_20150224222321Z_20150224222513Z_N_T_20150225003759Z_G09_NN.nc (ie it no longer sets PCD_open_loop, because this file doesn't contain /quality/ol_data_used. Doesn't affect the 'make tests' because the example dataset doesn't have raw sampling data anyway.

[Ian Culverwell]

After reflection, I think perhaps we should leave the data%georef%time_offset business alone. It's more of a responsibility of the data providers than of the data processors. Stig agrees.

[Ian Culverwell]

Christian Marquardt (EUM) has added the latest (30/6/2016) set of ropp_io_read_ncdf_get.f90 changes (attached).

[Ian Culverwell]

A patch describing these changes and some relating to #470 is also attached (bufr-fixes.patch.30062016).

[Ian Culverwell]

bufr-fixes.patch.30062016

[Ian Culverwell]

ropp_io_read_ncdf_get.f90.30062016

[Ian Culverwell]

According to CM's latest version, we should be setting PCD_offline unless the global attribute environment is either Operational or Validation.

This change tests OK, so commit at r4993.

[Ian Culverwell]

There are also some changes to the software_version global attribute - namely, to use the last digit of the fractional part (mmm) to indicate the bending angle processing method in case of bufr encoding.

Problem: we effectively always use BUFR encoding now, so what to do about the lines that say

    IF (getbufr) THEN      
      SELECT CASE (TRIM(data%bangle_method))
        CASE ("FSI")
          readint3 = 1
        CASE ("GO")
          readint3 = 0
        CASE DEFAULT
          readint3 = 9
      END SELECT
      WRITE(data%software_version, '("v",i2.2,".",i2.2,i1)') readint, readint2, readint3
    ELSE
      WRITE(data%software_version, '("v",i2.2,".",i2.2)') readint, readint2
    ENDIF

getbufr doesn't exist any longer, so I suggest replacing

    IF (getbufr) THEN

by

    IF ( TRIM(ADJUSTL(getlevel1a)) == 'none' ) THEN

(ie, we don't want any lev1a data in the file at all, which is sort of the same thing). But why make the distinction at all? Ask Chris M.

[Ian Culverwell]

He's not bothered. (He didn't know we had done away with the getbufr option at ROPP9.0.) Agreed with him to always write the 'augmented' software_version, and to add bangle_method to processing_software, thus:

  WRITE(data%software_version, '("v",i2.2,".",i2.2,i1)') readint, readint2, readint3 
 
  data%processing_software = TRIM(readstr)  // ' ' // & 
                             TRIM(readstr2) // ' ' // & 
                             TRIM(data%bangle_method)

Do this at r4997.

[Ian Culverwell]

Lots of work on this, which checks out fine in the test folder. Closing ticket.