netCDF 4.1.3 compatibility

[Dave Offiler]

The latest netCDF release is v4.1.3 (17 June 2011). On attempting to install & test this version with ROPP, some compatibility issues have arisen.

The principal one relates to separate basic C and Fortran API object libraries.

• For v3.x.x, the netCDF build system installed all C, F77 & F90 code in one library (<prefix>/lib/libnetcdf.a).
• This is also the default for v4.0.x, though ./configure has an option (--enable-separate-fortran) to enable separate Fortran & C libraries (C as above, F77/F90 in <prefix>/lib/libnetcdff.a).
• At v4.1.1 (and 4.1.2) the default is to enable separate libraries, but this can be optionally disabled (--disable-separate-fortran); this option is given in the netcdf_configure_* mini-scripts for ROPP-4.1/ROPP-5
• At v4.1.3, separate libraries is not only default, but there is no longer an option switch to disable this behaviour. Therefore to support netCDF v4.1.3 (and later), the ROPP configure must be able to deal with these two netCDF object libraries.

Secondly, to help with backwards compatibility, netCDF v4 ./configure had an option --disable-option-checking which allowed unknown options meant for a different release to be ignored if not supported in this one. netCDF v4.1.3 no longer has this option - the default is to silently ignore unknown options (including this one). Therefore leaving this option in the mini-scripts shouldn't cause any problems, and retains compatibility for previous v4 releases, so should be left in.

Finally, checking the xlf95_aix mini-script for netCDF v4.1.3 (and in particular noting the guidance at ​http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-install.html#Quick-Instructions) some of the mini-script details are likely legacy from v3 (or where never really necessary in the first place) - all netcdf_configure_* scripts should be reviewed, removing redundant options and variable settings (in some cases, let the netCDF ./configure script work them out).

I propose that for the next release of ROPP, we drop support for building netCDF v3.x.x, and further, support only the (now unavoidable) 2-library netCDF installs at all v4.x.x.

[Dave Offiler]

Progress update No.1.

1) In netcdf_configure_xlf95_aix, removing CPPFLAGS="-DIBMR2Fortran" allowed netCDF v4.1.3 to build and test ok on AIX (generating the separate C & F libraries). So this overcomes the failed builds of v4.1.x we had with ROPP-4.1 which forced us to stick with v4.0.1 (even though the builds on Linux were OK).

2) Consolidated all netcdf_configure_ifort*_linux also removing CPPFLAGS="-DNAGf90Fortran" and F90=ifort[n] lines; re-align FCFLAGS to FC and FFLAGS to F77. Changed --disable-separate-fortran (which no longer works) to --enable-separate-fortran to force all earlier netCDF-4 releases to adopt the twin-lib system. Rebuilt netCDF v4.1.3 with all variants of ifort.

Found that LDFLAGS, set by the sa_app system, was interfering with Linux netCDF builds; adding LDFLAGS="" to config scripts solved that problem.

Note that all iforts have -fp-port in the list of compiler flags. This switch causes floating point numbers to be rounded on assignment, etc, so giving a consistent result, but with a performance hit. The default is to retain native precision. I've left this switch in place for now, but it should be investigated whether removing it would cause any problems, and if not, to remove it permanently (other compilers don't have an equivalent option, at least not one we explicitly use). Since this switch has been in since day-1, initially for ifort v7 (no longer supported), this may well be legacy.

3) Modified configure.ac in all ROPP modules (except UTILS) to check nf_open using both -lnetcff and -lnetcdf. Manually run autoreconf and ropp_configure_ifort_linux to check correct configure; built & tested all ROPP modules with buildpack ropp_[xxx] ifort.

[Dave Offiler]

Progress update No.2

1) Consolidated all netcdf_configure_* scripts as per ifort; removed all CPPFLAGS=-D<some>Fortran & tested correct build (with buildpack) with all compilers and all platforms that can be tested, except:

• sunf95 : testing confirms that -DsunFortran is required (gcc fails in cfortran.h without it)
• compilers/platforms no longer available for testing (HP-UX, NEC, SUNOS) : left the existing CPPFLAGS settings in place, since they worked at some point and it can't be confirmed if they would still work without.

All Linux, Cygwin & AIX builds OK (twin-libs generation confirmed); checked in as changeset [2960].

Note that I've also

• added SVN $ Id: $ strings to all the mini-scripts
• moved redundant udunits and zlib mini-scripts to the legacy directory.

[Dave Offiler]

Progress update No.3

The xlc/xlf95 on AIX build of netCDF (4.1.3) worked only when removing the -DIBMR2Fortran option and all references to -qextname. The latter compiler options causes underscores to be appended to external names in xlf95. But there is no equivalent in xlc; underscores are controlled by the <some>Fortran option - and with this removed the default was no underscores. Hence the Fortran setting must be the same to allowing cross-language linking without errors due to 'missing symbols'.

But the consequence is that all code must have the same 'no underscores' setting, which meant all AIX configurations for ROPP, MOBUFR and ECBUFR also had to have -qextname switches removed.

All of the *_configure_xlf95_aix mini-scripts have been modified accordingly, plus the MetDB BUFR package f90_compilers.dat file and the latter package re-built (still v19.1). Also needed to remove this flag from cm_compiler_defaults.m4 in each ROPP module, rebuilding configure with roppauto. Built new v5.1 tarballs and copied to IBM. Ran buildnetcdf_ropp, buildmobufr_ropp and buildecbufr_ropp on AIX; all build & test OK. (buildpack needed a minor fix to use the correct version of C code; the Linux version didn't work).

Then built all ROPP modules in turn with buildpack - all ok. Lastly, moved the MetDB BUFR library and re-built ropp_io against the ECMWF BUFR library - all ok.

This (at least) completes the IBM/AIX compatibility updates to support all the latest dependency packages. (NOT TESTED: regression against prior releases of netCDF v4.x.x)

[Dave Offiler]

Progress update No.4

Built & tested netcdf-4.1.3 / bufr-19.1 / bufr_000387 with all Linux compilers, (including Cygwin[*], and, as a bonus, lf95 on gpswv4) using the build<dep-pack>_ropp wrapper scripts.

[*] ECBUFR fails to link test programs (for both g95 & gfortran); this is outside the scope of this ticket to fix.

Then built all ROPP modules (v5.1_prototype [2965]) likewise with build_ropp. All configure, compile & test OK.

As a regression test, and for ifort only. but with no other changes, replaced netcdf with v4.0.1 and re-build ropp_io against it - all OK. Since we've never formally supported netCDF v4.1.1 or v4.1.2, these have not been explicitly tested, but we can be pretty certain they would be OK too.