.svn directory and netcdf options

[frae]

Just had a talk with Paul Poli and he asked if ropp can be compiled without the netcdf library present. He is only interested in bufr. I believe there is no switch to deactivate netcdf, or is there? What about udunits, is that also only needed if netcdf in-/output is performed?

Another issue is the .svn directory. Now that we have removed it everywhere I started to wonder if we should keep the top level .svn in the distribution (e.g. ropp_tools/.svn) because that has the actual svn revision number in it. That could be useful for bug tracing. Ideally we should always update the version if we distribute an updated package but having a possible trace back could be helpful.

Rather minor issues.

[marq]

On the netcdf issue: No, we currently can't, because we use it as default output format. I would, in principle, be more than happy to provide a non-netcdf version using bufr instead. However, we do not have a direct read or write interface to bufr; all we can do is to use the roppbufr tools to convert from ropp (i.e., netcdf by default) to bufr and vice versa. That's because we do not have the bufr reader / writer as a subroutine getting an ro_prof structure as input. The way out would be that Dave provides a subroutine version for reading and writing bufr that can be integrated into the mechanism.

udunits is mainly used in the unit conversion within the netcdf frontend, but I also use it to convert units in the forward model examples - the internal units are SI, the external ones not (e.g., if I remember correctly, we have hPa in the files, but the forward models uses Pa). We could get rid of that by reworking our example files, of course. The drawback is that we probably get many users who do not remember to put the right units into data files, and then bother us with their problems. As long as we keep netcdf and udunits, users can use what they want, as long as it is physically meaningful. And we force them to think about units. Sean had a couple of cases where someone (GFZ or DMI or...) apparently didn't know that Pa is different from hPa, and who claimed they had found an huge, huge bug in his software.

For the version: I wouldn't want to have entire directories from subversion being around... If we put a $Rev$ keyword in a dedicated file (say 'version') in the top level directory, would that always point to the actual version, or only to the last version where that file was changed? Or could we, with a few shell commands, simply extract the version information in the .svn directory and write it to such a file when doing the make dist? I don't know if automake supports such a hook in it's mechanism, but it would be worth to find out...

Oh, yeah. I've just checked. There is a dist-hook in automake. So we could do a cat .svn/format > version in the dist-hook and include the generated file in the distribution. The only drawback is that we need to make sure that the working directory is up-to-date and doesn't have mixed revisions - i.e. an 'svn update' would be mandatory before packing up a distribution.

Shall I implement that?

Chris.

[frae]

Thanks for the netCDF and udunit info.

The svn solution sounds even better than the simple one I suggested. Perfect.

Caption Picard would say: Make it so.

[Dave Offiler]

As it stands, the BUFR tool reads ONLY via the ropp_io_read interface, which of course is netCDF (or text). What did Paul want to convert to BUFR? He is at liberty to re-use the BUFR part and read (or write) his own files of course. As it stands, ropp_tools doesn't require BUFR, but BUFR requires netCDF, so there's no point in trying to build the BUFR tools without the netCDF library being present or 'switched off'.

The BUFR encoder/decoder are stand-alone tools (from ropp2ropp) because: a) the ropp-->BUFR and BUFR--> ROPP command line interfaces are both rather more complex and significantly different from each other; including the BUFR function in ropp2ropp would be a real pain for the programmer and even more confusing for the user. b) The current version does not have a single BUFR subroutine interface - though a wrapper routine could be provided. Maybe for a future release, but a won't-do for ROPP-1. c) If BUFR were implemented in ropp2ropp, then users whould be forced to have the BUFR library to build ropp2ropp even if they never want to interface with BUFR.

• So I have no plans to merge these tools.

Re the svn version; A release version ID (e.g. ROPP-1.2) will tell us exactly which svn version a user has (since a release will freeze on that particular svn version, and the look-up will be fully documented, won't it). Nevertheless, having the release also include thes svn - and to quote it when reporting a bug via the webpage - would confirm the exact s/w set. But it shouldn't be necessary.

I would have though we have all learnt by now that svn update is mandatory before building a distribution for our own testing, nevermind a formal release for external users.

[marq]

As it stands, ropp_tools doesn't require BUFR, but BUFR requires netCDF, 
so there's no point in trying to build the BUFR tools without the netCDF
library being present or 'switched off'.

Yes, but we could make the netCDF part optional, if we would have read and write from the bufr directly. Then, users could throw in their bufr and ingest it directly (and also produce bufr, I they want to). But it depends on the availability of a bufr read and and write engine that we can integrate into the ropp_io_read and ropp_io_write.

In a similar way, we could make the bufr library optional - we would just require that at least either netCDF or bufr is available. We could have done the same with the ascii interface, but then we decided that ascii isn't the recommended strategy any more. With bufr it's a bit different, as that's kind of a formal operational one; while we wouldn't have all the functionality we get with netCDF, we all we need for NWP.

But it completely depends on your plans, Dave, where you want to go with the bufr interface.

svn: In principle, I agree with you; but in practice, what we will do during beta testing is that someone tells us a bug, we hopefully fix it, and then have a new svn version number - but we do not necessry increase the formal version number each time, as this is a bit more effort (going through all the configure.ac's and make a change, run autoreconf, etc. So if we would have the revision somewhere in the distributables, we would immediately know from what revision the code is. We could also give out patched versions to people (and even update the ftp server or wherever you make the code available), and would still know precisely which version the user is having trouble with. I think it would simplify Axel's work on dealing with the bugs a great deal...

Just my thoughts,

Chris.

[marq]

The generation of a ChangeLog from svn (as proposed in #41) is even better than a simple version file; so I close this as a duplicate.