#273 closed enhancement (fixed)
Overhaul the "test" aspects of the build system
| Reported by: | Ian Culverwell | Owned by: | Ian Culverwell |
|---|---|---|---|
| Priority: | normal | Milestone: | 11.1 |
| Component: | ROPP (all) | Version: | 7.1 |
| Keywords: | make test | Cc: |
Description
The ROPP6.0 beta reviewer described the voluminous output of text and graphics during the "make test" part of the build procedure as a "tedious adventure". He made several suggestions for improvement:
- Introduction of environment variable $ROPP_SHOW_PLOTS=[TRUE | FALSE], to suppress pop-up output if desired;
- Use of one graphics output format only (preferably eps);
- Concatenation of (better annotated) reference and test plots into single file;
- Storage in same directory as build_ropp.log file;
- Clear announcement if any test graphics files are missing;
- Inclusion of test summary table/list, inc pass/fail status.
He also additionally recommended to provide a complete list of tests for all test groups or modules, containing the
- exact test name or test identifier, the
- path/name of the script used to generate the test, the
- name of the REFERENCE data/plot file, and the
- name of the TEST output data/plot file.
This list could be provided either as a tests.README in the ./tests/ sub-directory or as a list or table in the *.pdf documentation for all test groups or modules.
He also thought it was noticeable that the annotation of graphics is incomplete: it would be an improvement to include a small table or list on each plot (i.e., on the REFERENCE and TEST plots) indicating the following information:
- the exact test name or test identifier,
- the path/name of the script used to generate the plot,
- a clear/prominent indication, if the plot is a REFERENCE or a TEST plot,
- the exact name of the plot graphics file, and the plot generation date (in YYYY-MM-DDHH:MM:SSS.S format)
Finally, he felt that the output of the final build and test message
------------------------------------------------------------------ Done. Check messages to confirm successful build & installation ------------------------------------------------------------------
is not completely satisfying. An automatic creation of a test summary list or table, indicating the test result status, would be welcomed.
We agree that the “make test” aspects of building ROPP need a “make over”. The beta reviewer's ideas resonate with similar ones we were having in connection with the testing of changes for potential inclusion in the official ROPP release, maybe my means of a “light” ROPP test folder, or some such.
Attachments (33)
Change history (48)
comment:1 by , 11 years ago
| Milestone: | 7.0 → 8.0 |
|---|
comment:2 by , 11 years ago
Still thinking about this. I think I can do it, but it'll take time. Try to include at least one module at ROPP8.0.
comment:3 by , 10 years ago
A version of this has been implemented in the FM, 1DVAR and PP modules in branch https://trac.romsaf.org/ropp/log/ropp_src/branches/dev/Share/ic_new_make_test
The basic elements are:
- Introduction of ropp_io/ropp/ropp_io_fields_compare.f90, which numerically compares a user-selectable subset of the fields in 2 RO profiles;
- Introduction of ropp_io/ropp/ropp_io_success.f90, which summarises the PASS/FAIL status of all the tests run for each module;
- Introduction of ropp_<module>/tests/ropp_<module>_compare.f90, which defines the tests to be run in each <module>; and
- Changes to all the shell scripts, Makefiles etc to accommodate these changes.
One, possibly useful, side effect of this is that the (thinned) results of ropp_pp/preprocess/ropp_pp_spectra.f90 are now also output in ROPP netCDF format, so that ropp_io_fields_compare.f90 can compare spectra.
The advantages of this method of automatic build testing are:
- There is no longer a need for IDL to be installed on the tester's system. The old plots are still produced, if possible, but there is no need for them. They are no longer displayed by default.
- The testing is more rigorous and thorough, because it checks whether the differences between each of the relevant fields in the profile is less than some predefined tolerance. This should be compared to the earlier method of visually comparing profiles of (eg) refractivity against height in two figures.
- The method is more general, and easier to extend should new variables be defined in future (eg PBLH).
Thus, for example, after running buildpack ropp_1dvar ifort, the user is no longer presented by a set of 13 (!) figures and their corresponding references to compare by eye, but instead a series of messages to the screen like
Testing basic 1D-Var functionality Running t_1dvar_01_refrac (1DVAR basic; default) ... **************************** ********** PASS ********** **************************** ... examine t_1dvar_01_refrac.log for details Running t_1dvar_01comp_refrac (1DVAR basic; comp factors) ... **************************** ********** PASS ********** **************************** ... examine t_1dvar_01comp_refrac.log for details
etc, followed by a summary table like
Making summary of test result ************************** SUMMARY OF ROPP_1DVAR TEST RESULTS ************************ -------------------------------------------------------------------------------------- | Test name | Description | Run? | PASS? | -------------------------------------------------------------------------------------- | t_1dvar_01_refrac | 1DVAR basic; default | Run | PASS | | t_1dvar_01comp_refrac | 1DVAR basic; comp factors | Run | PASS | | t_1dvar_GRAS_05_bangle | 1DVAR GRAS; default | Run | PASS | | t_1dvar_GRAS_05comp_bangle | 1DVAR GRAS; comp factors | Run | PASS | | t_1dvar_GRAS_05newop_bangle | 1DVAR GRAS; new interp | Run | PASS | | t_1dvar_GRAS_05_refrac | 1DVAR GRAS; default | Run | PASS | | t_1dvar_GRAS_05comp_refrac | 1DVAR GRAS; comp factors | Run | PASS | | t_1dvar_GRAS_05newop_refrac | 1DVAR GRAS; new interp | Run | PASS | | t_1dvar_COSMIC_04_bangle | 1DVAR COSMIC; default | Run | PASS | | t_1dvar_COSMIC_04comp_bangle | 1DVAR COSMIC; comp factors | Run | PASS | | t_1dvar_COSMIC_04newop_bangle | 1DVAR COSMIC; new interp | Run | PASS | | t_1dvar_COSMIC_04_refrac | 1DVAR COSMIC; default | Run | PASS | | t_1dvar_COSMIC_04comp_refrac | 1DVAR COSMIC; comp factors | Run | PASS | | t_1dvar_COSMIC_04newop_refrac | 1DVAR COSMIC; new interp | Run | PASS | | t_1dvar_iono_bangle | 1DVAR L1 and L2 | Run | PASS | -------------------------------------------------------------------------------------- This table is saved in ropp_1dvar_summary.txt. Each test has its own log file.
This should make the 'tedious adventure', to which our friend from southeast Austria alluded in his ROPP6.0 beta review, a little less exacting.
comment:4 by , 10 years ago
| Keywords: | make test added |
|---|---|
| Type: | defect → enhancement |
| Version: | 5.0 → 7.1 |
This branch was merged with the ROPP8.0 prototype branch at r4327.
comment:5 by , 10 years ago
| Milestone: | 8.0 → 9.0 |
|---|
The IO module has beaten us this time, mainly because of the diverse nature of the tests and of the outputs. Though it has to be said that the testing adventure is less tedious for this module than it is for the others. Nevertheless, we will try to include it in ROPP9.0. The ticket is being reclassified accordingly.
comment:6 by , 8 years ago
| Milestone: | 9.0 → 10.0 |
|---|
Ran out of energy for ROPP9.0 too. Pity, because it would be really useful to fix this for ropp_io, as this is the module that probably needs it most, because this is the one that has the most variable set of tests to run (because of the (un)availability of external dependency libraries).
While we're doing it, make sure the +/- 0.1 m differences in {impact, impact_L1, impact_L2} between bufr_test.nc' and ../data/ropp_testb.nc (ropp --> bufr --> ropp in t_ropp2bufr) are explained or disappeared. (These have been there since ROPP7.0.)
Note that mo_bufr and ec_bufr differ here anyway:
mo_bufr
INFO (from ropp2bufr): Total of 14742 bytes written to bufr_test.bfr
ec_bufr
INFO (from ropp2bufr): Total of 14741 bytes written to bufr_test.bfr
But both flag a +/- 0.1 m (or multiple thereof) difference between the netCDF versions of the dataset.
This impact params in the test file haven't changed since at least GRIB2ROPP in Sep 2012. We could fix this by simply copy the latest bufr_test.nc to ../data/ropp_testb.nc, but let's decline the temptation for now.
Moving ticket to ROPP10.0.
comment:7 by , 5 years ago
| Milestone: | 10.0 → 11.0 |
|---|
comment:8 by , 3 years ago
See the Suggested solutions in comment:ticket:699:10 for a connection to this work.
comment:9 by , 3 years ago
| Milestone: | 11.0 → 11.1 |
|---|
I have some ideas of how to improve the ropp_io testing, which I hope to include in ROPP-11.1. Changing milestone accordingly.
comment:10 by , 3 years ago
This core tests of the ropp_io module have been overhauled in https://svn.romsaf.org/ropp/ropp_src/branches/dev/Share/ROPP111_make_test.
Key developments:
- Reconcile the outputs from the BUFR encode/decode tools of the three BUFR libs;
- Test as many BUFR tools as the build system can generate (see #688);
- As far as possible, replace tests based on differences in CDL output by checks that differences between fields are less than pre-determined tolerances, and report the results in a PASS/FAIL summary table, as for the other modules;
- Where this is not possible, still arrange for output of results of test to appear in the summary table;
- Add extra tests of reading level 1a data, for GFZ and UCAR data readers (previously we only tested the reading of level 1b/2a data).
- Reduce volume of text written to stdout, but make each test generate a more extensive log file, for later scrutiny if necessary.
All this means that the ropp_io core tests now look very similar to those of the other modules. For example, if all the libraries are available and everything works OK (e.g. for gfortran), running make test gives this output:
Testing netCDF Fortran 90 write interfaces... Running t_netcdf (IO (netCDF basic)) ... **************************** ********** PASS ********** **************************** ... examine t_netcdf.log for details Testing ROPP conversion tool... Running t_ropp2ropp_i (IO (missing/invalid data)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2ropp_i.log for details Running t_ropp2ropp_v (IO (valid data)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2ropp_v.log for details Running t_ropp2ropp_n (IO (netCDF data)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2ropp_n.log for details Running t_ropp2ropp_m1 (IO (single -> multi)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2ropp_m1.log for details Running t_ropp2ropp_m2 (IO (multi -> single)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2ropp_m2.log for details Running t_ropp2ropp_2d (IO (2D data)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2ropp_2d.log for details PASS: 6 tests; NOT PASS: 0 tests; TOTAL: 6 tests Testing BUFR encode/decode [MetDB BUFR library] ... Running t_ropp2bufr_mobufr (IO (ropp2bufr_mobufr)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2bufr_mobufr.log for details Running t_bufr2ropp_mobufr (IO (bufr2ropp_mobufr)) ... **************************** ********** PASS ********** **************************** ... examine t_bufr2ropp_mobufr.log for details Testing BUFR encode/decode [ECMWF BUFR library] ... Running t_ropp2bufr_ecbufr (IO (ropp2bufr_ecbufr)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2bufr_ecbufr.log for details Running t_bufr2ropp_ecbufr (IO (bufr2ropp_ecbufr)) ... **************************** ********** PASS ********** **************************** ... examine t_bufr2ropp_ecbufr.log for details Running t_eum2bufr_ecbufr (IO (eum2bufr_ecbufr)) ... **************************** ********** PASS ********** **************************** ... examine t_eum2bufr_ecbufr.log for details Testing BUFR encode/decode [ecCodes library] ... Running t_ropp2bufr_eccodes (IO (ropp2bufr_eccodes)) ... **************************** ********** PASS ********** **************************** ... examine t_ropp2bufr_eccodes.log for details Running t_bufr2ropp_eccodes (IO (bufr2ropp_eccodes)) ... **************************** ********** PASS ********** **************************** ... examine t_bufr2ropp_eccodes.log for details Running t_eum2bufr_eccodes (IO (eum2bufr_eccodes)) ... **************************** ********** PASS ********** **************************** ... examine t_eum2bufr_eccodes.log for details Testing GFZ-to-netCDF conversion tool on NRT data ... Running t_gfz2ropp_1 (IO (GFZ NRT data)) ... **************************** ********** PASS ********** **************************** ... examine t_gfz2ropp_1.log for details Testing GFZ-to-netCDF conversion tool on PD data ... Running t_gfz2ropp_2 (IO (GFZ PD data)) ... **************************** ********** PASS ********** **************************** ... examine t_gfz2ropp_2.log for details Testing UCAR-to-ROPP netCDF conversion tool on atmPrf data ... Running t_ucar2ropp_1 (IO (UCAR atmPrf data)) ... **************************** ********** PASS ********** **************************** ... examine t_ucar2ropp_1.log for details Testing UCAR-to-ROPP netCDF conversion tool on atmPhs data ... Running t_ucar2ropp_2 (IO (UCAR atmPhs data)) ... **************************** ********** PASS ********** **************************** ... examine t_ucar2ropp_2.log for details Testing EUM netCDF4-to-ROPP conversion tool... Running t_eum2ropp (IO (EUM -> ROPP)) ... **************************** ********** PASS ********** **************************** ... examine t_eum2ropp.log for details Testing [ECMWF ECCODES] GRIB-to-BGRASC conversion tool ... Running t_grib2bgrasc (IO (GRIB -> ASCII)) ... **************************** ********** PASS ********** **************************** ... examine t_grib2bgrasc.log for details Testing BGRASC-to-ROPP conversion tool ... Running t_bgrasc2ropp (IO (ASCII -> ROPP)) ... **************************** ********** PASS ********** **************************** ... examine t_bgrasc2ropp.log for details Making summary of test result ************************** SUMMARY OF ROPP_IO TEST RESULTS *************************** -------------------------------------------------------------------------------------- | Test name | Description | Run? | PASS? | -------------------------------------------------------------------------------------- | t_netcdf | IO (netCDF basic) | Run | PASS | | t_ropp2ropp_i | IO (missing/invalid data) | Run | PASS | | t_ropp2ropp_v | IO (valid data) | Run | PASS | | t_ropp2ropp_n | IO (netCDF data) | Run | PASS | | t_ropp2ropp_m1 | IO (single -> multi) | Run | PASS | | t_ropp2ropp_m2 | IO (multi -> single) | Run | PASS | | t_ropp2ropp_2d | IO (2D data) | Run | PASS | | t_ropp2bufr_mobufr | IO (ropp2bufr_mobufr) | Run | PASS | | t_bufr2ropp_mobufr | IO (bufr2ropp_mobufr) | Run | PASS | | t_ropp2bufr_ecbufr | IO (ropp2bufr_ecbufr) | Run | PASS | | t_bufr2ropp_ecbufr | IO (bufr2ropp_ecbufr) | Run | PASS | | t_eum2bufr_ecbufr | IO (eum2bufr_ecbufr) | Run | PASS | | t_ropp2bufr_eccodes | IO (ropp2bufr_eccodes) | Run | PASS | | t_bufr2ropp_eccodes | IO (bufr2ropp_eccodes) | Run | PASS | | t_eum2bufr_eccodes | IO (eum2bufr_eccodes) | Run | PASS | | t_gfz2ropp_1 | IO (GFZ NRT data) | Run | PASS | | t_gfz2ropp_2 | IO (GFZ PD data) | Run | PASS | | t_ucar2ropp_1 | IO (UCAR atmPrf data) | Run | PASS | | t_ucar2ropp_2 | IO (UCAR atmPhs data) | Run | PASS | | t_eum2ropp | IO (EUM -> ROPP) | Run | PASS | | t_grib2bgrasc | IO (GRIB -> ASCII) | Run | PASS | | t_bgrasc2ropp | IO (ASCII -> ROPP) | Run | PASS | -------------------------------------------------------------------------------------- This table is saved in ropp_io_summary.txt. Each test has its own log file.
This should be compared against the previous version:
Testing netCDF Fortran 90 write interfaces...
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
*** Netcdf: Success ***
Testing ROPP conversion tool...
$NCDUMP = /project/ukmo/rhel7/R/R-3.6.3/bin/ncdump
Test 1: Testing range checks - missing & invalid data --> missing data
======================================================================
---------------------------------------------------------------------
ROPP Test Data Generator
---------------------------------------------------------------------
INFO (from test2ropp): Generating default MISSING data...
INFO (from test2ropp): Profile 1 : OC_99999999999999_UNKN_U999_UNKN
INFO (from test2ropp): Writing ropp_test_1m.nc
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ropp_test_1m.nc
INFO (from ropp2ropp): Profile 1 : OC_99999999999999_UNKN_U999_UNKN
INFO (from ropp2ropp): Writing ropp_test_1m.nc
---------------------------------------------------------------------
ROPP Test Data Generator
---------------------------------------------------------------------
INFO (from test2ropp): Generating default MISSING data...
INFO (from test2ropp): Generating INVALID data...
INFO (from test2ropp): Profile 1 : OC_21729999999999_UNKN_U999_UNKN
INFO (from test2ropp): Writing ropp_test_1i.nc
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ropp_test_1i.nc
INFO (from ropp2ropp): Profile 1 : OC_99999999999999_UNKN_U999_UNKN
INFO (from ropp2ropp): Writing ropp_test_1i.nc
To check results, compare ropp_test_1m.cdl and ropp_test_1i.cdl
(Ignore differences in file names and processing_date)
****************************************
*** Test 1. Range INVALID test: PASS ***
****************************************
Test 2: Testing range checks - valid data --> valid data
========================================================
---------------------------------------------------------------------
ROPP Test Data Generator
---------------------------------------------------------------------
INFO (from test2ropp): Generating default MISSING data...
INFO (from test2ropp): Generating VALID data...
INFO (from test2ropp): Profile 1 : OC_20220120092518_TSRX_G002_GFZ_
INFO (from test2ropp): Writing ropp_test_2v.nc
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ropp_test_2v.nc
INFO (from ropp2ropp): Profile 1 : OC_20220120092518_TSRX_G002_GFZ_
INFO (from ropp2ropp): Writing ropp_test_2o.nc
To check results, compare ropp_test_2v.cdl and ropp_test_2o.cdl
(Ignore differences in file names and processing_date)
**************************************
*** Test 2. Range VALID test: PASS ***
**************************************
Test 3: Testing ROPP (netCDF) --> ROPP (netCDF)
===============================================
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ../data/ropp_test.nc
INFO (from ropp2ropp): Profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from ropp2ropp): Writing ropp_test_4o.nc
To check results, compare ropp_test_4n.cdl and ropp_test_4o.cdl
(Ignore differences in file names & thin_method)
*******************************************
*** Test 3. ROPP netCDF -> netCDF: PASS ***
*******************************************
Test 4: Testing ROPP singlefiles (netCDF) --> ROPP multifile (netCDF)
=====================================================================
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ../data/ropp_test.nc
INFO (from ropp2ropp): Profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from ropp2ropp): Writing ropp_test_5o.nc
INFO (from ropp2ropp): Reading ../data/ropp_test.nc
INFO (from ropp2ropp): Profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from ropp2ropp): Writing ropp_test_5o.nc
INFO (from ropp2ropp): 2 profiles processed
To check results, compare ropp_test_5m.cdl and ropp_test_5o.cdl
(Ignore difference in file names & thin_method))
**************************************************
*** Test 4. ROPP singlefile -> multifile: PASS ***
**************************************************
Test 5: Testing ROPP multifile (netCDF) --> ROPP singlefiles (netCDF)
=====================================================================
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ropp_test_5o.nc
INFO (from ropp2ropp): Profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from ropp2ropp): Writing ropp_test_6o.nc
INFO (from ropp2ropp): Profile 2 : OC_20090817215807_META_G027_DMI_
INFO (from ropp2ropp): Writing oc_20090817215807_meta_g027_dmi_.nc
INFO (from ropp2ropp): 2 profiles processed
To check results, compare ropp_test_6m.cdl and ropp_test_6o.cdl
(Ignore differences in file names & thin_method)
**************************************************
*** Test 5. ROPP multifile -> singlefile: PASS ***
**************************************************
Test 6: Testing ROPP 2D (netCDF) --> ROPP 2D (netCDF)
=====================================================
---------------------------------------------------------------------
ROPP-to-ROPP generic netCDF tool
---------------------------------------------------------------------
INFO (from ropp2ropp): Reading ../data/ropp_test2d.nc
INFO (from ropp2ropp): Profile 1 : UNKNOWN
INFO (from ropp2ropp): Writing ropp_test_7o.nc
To check results, compare ropp_test_7n.cdl and ropp_test_7o.cdl
(Ignore differences in file names & thin_method)
*************************************************
*** Test 6. ROPP 2d netCDF -> 2d netCDF: PASS ***
*************************************************
PASS: 6 NOT PASS: 0 TOTAL: 6 tests
Testing BUFR encode/decode [ECMWF library] ...
$NCDUMP = /project/ukmo/rhel7/R/R-3.6.3/bin/ncdump
1. Encoding ROPP netCDF -> BUFR
===============================
---------------------------------------------------------------------
ROPP to BUFR (ecbufr) Encoder
---------------------------------------------------------------------
INFO (from ropp2bufr): Reading ROPP data from ../data/ropp_test.nc
INFO (from ropp2bufr): Encoding profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from ropp2bufr): Total of 14741 bytes written to bufr_test.bfr
INFO (from ropp2bufr): Generated 1 BUFR message to bufr_test.bfr
2. Comparing BUFR with reference file
=====================================
OK - found no differences in the test BUFR files
3. Decoding BUFR --> ROPP netCDF
================================
---------------------------------------------------------------------
BUFR (ecbufr) to ROPP Decoder
---------------------------------------------------------------------
INFO (from bufr2ropp): Reading bufr_test.bfr
INFO (from bufr2ropp): Decoded profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from bufr2ropp): Writing bufr_test.nc
INFO (from bufr2ropp): 1 new profile saved
4. Comparing CDL dump with reference file
=========================================
/project/ukmo/rhel7/R/R-3.6.3/bin/ncdump ../data/ropp_testb.nc > ropp_test.cdl
/project/ukmo/rhel7/R/R-3.6.3/bin/ncdump bufr_test.nc > bufr_test.cdl
To check results, compare ropp_test.cdl and bufr_test.cdl
(Ignore the difference in file names and processing_date value)
***********************
*** BUFR test: PASS ***
***********************
Testing BUFR encode/decode [EUM netCDF4 files] ...
==============================================================
1. eum2bufr: test eum2bufr by comparing against reference file
==============================================================
--------------------------------------------
1a. Converting EUM netCDF4 file -> BUFR file
--------------------------------------------
../tools/eum2bufr ../data/eum_test.n4 -o eum_test_l.bfr
---------------------------------------------------------------------
EUMETSAT to BUFR (ecbufr) Encoder
---------------------------------------------------------------------
INFO (from eum2bufr): Reading EUM data from ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from eum2bufr): Encoding profile 1 : OC_20120909000057_META_G015_EUME
ECMWF
BUFR ENCODING SOFTWARE VERSION - 7.2
1 April 2007.
Your path for bufr tables is :
/data/users/idculv/ROPP/data/bufr/
BUFR TABLES TO BE LOADED B0000000000000012000.TXT,D0000000000000012000.TXT
INFO (from eum2bufr): Total of 10466 bytes written to eum_test_l.bfr
INFO (from eum2bufr): Generated 1 BUFR message to eum_test_l.bfr
-------------------------------------------------
1b. Comparing output BUFR file and reference file
-------------------------------------------------
Using cmp to compare eum_test_l.bfr and ../data/eum_test_l.bfr_ref
*** No differences found in test BUFR files
******************************
*** eum2bufr test: PASS ***
******************************
===============================================================================
2. eum2bufrb: test eum2bufr by comparing against eum2ropp followed by ropp2bufr
===============================================================================
--------------------------------------------------------------------
2a. Converting EUM netCDF4 file -> BUFR file directly, with eum2bufr
--------------------------------------------------------------------
../tools/eum2bufr ../data/eum_test.n4 -o eum_test_b.bfr
---------------------------------------------------------------------
EUMETSAT to BUFR (ecbufr) Encoder
---------------------------------------------------------------------
INFO (from eum2bufr): Reading EUM data from ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from eum2bufr): Encoding profile 1 : OC_20120909000057_META_G015_EUME
ECMWF
BUFR ENCODING SOFTWARE VERSION - 7.2
1 April 2007.
Your path for bufr tables is :
/data/users/idculv/ROPP/data/bufr/
BUFR TABLES TO BE LOADED B0000000000000012000.TXT,D0000000000000012000.TXT
INFO (from eum2bufr): Total of 10466 bytes written to eum_test_b.bfr
INFO (from eum2bufr): Generated 1 BUFR message to eum_test_b.bfr
-----------------------------------------------------------
2b. Converting EUM netCDF4 file -> ROPP file, with eum2ropp
-----------------------------------------------------------
../tools/eum2ropp ../data/eum_test.n4 -o eum_test_b.nc
---------------------------------------------------------------------
EUMETSAT to ROPP netCDF Converter
---------------------------------------------------------------------
INFO (from eum2ropp): Reading file ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from eum2ropp): Profile 1 : OC_20120909000057_META_G015_EUME
INFO (from eum2ropp): Writing eum_test_b.nc
-----------------------------------------------------
2c. Converting ROPP file to BUFR file, with ropp2bufr
-----------------------------------------------------
../tools/ropp2bufr eum_test_b.nc -o eum_test_b_viaropp.bfr
---------------------------------------------------------------------
ROPP to BUFR (ecbufr) Encoder
---------------------------------------------------------------------
INFO (from ropp2bufr): Reading ROPP data from eum_test_b.nc
INFO (from ropp2bufr): Encoding profile 1 : OC_20120909000057_META_G015_EUME
ECMWF
BUFR ENCODING SOFTWARE VERSION - 7.2
1 April 2007.
Your path for bufr tables is :
/data/users/idculv/ROPP/data/bufr/
BUFR TABLES TO BE LOADED B0000000000000012000.TXT,D0000000000000012000.TXT
INFO (from ropp2bufr): Total of 10466 bytes written to eum_test_b_viaropp.bfr
INFO (from ropp2bufr): Generated 1 BUFR message to eum_test_b_viaropp.bfr
----------------------------------------------
2d. Comparing eum2bufr to eum2ropp | ropp2bufr
----------------------------------------------
Using cmp to compare eum_test_b.bfr and eum_test_b_viaropp.bfr
*** No differences found in test BUFR files
******************************
*** eum2bufrb test: PASS ***
******************************
Testing BUFR encode/decode [ecCodes library] ...
$NCDUMP = /project/ukmo/rhel7/R/R-3.6.3/bin/ncdump
1. Encoding ROPP netCDF -> BUFR
===============================
---------------------------------------------------------------------
ROPP to BUFR (ecCodes) Encoder
---------------------------------------------------------------------
INFO (from ropp2bufr_eccodes): Reading ROPP data from ../data/ropp_test.nc
INFO (from ropp2bufr_eccodes): Encoding profile 1 : OC_20090817215807_META_G027_DMI_
2. Comparing BUFR with reference file
=====================================
OK - found no differences in the test BUFR files
3. Decoding BUFR --> ROPP netCDF
================================
---------------------------------------------------------------------
BUFR (ecCodes) to ROPP Decoder
---------------------------------------------------------------------
INFO (from bufr2ropp_ecodes): Reading bufr_test.bfr
INFO (from bufr2ropp_ecodes): Decoded profile 1 : OC_20090817215807_META_G027_DMI_
INFO (from bufr2ropp_ecodes): Writing bufr_test.nc
INFO (from bufr2ropp_ecodes): 1 new profile saved
4. Comparing CDL dump with reference file
=========================================
To check results, compare ropp_test.cdl and bufr_test.cdl
(Ignore the difference in file names and processing_date value)
*** Differences found; please review bufr_test.dif to check significance.
Differences of +/-1 in the last decimal place can be expected and
these are not considered to be significant.
Testing BUFR encode/decode [EUM netCDF4 files, ecCodes library] ...
==============================================================================
1. eum2bufr_eccodes: test eum2bufr_eccodes by comparing against reference file
==============================================================================
--------------------------------------------
1a. Converting EUM netCDF4 file -> BUFR file
--------------------------------------------
../tools/eum2bufr_eccodes ../data/eum_test.n4 -o eum_test_l.bfr
---------------------------------------------------------------------
EUMETSAT to BUFR (ecCodes) Encoder
---------------------------------------------------------------------
INFO (from eum2bufr_eccodes): Reading EUM data from ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from eum2bufr_eccodes): Encoding profile 1 : OC_20120909000057_META_G015_EUME
-------------------------------------------------
1b. Comparing output BUFR file and reference file
-------------------------------------------------
Using cmp to compare eum_test_l.bfr and ../data/eum_test_l.bfr_ref
*** No differences found in test BUFR files
************************************
*** eum2bufr_eccodes test: PASS ***
************************************
Testing GFZ-to-netCDF conversion tool ...
$NCDUMP = /project/ukmo/rhel7/R/R-3.6.3/bin/ncdump
1. Converting GFZ file pair -> ROPP netCDF
==========================================
---------------------------------------------------------------------
GFZ to ROPP Converter
---------------------------------------------------------------------
INFO (from gfz2ropp): Reading file ../data/gfz_test.dat
INFO (from gfz2ropp): Reading file ../data/gfz_test.dsc
INFO (from gfz2ropp): Profile 1: OC_20130101235611_TSRX_G017_GFZ_
INFO (from gfz2ropp): Writing gfz_test.nc
Generating CDL dumps...
/project/ukmo/rhel7/R/R-3.6.3/bin/ncdump ../data/gfz_test.nc > ../data/gfz_test.cdl
/project/ukmo/rhel7/R/R-3.6.3/bin/ncdump gfz_test.nc > gfz_test.cdl
2. Comparing CDL dump against reference file
============================================
To check results, compare ../data/gfz_test.cdl and gfz_test.cdl
(Ignore the difference in file names and processing_date value)
***************************
*** gfz2ropp test: PASS ***
***************************
Testing UCAR-to-ROPP netCDF conversion tool ...
$NCDUMP = /project/ukmo/rhel7/R/R-3.6.3/bin/ncdump
1. Converting UCAR netCDF -> ROPP netCDF
========================================
---------------------------------------------------------------------
UCAR to ROPP netCDF Converter
---------------------------------------------------------------------
INFO (from ucar2ropp): Reading file ../data/ucar_test.nc
INFO (from ucar2ropp): Profile 1 : OC_20090827202208_C006_G029_UCAR
INFO (from ucar2ropp): Writing ucar_testr.nc
Generating CDL dumps...
/project/ukmo/rhel7/R/R-3.6.3/bin/ncdump ../data/ucar_testr.nc > ../data/ucar_testr.cdl
/project/ukmo/rhel7/R/R-3.6.3/bin/ncdump ucar_testr.nc > ucar_testr.cdl
2. Comparing CDL dump against reference file
============================================
To check results, compare ../data/ucar_testr.cdl and ucar_testr.cdl
(Ignore the difference in file names and processing_date value)
****************************
*** ucar2ropp test: PASS ***
****************************
Testing [ECMWF ECCODES] GRIB-to-BGRASC conversion tool ...
1. Converting ECMWF GRIB -> Fortran namelist ascii
==================================================
../tools/grib2bgrasc ../data/fc_20121001000000_T+6.grib -lat 50.72 -lon -3.53 -date 20121001 -time 0000
-g ../data/fc_20121001000000_T+12.grib -z ../data/an_20121001000000_T+0.grib -o hy_20121001000000_T+0.nml
---------------------------------------------------------------------
GRIB to background ascii converter
---------------------------------------------------------------------
INFO (from grib2bgrasc): Reading file ../data/fc_20121001000000_T+6.grib
INFO (from grib2bgrasc): Reading file ../data/fc_20121001000000_T+12.grib
INFO (from grib2bgrasc): Reading file ../data/an_20121001000000_T+0.grib
INFO (from grib2bgrasc): Writing file hy_20121001000000_T+0.nml
2. Comparing output namelist with reference file
================================================
INFO (from nml_diff): Using nml_diff to compare hy_20121001000000_T+0.nml and ../data/hy_20121001000000_T+0.nml_ref
INFO (from nml_diff): No significant differences between hy_20121001000000_T+0.nml and ../data/hy_20121001000000_T+0.nml_ref
******************************
*** grib2bgrasc test: PASS ***
******************************
Testing BGRASC-to-ROPP conversion tool ...
1. Converting Fortran namelist ascii -> ROPP netCDF
===================================================
../tools/bgrasc2ropp ../data/hy_20121001000000_T+0.nml_ref -o hy_20121001000000_T+0.nc
---------------------------------------------------------------------
bgrasc to ROPP converter
---------------------------------------------------------------------
INFO (from bgrasc2ropp): Reading file ../data/hy_20121001000000_T+0.nml_ref
INFO (from bgrasc2ropp): Writing hy_20121001000000_T+0.nc
2. Comparing output ROPP file with reference file
=================================================
INFO (from nc_diff): Using nc_diff to compare hy_20121001000000_T+0.nc and ../data/hy_20121001000000_T+0.nc_ref
INFO (from nc_diff): No significant differences between hy_20121001000000_T+0.nc and ../data/hy_20121001000000_T+0.nc_ref
******************************
*** bgrasc2ropp test: PASS ***
******************************
Testing EUM netCDF4-to-ROPP conversion tool...
==============================================================
1. eum2ropp: test eum2ropp by comparing against reference file
==============================================================
--------------------------------------------
1a. Converting EUM netCDF4 file -> ROPP file
--------------------------------------------
../tools/eum2ropp ../data/eum_test.n4 -l cl+rs -o eum_test_l.nc
---------------------------------------------------------------------
EUMETSAT to ROPP netCDF Converter
---------------------------------------------------------------------
INFO (from eum2ropp): Reading file ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from ropp_io_read_eumdata): raw_sampling data requested but are not available in file
INFO (from eum2ropp): Profile 1 : OC_20120909000057_META_G015_EUME
INFO (from eum2ropp): Writing eum_test_l.nc
-------------------------------------------------
1b. Comparing output ROPP file and reference file
-------------------------------------------------
INFO (from nc_diff): Using nc_diff to compare eum_test_l.nc and ../data/eum_test_l.nc_ref
INFO (from nc_diff): No significant differences between eum_test_l.nc and ../data/eum_test_l.nc_ref
******************************
*** eum2ropp test: PASS ***
******************************
===============================================================================
2. eum2roppb: test eum2ropp by comparing against eum2bufr followed by bufr2ropp
===============================================================================
--------------------------------------------------------------------
2a. Converting EUM netCDF4 file -> ROPP file directly, with eum2ropp
--------------------------------------------------------------------
../tools/eum2ropp ../data/eum_test.n4 -o eum_test_b.nc
---------------------------------------------------------------------
EUMETSAT to ROPP netCDF Converter
---------------------------------------------------------------------
INFO (from eum2ropp): Reading file ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from eum2ropp): Profile 1 : OC_20120909000057_META_G015_EUME
INFO (from eum2ropp): Writing eum_test_b.nc
-----------------------------------------------------------
2b. Converting EUM netCDF4 file -> BUFR file, with eum2bufr
-----------------------------------------------------------
../tools/eum2bufr ../data/eum_test.n4 -o eum_test_b.bfr
---------------------------------------------------------------------
EUMETSAT to BUFR (ecbufr) Encoder
---------------------------------------------------------------------
INFO (from eum2bufr): Reading EUM data from ../data/eum_test.n4
INFO (from ropp_io_read_eumdata): Format Version 11.0
INFO (from eum2bufr): Encoding profile 1 : OC_20120909000057_META_G015_EUME
ECMWF
BUFR ENCODING SOFTWARE VERSION - 7.2
1 April 2007.
Your path for bufr tables is :
/data/users/idculv/ROPP/data/bufr/
BUFR TABLES TO BE LOADED B0000000000000012000.TXT,D0000000000000012000.TXT
INFO (from eum2bufr): Total of 10466 bytes written to eum_test_b.bfr
INFO (from eum2bufr): Generated 1 BUFR message to eum_test_b.bfr
-----------------------------------------------------
2c. Converting BUFR file to ROPP file, with bufr2ropp
-----------------------------------------------------
../tools/bufr2ropp eum_test_b.bfr -o eum_test_b_viabufr.nc
---------------------------------------------------------------------
BUFR (ecbufr) to ROPP Decoder
---------------------------------------------------------------------
INFO (from bufr2ropp): Reading eum_test_b.bfr
ECMWF
BUFR DECODING SOFTWARE VERSION - 7.2
1 APRIL 2007.
Your path for bufr tables is :
/data/users/idculv/ROPP/data/bufr/
BUFR TABLES TO BE LOADED B0000000000000012000.TXT,D0000000000000012000.TXT
INFO (from bufr2ropp): Decoded profile 1 : OC_20120909000057_META_G015_EUME
INFO (from bufr2ropp): Writing eum_test_b_viabufr.nc
INFO (from bufr2ropp): 1 new profile saved
----------------------------------------------
2d. Comparing eum2ropp to eum2bufr | bufr2ropp
----------------------------------------------
INFO (from nc_diff): Using nc_diff to compare eum_test_b.nc and eum_test_b_viabufr.nc
INFO (from nc_diff): No significant differences between eum_test_b.nc and eum_test_b_viabufr.nc
******************************
*** eum2roppb test: PASS ***
******************************
I think that's a big improvement for the general ROPP user.
comment:11 by , 3 years ago
Part of this work has involved the construction of a new tool, robufr2ropp, which emulates bufr2ropp without the need for BUFR libraries (!). This allows us to compare the results of ropp2bufr or eum2bufr without the rather incestuous reliance on bufr2ropp, which will have been built using the same libraries and (some of the same) routines that are being tested.
We have also removed the circular and convoluted tests that eum2bufr = eum2ropp | ropp2bufr etc. It's cleaner to test each tool in isolation.
A version of robufr2ropp.f90 is attached. The ROPP version can be found in ropp_io/tests. It works by reading and translating the bytes in BUFR Secs 1-3 and 5, and by chopping up the long bit stream that constitutes Sec 4 into chunks (one chunk per datum) and then translating each chunk from binary into floating point format, so that it can be part of an ROprof structure. The size of each chunk, and the number of chunks (which depends on the number of frequencies, the number of vertical levels and the number of available data 'types' (lev 2a etc)) are taken from the ROM SAF BUFR document (http://www.romsaf.org/romsaf_bufr.pdf).
comment:12 by , 3 years ago
(The hard-wired numbers in robufr2ropp.f90 will need adjusting when the RO BUFR format changes, but that's the price to be paid for avoiding BUFR libraries.)
comment:13 by , 3 years ago
It all checks out OK when passed through all available compilers (gfortran, ifort17, ifort16, pgf95, g95, nagfor61 and ftn). Note that when make test is tried on the nagfor61 build, the final summary table says
************************** SUMMARY OF ROPP_IO TEST RESULTS *************************** -------------------------------------------------------------------------------------- | Test name | Description | Run? | PASS? | -------------------------------------------------------------------------------------- | t_netcdf | IO (netCDF basic) | Run | PASS | | t_ropp2ropp_i | IO (missing/invalid data) | Run | PASS | | t_ropp2ropp_v | IO (valid data) | Run | PASS | | t_ropp2ropp_n | IO (netCDF data) | Run | PASS | | t_ropp2ropp_m1 | IO (single -> multi) | Run | PASS | | t_ropp2ropp_m2 | IO (multi -> single) | Run | PASS | | t_ropp2ropp_2d | IO (2D data) | Run | PASS | | t_ropp2bufr_mobufr | IO (ropp2bufr_mobufr) | Run | PASS | | t_bufr2ropp_mobufr | IO (bufr2ropp_mobufr) | Run | PASS | | t_ropp2bufr_ecbufr | IO (ropp2bufr_ecbufr) | Not run | ------ | | t_bufr2ropp_ecbufr | IO (bufr2ropp_ecbufr) | Not run | ------ | | t_eum2bufr_ecbufr | IO (eum2bufr_ecbufr) | Not run | ------ | | t_ropp2bufr_eccodes | IO (ropp2bufr_eccodes) | Not run | ------ | | t_bufr2ropp_eccodes | IO (bufr2ropp_eccodes) | Not run | ------ | | t_eum2bufr_eccodes | IO (eum2bufr_eccodes) | Not run | ------ | | t_gfz2ropp_1 | IO (GFZ NRT data) | Run | PASS | | t_gfz2ropp_2 | IO (GFZ PD data) | Run | PASS | | t_ucar2ropp_1 | IO (UCAR atmPrf data) | Run | PASS | | t_ucar2ropp_2 | IO (UCAR atmPhs data) | Run | PASS | | t_eum2ropp | IO (EUM -> ROPP) | Run | PASS | | t_grib2bgrasc | IO (GRIB -> ASCII) | Not run | ------ | | t_bgrasc2ropp | IO (ASCII -> ROPP) | Run | PASS | --------------------------------------------------------------------------------------
This is as it should be, because nagfor61 cannot build the ecbufr, eccodes or grib_api libraries, so tools that depend on them cannot be built, and tests of those tools are not run.
comment:14 by , 3 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
I think we can probably consider this work complete now, so closing ticket.
by , 3 years ago
| Attachment: | test4_000_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_001_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_010_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_011_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_100_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_101_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_110_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_111_gfortran.out added |
|---|
by , 3 years ago
| Attachment: | test4_000_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_001_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_010_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_011_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_100_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_101_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_110_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_111_ifort17.out added |
|---|
by , 3 years ago
| Attachment: | test4_000_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_001_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_010_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_011_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_100_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_101_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_110_nagfor61.out added |
|---|
by , 3 years ago
| Attachment: | test4_111_nagfor61.out added |
|---|
comment:15 by , 3 years ago
For completeness, I've attached test4.sh, which is the test script that builds the execs and runs through the tests. Also attached are the logs of the results of those tests, called test4_${mobufr}${ecbufr}${eccodes}_${comp}.out, where $mobufr=1 if the metDB BUFR library is available and 0 if not. Similarly for the ECMWF BUFRDC and ecCodes libraries. The log files show that everything is working as planned.
I've also run buildpack ropp_io successfully through all the available compilers, namely (gfortran, ifort17, ifort16, pgf95, g95, nagfor61, ftn), with as many libraries available as possible. Logs also attached. These show that robufr2ropp.f90 is written in reasonably robust fortran.
by , 3 years ago
| Attachment: | buildpack_ropp_io_ftn added |
|---|
by , 3 years ago
| Attachment: | buildpack_ropp_io_g95 added |
|---|
by , 3 years ago
| Attachment: | buildpack_ropp_io_gfortran added |
|---|
by , 3 years ago
| Attachment: | buildpack_ropp_io_ifort16 added |
|---|
by , 3 years ago
| Attachment: | buildpack_ropp_io_ifort17 added |
|---|
by , 3 years ago
| Attachment: | buildpack_ropp_io_nagfor61 added |
|---|
by , 3 years ago
| Attachment: | buildpack_ropp_io_pgf95 added |
|---|
Note:
See TracTickets
for help on using tickets.
Made some progress, but got distracted by something interesting. Shouldn't be too difficult, but lots of checking needed, and impact on test folder needs to be assessed. Therefore relegating to ROPP8.0.