Use /bin/bash in shell scripts

[Ian Culverwell]

ROPP6.0 beta reviewer had problems building scripts on a Debian platform, for which /bin/sh --> /bin/dash. He suggests hard-wiring

#!/bin/bash

at the top of all test scripts.

(Some scripts use

!@SHELL@

instead.)

[Dave Offiler]

The !@SHELL@ form is a macro in a template shell script which is expanded to the shell detected by the package configure via make test to generate the actual test script which is then run. In rare cases, the expanded shell may not be compatible with the script's coding. Hard-coding the 'correct' shell makes the assumptions that Bash is (a) installed on the platform and (b) if so, is installed in /bin (although this is normal, it may not be universal). Therefore hard-coding could cause more problems than it solves.

For the longer term, we should review whether it is necessary or desirable to use such templates at all, or just provide the final run-time script in the ROPP package.

[Dave Offiler]

There are two classes of shell scripts:

1) Static scripts:

• build*, makedistro, etc in ROPP_SRC root directory. All use /bin/bash anyway.
• ./configure/*_configure_* miniscripts: all currently have /bin/sh, now changed to /bin/bash in branch do_exitcodes.

2) Dynamic scripts:

• all the ropp_*/tests/t_*.sh template scripts which the Makefile turns into the same name without .sh. The template #!@SHELL@ is mapped to the current shell. This appears to be the shell in which the main ./configure was running when it generated the final Makefile from Makefile.in (the template Makefile.am and intermediate Makefile.in don't specify any shell, and neither does configure.ac). Since my login shell is /bin/ksh, the autotools must be setting up configure (and the SHELL variable) to /bin/sh. Whether that's fixed and assumed every system has it, or it uses whatever it finds in some search order, I don't know, but it's best not tinkered with.

Therefore, I suggest we change all the static scripts but leave the dynamic ones as-is on the assumption that some appropriate (installed) shell will be used on the target system; this may be bash or something else. As long as our test scripts don't use shell-specific feature and are generic enough, they should run on most, if not all, common shells.

In short, for the beta-tester's specific suggestion, this is a "won't fix" on the grounds that we're bound to break someone else's build by hard-coding /bin/bash.

[Ian Culverwell]

In more detail, the ROPP6.0 beta reviewer's thoughts were:

Inconsistent use of shells

It was observed that the test scripts failed on those platforms, which had the 
/bin/sh linked to something else but /bin/bash, e.g. , the current Debian
distributions have /bin/sh -> /bin/dash. On a machine, which has /bin/sh -> 
/bin/dash, the scripts get interpreted by the dash, resulting in a test failure and 
an error message like
...
./test_fm_2D.sh: 144: [[: not found
Output file ECMWF_10_OCCS_out.nc was not created - test failed.

In some test scripts, the first line reads #!/bin/sh, in others, like in the script 
file ropp-6.0/ropp_pp/tests/t_ropp_pp.sh it reads !@SHELL@. These settings are 
problematic and may cause an error during the execution of the test scripts, if 
either the user’s default shell is not the bash or if the system specific /bin/sh is 
linked to any non-bash shell.

The only way to avoid such problems is to explicitly specify the shell which should 
be used: in case of the test scripts, it is the bash and therefore there should be a 
#!/bin/bash in every first line.

This would be fine, if we could rely on bash always being present. The better solution, surely, is to write shell-independent scripts, which avoid constructions like 'if ... ; then ...'. This would appear to be less risky than overruling the autotools' cleverness.

This invites the prospect of testing our scripts on every shell we can find at each release - an invitation I suggest we politely decline. But maybe occasionally? And for entirely new scripts?

[Dave Offiler]

In Makefiles - which might run any old shell, I avoid ' .. ' syntax and use only single brackets, for instance. If we follow basic syntax which ought to work in all the common shells, then this would be preferable to mucking about with SHELL or assuming/hardcoding bash. So we could start by removing any ' .. ' from all scripts not explicitly using /bin/bash - which is in practice just those in the tests directories with template @SHELL@ hash-bangs.

At the end of the day, we support a sub-set of all the possible operating systems and all their possible compilers (and sub-version of those). I see no problem in also supporting only a limited number of shells, viz. bash & ksh [I see on RHEL6 that /bin/sh => /bin/bash]). By 'supporting' I mean testing under that shell; if other shells happen to work anyway, that's a bonus.

If our user can't change their linkage for /bin/sh from dash to bash, then we can't do the reverse to test under dash!

Our defence is: "The @SHELL@ template is the way the autotools works, and we're not going to interfere with that universally used system of software building, but we'll modify the syntax used in ROPP scripts to maximise portability between common POSIX shells, but support for all shells cannot be guaranteed."

[Ian Culverwell]

I've done the first of these in revisions r4183 - r4186.

[Dave Offiler]

Closing a fixed (as well as we can make it).