(autoconf.info)Limitations of Builtins

Next: Limitations of Usual Tools Prev: Shell Functions Up: Portable Shell

10.12 Limitations of Shell Builtins

No, no, we are serious: some shells do have limitations!  :)

   You should always keep in mind that any builtin or command may
support options, and therefore differ in behavior with arguments
starting with a dash.  For instance, the innocent `echo "$word"' can
give unexpected results when `word' starts with a dash.  It is often
possible to avoid this problem using `echo "x$word"', taking the `x'
into account later in the pipe.

     Use `.' only with regular files (use `test -f').  Bash 2.03, for
     instance, chokes on `. /dev/null'.  Remember that `.' uses `PATH'
     if its argument contains no slashes.  Also, some shells, including
     bash 3.2, implicitly append the current directory to this `PATH'
     search, even though Posix forbids it.  So if you want to use `.'
     on a file `foo' in the current directory, you must use `. ./foo'.

     The Unix version 7 shell did not support negating the exit status
     of commands with `!', and this feature is still absent from some
     shells (e.g., Solaris `/bin/sh').  Shell code like this:

          if ! cmp file1 file2 >/dev/null 2>&1; then
            echo files differ or trouble

     is therefore not portable in practice.  Typically it is easy to
     rewrite such code, e.g.:

          cmp file1 file2 >/dev/null 2>&1 ||
            echo files differ or trouble

     More generally, one can always rewrite `! COMMAND' as:

          if COMMAND; then (exit 1); else :; fi

     Bash 3.2 (and earlier versions) sometimes does not properly set
     `$?' when failing to write redirected output of a compound command.
     This problem is most commonly observed with `{...}'; it does not
     occur with `(...)'.  For example:

          $ bash -c '{ echo foo; } >/bad; echo $?'
          bash: line 1: /bad: Permission denied
          $ bash -c 'while :; do echo; done >/bad; echo $?'
          bash: line 1: /bad: Permission denied

     To work around the bug, prepend `:;':

          $ bash -c ':;{ echo foo; } >/bad; echo $?'
          bash: line 1: /bad: Permission denied

     The use of `break 2' etc. is safe.

     You don't need to quote the argument; no splitting is performed.

     You don't need the final `;;', but you should use it.

     Posix requires support for `case' patterns with opening
     parentheses like this:

          case $file_name in
          (*.c) echo "C source code";;

     but the `(' in this example is not portable to many Bourne shell
     implementations.  It can be omitted safely.

     Zsh handles pattern fragments derived from parameter expansions or
     command substitutions as though quoted:

          $ pat=\?; case aa in ?$pat) echo match;; esac
          $ pat=\?; case a? in ?$pat) echo match;; esac

     Because of a bug in its `fnmatch', Bash fails to properly handle
     backslashes in character classes:

          bash-2.02$ case /tmp in [/\\]*) echo OK;; esac

     This is extremely unfortunate, since you are likely to use this
     code to handle Posix or MS-DOS absolute file names.  To work
     around this bug, always put the backslash first:

          bash-2.02$ case '\TMP' in [\\/]*) echo OK;; esac
          bash-2.02$ case /tmp in [\\/]*) echo OK;; esac

     Many Bourne shells cannot handle closing brackets in character
     classes correctly.

     Some shells also have problems with backslash escaping in case you
     do not want to match the backslash: both a backslash and the
     escaped character match this pattern.  To work around this,
     specify the character class in a variable, so that quote removal
     does not apply afterwards, and the special characters don't have
     to be backslash-escaped:

          $ case '\' in [\<]) echo OK;; esac
          $ scanset='[<]'; case '\' in $scanset) echo OK;; esac

     Even with this, Solaris `ksh' matches a backslash if the set
     contains any of the characters `|', `&', `(', or `)'.

     Conversely, Tru64 `ksh' (circa 2003) erroneously always matches a
     closing parenthesis if not specified in a character class:

          $ case foo in *\)*) echo fail ;; esac
          $ case foo in *')'*) echo fail ;; esac

     Some shells, such as Ash 0.3.8, are confused by an empty

          ash-0.3.8 $ case foo in esac;
          error-->Syntax error: ";" unexpected (expecting ")")

     Many shells still do not support parenthesized cases, which is a
     pity for those of us using tools that rely on balanced
     parentheses.  For instance, Solaris `/bin/sh':

          $ case foo in (foo) echo foo;; esac
          error-->syntax error: `(' unexpected

     Posix 1003.1-2001 requires that `cd' must support the `-L'
     ("logical") and `-P' ("physical") options, with `-L' being the
     default.  However, traditional shells do not support these
     options, and their `cd' command has the `-P' behavior.

     Portable scripts should assume neither option is supported, and
     should assume neither behavior is the default.  This can be a bit
     tricky, since the Posix default behavior means that, for example,
     `ls ..' and `cd ..' may refer to different directories if the
     current logical directory is a symbolic link.  It is safe to use
     `cd DIR' if DIR contains no `..' components.  Also,
     Autoconf-generated scripts check for this problem when computing
     variables like `ac_top_srcdir' (Note: Configuration Actions), so
     it is safe to `cd' to these variables.

     See Note: Special Shell Variables, for portability problems
     involving `cd' and the `CDPATH' environment variable.  Also please
     see the discussion of the `pwd' command.

     The simple `echo' is probably the most surprising source of
     portability troubles.  It is not possible to use `echo' portably
     unless both options and escape sequences are omitted.  New
     applications which are not aiming at portability should use
     `printf' instead of `echo'.

     Don't expect any option.  Note: Preset Output Variables, `ECHO_N'
     etc. for a means to simulate `-n'.

     Do not use backslashes in the arguments, as there is no consensus
     on their handling.  For `echo '\n' | wc -l', the `sh' of Solaris
     outputs 2, but Bash and Zsh (in `sh' emulation mode) output 1.
     The problem is truly `echo': all the shells understand `'\n'' as
     the string composed of a backslash and an `n'.

     Because of these problems, do not pass a string containing
     arbitrary characters to `echo'.  For example, `echo "$foo"' is safe
     if you know that FOO's value cannot contain backslashes and cannot
     start with `-', but otherwise you should use a here-document like

          cat <<EOF

     The `eval' command is useful in limited circumstances, e.g., using
     commands like `eval table_$key=\$value' and `eval
     value=table_$key' to simulate a hash table when the key is known
     to be alphanumeric.  However, `eval' is tricky to use on arbitrary
     arguments, even when it is implemented correctly.

     It is obviously unwise to use `eval $cmd' if the string value of
     `cmd' was derived from an untrustworthy source.  But even if the
     string value is valid, `eval $cmd' might not work as intended,
     since it causes field splitting and file name expansion to occur
     twice, once for the `eval' and once for the command itself.  It is
     therefore safer to use `eval "$cmd"'.  For example, if CMD has the
     value `cat test?.c', `eval $cmd' might expand to the equivalent of
     `cat test;.c' if there happens to be a file named `test;.c' in the
     current directory; and this in turn mistakenly attempts to invoke
     `cat' on the file `test' and then execute the command `.c'.  To
     avoid this problem, use `eval "$cmd"' rather than `eval $cmd'.

     However, suppose that you want to output the text of the evaluated
     command just before executing it.  Assuming the previous example,
     `echo "Executing: $cmd"' outputs `Executing: cat test?.c', but
     this output doesn't show the user that `test;.c' is the actual name
     of the copied file.  Conversely, `eval "echo Executing: $cmd"'
     works on this example, but it fails with `cmd='cat foo >bar'',
     since it mistakenly replaces the contents of `bar' by the string
     `cat foo'.  No simple, general, and portable solution to this
     problem is known.

     You should also be wary of common bugs in `eval' implementations.
     In some shell implementations (e.g., older `ash', OpenBSD 3.8
     `sh', `pdksh' v5.2.14 99/07/13.2, and `zsh' 4.2.5), the arguments
     of `eval' are evaluated in a context where `$?' is 0, so they
     exhibit behavior like this:

          $ false; eval 'echo $?'

     The correct behavior here is to output a nonzero value, but
     portable scripts should not rely on this.

     You should not rely on `LINENO' within `eval'.  Note: Special
     Shell Variables.

     The default value of `exit' is supposed to be `$?'; unfortunately,
     some shells, such as the DJGPP port of Bash 2.04, just perform
     `exit 0'.

          bash-2.04$ foo=`exit 1` || echo fail
          bash-2.04$ foo=`(exit 1)` || echo fail
          bash-2.04$ foo=`(exit 1); exit` || echo fail

     Using `exit $?' restores the expected behavior.

     Some shell scripts, such as those generated by `autoconf', use a
     trap to clean up before exiting.  If the last shell command exited
     with nonzero status, the trap also exits with nonzero status so
     that the invoker can tell that an error occurred.

     Unfortunately, in some shells, such as Solaris `/bin/sh', an exit
     trap ignores the `exit' command's argument.  In these shells, a
     trap cannot determine whether it was invoked by plain `exit' or by
     `exit 1'.  Instead of calling `exit' directly, use the
     `AC_MSG_ERROR' macro that has a workaround for this problem.

     The builtin `export' dubs a shell variable "environment variable".
     Each update of exported variables corresponds to an update of the
     environment variables.  Conversely, each environment variable
     received by the shell when it is launched should be imported as a
     shell variable marked as exported.

     Alas, many shells, such as Solaris `/bin/sh', IRIX 6.3, IRIX 5.2,
     AIX 4.1.5, and Digital Unix 4.0, forget to `export' the
     environment variables they receive.  As a result, two variables
     coexist: the environment variable and the shell variable.  The
     following code demonstrates this failure:

          echo $FOO
          echo $FOO
          exec /bin/sh $0

     when run with `FOO=foo' in the environment, these shells print
     alternately `foo' and `bar', although they should print only `foo'
     and then a sequence of `bar's.

     Therefore you should `export' again each environment variable that
     you update.

     Don't expect `false' to exit with status 1: in native Solaris
     `/bin/false' exits with status 255.

     To loop over positional arguments, use:

          for arg
            echo "$arg"

     You may _not_ leave the `do' on the same line as `for', since some
     shells improperly grok:

          for arg; do
            echo "$arg"

     If you want to explicitly refer to the positional arguments, given
     the `$@' bug (Note: Shell Substitutions), use:

          for arg in ${1+"$@"}; do
            echo "$arg"

     But keep in mind that Zsh, even in Bourne shell emulation mode,
     performs word splitting on `${1+"$@"}'; see Note: Shell
     Substitutions, item `$@', for more.

     Using `!' is not portable.  Instead of:

          if ! cmp -s file file.new; then
            mv file.new file


          if cmp -s file file.new; then :; else
            mv file.new file

     There are shells that do not reset the exit status from an `if':

          $ if (exit 42); then true; fi; echo $?

     whereas a proper shell should have printed `0'.  This is especially
     bad in makefiles since it produces false failures.  This is why
     properly written makefiles, such as Automake's, have such hairy

          if test -f "$file"; then
            install "$file" "$dest"

     A format string starting with a `-' can cause problems.  Bash
     interprets it as an option and gives an error.  And `--' to mark
     the end of options is not good in the NetBSD Almquist shell (e.g.,
     0.4.6) which takes that literally as the format string.  Putting
     the `-' in a `%c' or `%s' is probably easiest:

          printf %s -foo

     Bash 2.03 mishandles an escape sequence that happens to evaluate
     to `%':

          $ printf '\045'
          bash: printf: `%': missing format character

     Large outputs may cause trouble.  On Solaris 2.5.1 through 10, for
     example, `/usr/bin/printf' is buggy, so when using `/bin/sh' the
     command `printf %010000x 123' normally dumps core.

     Not all shells support `-r' (Solaris `/bin/sh' for example).

     With modern shells, plain `pwd' outputs a "logical" directory
     name, some of whose components may be symbolic links.  These
     directory names are in contrast to "physical" directory names,
     whose components are all directories.

     Posix 1003.1-2001 requires that `pwd' must support the `-L'
     ("logical") and `-P' ("physical") options, with `-L' being the
     default.  However, traditional shells do not support these
     options, and their `pwd' command has the `-P' behavior.

     Portable scripts should assume neither option is supported, and
     should assume neither behavior is the default.  Also, on many hosts
     `/bin/pwd' is equivalent to `pwd -P', but Posix does not require
     this behavior and portable scripts should not rely on it.

     Typically it's best to use plain `pwd'.  On modern hosts this
     outputs logical directory names, which have the following

        * Logical names are what the user specified.

        * Physical names may not be portable from one installation host
          to another due to network file system gymnastics.

        * On modern hosts `pwd -P' may fail due to lack of permissions
          to some parent directory, but plain `pwd' cannot fail for this

     Also please see the discussion of the `cd' command.

     With the FreeBSD 6.0 shell, the `set' command (without any
     options) does not sort its output.

     The `set' builtin faces the usual problem with arguments starting
     with a dash.  Modern shells such as Bash or Zsh understand `--' to
     specify the end of the options (any argument after `--' is a
     parameter, even `-x' for instance), but many traditional shells
     (e.g., Solaris 10 `/bin/sh') simply stop option processing as soon
     as a non-option argument is found.  Therefore, use `dummy' or
     simply `x' to end the option processing, and use `shift' to pop it

          set x $my_list; shift

     Avoid `set -', e.g., `set - $my_list'.  Posix no longer requires
     support for this command, and in traditional shells `set -
     $my_list' resets the `-v' and `-x' options, which makes scripts
     harder to debug.

     Some nonstandard shells do not recognize more than one option
     (e.g., `set -e -x' assigns `-x' to the command line).  It is
     better to combine them:

          set -ex

     The BSD shell has had several problems with the `-e' option,
     partly because BSD `make' traditionally used `-e' even though this
     was incompatible with Posix (Note: Failure in Make Rules).
     Older versions of the BSD shell (circa 1990) mishandled `&&',
     `||', `if', and `case' when `-e' was in effect, causing the shell
     to exit unexpectedly in some cases.  This was particularly a
     problem with makefiles, and led to circumlocutions like `sh -c
     'test -f file || touch file'', where the seemingly-unnecessary `sh
     -c '...'' wrapper works around the bug.

     Even relatively-recent versions of the BSD shell (e.g., OpenBSD
     3.4) wrongly exit with `-e' if a command within `&&' fails inside
     a compound statement.  For example:

          #! /bin/sh
          set -e
          test -n "$foo" && exit 1
          echo one
          if :; then
            test -n "$foo" && exit 1
          echo two

     does not print `two'.  One workaround is to use `if test -n
     "$foo"; then exit 1; fi' rather than `test -n "$foo" && exit 1'.
     Another possibility is to warn BSD users not to use `sh -e'.

     Not only is `shift'ing a bad idea when there is nothing left to
     shift, but in addition it is not portable: the shell of MIPS
     RISC/OS 4.52 refuses to do it.

     Don't use `shift 2' etc.; it was not in the 7th Edition Bourne
     shell, and it is also absent in many pre-Posix shells.

     This command is not portable, as Posix does not require it; use
     `.' instead.

     The `test' program is the way to perform many file and string
     tests.  It is often invoked by the alternate name `[', but using
     that name in Autoconf code is asking for trouble since it is an M4
     quote character.

     The `-a', `-o', `(', and `)' operands are not portable and should
     be avoided.  Thus, portable uses of `test' should never have more
     than four arguments, and scripts should use shell constructs like
     `&&' and `||' instead.  If you combine `&&' and `||' in the same
     statement, keep in mind that they have equal precedence, so it is
     often better to parenthesize even when this is redundant.  For

          # Not portable:
          test "X$a" = "X$b" -a \
            '(' "X$c" != "X$d" -o "X$e" = "X$f" ')'

          # Portable:
          test "X$a" = "X$b" &&
            { test "X$c" != "X$d" || test "X$e" = "X$f"; }

     `test' does not process options like most other commands do; for
     example, it does not recognize the `--' argument as marking the
     end of options.

     It is safe to use `!' as a `test' operator.  For example, `if test
     ! -d foo; ...' is portable even though `if ! test -d foo; ...' is

`test' (files)
     To enable `configure' scripts to support cross-compilation, they
     shouldn't do anything that tests features of the build system
     instead of the host system.  But occasionally you may find it
     necessary to check whether some arbitrary file exists.  To do so,
     use `test -f' or `test -r'.  Do not use `test -x', because 4.3BSD
     does not have it.  Do not use `test -e' either, because Solaris
     `/bin/sh' lacks it.  To test for symbolic links on systems that
     have them, use `test -h' rather than `test -L'; either form
     conforms to Posix 1003.1-2001, but older shells like Solaris 8
     `/bin/sh' support only `-h'.

`test' (strings)
     Posix says that `test "STRING"' succeeds if STRING is not null,
     but this usage is not portable to traditional platforms like
     Solaris 10 `/bin/sh', which mishandle strings like `!' and `-n'.

     Posix also says that `test ! "STRING"', `test -n "STRING"' and
     `test -z "STRING"' work with any string, but many shells (such as
     Solaris, AIX 3.2, UNICOS, Digital Unix 4, etc.) get
     confused if STRING looks like an operator:

          $ test -n =
          test: argument expected
          $ test ! -n
          test: argument expected

     Similarly, Posix says that both `test "STRING1" = "STRING2"' and
     `test "STRING1" != "STRING2"' work for any pairs of strings, but
     in practice this is not true for troublesome strings that look
     like operators or parentheses, or that begin with `-'.

     It is best to protect such strings with a leading `X', e.g., `test
     "XSTRING" != X' rather than `test -n "STRING"' or `test !

     It is common to find variations of the following idiom:

          test -n "`echo $ac_feature | sed 's/[-a-zA-Z0-9_]//g'`" &&

     to take an action when a token matches a given pattern.  Such
     constructs should be avoided by using:

          case $ac_feature in
            *[!-a-zA-Z0-9_]*) ACTION;;

     If the pattern is a complicated regular expression that cannot be
     expressed as a shell pattern, use something like this instead:

          expr "X$ac_feature" : 'X.*[^-a-zA-Z0-9_]' >/dev/null &&

     `expr "XFOO" : "XBAR"' is more robust than `echo "XFOO" | grep
     "^XBAR"', because it avoids problems when `FOO' contains

     It is safe to trap at least the signals 1, 2, 13, and 15.  You can
     also trap 0, i.e., have the `trap' run when the script ends
     (either via an explicit `exit', or the end of the script).  The
     trap for 0 should be installed outside of a shell function, or AIX
     5.3 `/bin/sh' will invoke the trap at the end of this function.

     Posix says that `trap - 1 2 13 15' resets the traps for the
     specified signals to their default values, but many common shells
     (e.g., Solaris `/bin/sh') misinterpret this and attempt to execute
     a "command" named `-' when the specified conditions arise.  There
     is no portable workaround, except for `trap - 0', for which `trap
     '' 0' is a portable substitute.

     Although Posix is not absolutely clear on this point, it is widely
     admitted that when entering the trap `$?' should be set to the exit
     status of the last command run before the trap.  The ambiguity can
     be summarized as: "when the trap is launched by an `exit', what is
     the _last_ command run: that before `exit', or `exit' itself?"

     Bash considers `exit' to be the last command, while Zsh and
     Solaris `/bin/sh' consider that when the trap is run it is _still_
     in the `exit', hence it is the previous exit status that the trap

          $ cat trap.sh
          trap 'echo $?' 0
          (exit 42); exit 0
          $ zsh trap.sh
          $ bash trap.sh

     The portable solution is then simple: when you want to `exit 42',
     run `(exit 42); exit 42', the first `exit' being used to set the
     exit status to 42 for Zsh, and the second to trigger the trap and
     pass 42 as exit status for Bash.

     The shell in FreeBSD 4.0 has the following bug: `$?' is reset to 0
     by empty lines if the code is inside `trap'.

          $ trap 'false

          echo $?' 0
          $ exit

     Fortunately, this bug only affects `trap'.

     Don't worry: as far as we know `true' is portable.  Nevertheless,
     it's not always a builtin (e.g., Bash 1.x), and the portable shell
     community tends to prefer using `:'.  This has a funny side
     effect: when asked whether `false' is more portable than `true'
     Alexandre Oliva answered:

          In a sense, yes, because if it doesn't exist, the shell will
          produce an exit status of failure, which is correct for
          `false', but not for `true'.

     In some nonconforming shells (e.g., Bash 2.05a), `unset FOO' fails
     when `FOO' is not set.  Also, Bash 2.01 mishandles `unset MAIL' in
     some cases and dumps core.

     A few ancient shells lack `unset' entirely.  Nevertheless, because
     it is extremely useful to disable embarrassing variables such as
     `PS1', you can test for its existence and use it _provided_ you
     give a neutralizing value when `unset' is not supported:

          # "|| exit" suppresses any "Segmentation fault" message.
          if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then
          $unset PS1 || PS1='$ '

     Note: Special Shell Variables, for some neutralizing values.
     Also, see Note: Limitations of Builtins, documentation of
     `export', for the case of environment variables.

automatically generated by info2www