#! - Some details about the shebang mechanism on various Unix flavours

Here you'll find:

• More reading about the history.
• Some specialities and selected details.
• A table with detailed test results on numerous systems.

More reading:

• The Unix FAQ, paragraph "3.16) Why do some scripts start with #! ... ?" (reproduced here for your convenience).
  With some emphasis about the history concerning shells.

• An old mail from Dennis Ritchie about the "new feature". Found in 4.0BSD /usr/src/sys/newsys/sys1.c

  So this mechanism was invented between version7 and version8. It was also available in 4BSD (probably not activated in the Makefile until 4.3), but not yet in 3BSD.

  (Pointed out by Gunnar Ritter in <3B5B0BA4.XY112IX2@bigfoot.de> in de.comp.os.unix.shell.)

Some specialities:

• env(1) is often used with the shebang mechanism to start an interpreter, which then only needs to be somewhere in your PATH, e.g. "#!/usr/bin/env perl".

  However, env(1) is not /usr/bin/env but /bin/env on some Linux distributions. So the env-mechanism is increasing convenience, but not really improving portability.

• There is a rumor, that a very few and very special, earlier Unix versions (particularly 4.2BSD derivatives) require you to separate the "#!" from the following path with a blank. You may also read that, allegedly, such a kernel would parse "#! /" as a 32-bit (long) magic.

  But it turns out that it is impossible to find a Unix which actually requires this.

  4.2BSD in fact doesn't, although the GNU autoconf tutorial wrongly claims this (5.5: Portable Shell Programming).

  But instead, see the 4.2BSD source: sys/kern_exec.c, (mainly l.123 ff.). It (certainly) accepts, but doesn't require a blank.

  (All this pointed out by Gunnar Ritter in <3B5B0BA4.XY112IX2@bigfoot.de> in de.comp.os.unix.shell.)

• About the the maximum length of a shebang line and some more details:
  • Originally it was 16 bytes.

  • 32 bytes on SunOS 4
    This is "sizeof(struct a.out)". The reason is a union, which contains both struct a.out/exec and a string with the same length (to access the shebang line). (For probably very similar code see also e.g. 4.3BSD sys/kern_exec.c)

  • On some systems, the line doesn't get truncated if it is too long, but the kernel refuses to execute the file and returns an ENOEXEC. Usually you might not even get an according exit status in the command line. So this is a rather silent failure.

  • About the 32 bytes on other flavours below: apparently the limit is coded and not the union mentioned above.

  • 127 bytes on Linux
    See also do_load_script() in linux/fs/binfmt_script.c.

  • On other flavours, the maximum length varies between _POSIX_PATH_MAX (255) and PATH_MAX, f.i. 1024 (see <limits.h>). I haven't tried all, as it's really not important. About 80 characters really are sufficient in general.

• If you want to be as portable as SUSv2/POSIX.2 allows, then you consequently should not use the shebang mechanism:

      Shell Introduction
      [...]
      If the first line of a file of shell commands starts with the
      characters #!, the results are unspecified.
  
      The construct #! is reserved for implementations wishing to provide
      that extension. A portable application cannot use #! as the first
      line of a shell script; it might not be interpreted as a comment.
      [...]
  
      Command Search and Execution
      [...]
      This description requires that the shell can execute shell
      scripts directly, even if the underlying system does not support
      the common #! interpreter convention. That is, if file foo contains
      shell commands and is executable, the following will execute foo:
  
  	./foo 

  Well, speaking about "#!/bin/sh", this is a really rocksolid convention, if you accept anything from the Bourne Shell family and its descendants to be called.

Test results on various systems:

I was using the following as program "showargs":

	#include <stdio.h>
	int main(int argc, char** argv) {
	    int i;
	    for (i=0; i<argc; i++)
		fprintf(stdout, "argv[%d]: \"%s\"\n", i, argv[i]);
	    return(0);
	} 

and a one liner script "invoker.sh" to call it, similar to

	#!/tmp/showargs -1 -2 -3 

to produce the following results (tried them myself, but i'd like to add your results from yet different systems).

Typically, a result from the above would look like this:

	argv[0]: "/tmp/showargs"
	argv[1]: "-1 -2 -3"
	argv[2]: "./invoker.sh"

                  | maximum  |cut-off(c)|only the|each arg|invoker in|not full
                  | length of|error(err)|1st arg |in its  |argv[0]   |path in
                  | shebang- |or ENOEXEC|        |own     | (not the |argv[0]
                  | line:    |( )    [1]|        |argv[x] |  invoked)|
------------------+----------+----------+--------+--------+----------+-------
AIX 3.2.5         | > 80     | [2]      |        |        |          | [2]
AIX 4.3.2         | > 80     |          |        |        |          |  X
FreeBSD 4.3(alpha)|   64     |          |        | X      |          |
FreeBSD 4.5(alpha)| > 80     | err      |        | X      |          |
HP-UX A.08.07     |   32     |          |        |        | X        |  -
HP-UX B.09.03     |   32     |          |        |        | X        |  -
HP-UX B.10.10     | > 80     | [3]      |        |        |          | [3]
HP-UX B.10.20     | > 80     |          |        |        |          |
IRIX 5.3          | > 80     | err      |        |        |          |
IRIX 6.5          | > 80     | err      |        |        |          |
Linux 2.2.9 (x86) | > 80     |  c       |        |        |          |  X
MUNIX 3.1 (SVR3.x)|   32     |          |        |        | X        |
NetBSD 1.5(alpha) |   64     |          |        |        |          |
OpenBSD 2.9       |   64     |          |        |        |          |
OSF1 V4.0(alpha)  | > 80     |          |        |        |          |  X
OSF1 T5.1(alpha)  | > 80     |          |        |        |          |  X
Sinix 5.20        |   32     |          |        |        |          |
SunOS 4.1.4(sparc)|   32     |  c       |        |        |          |
SunOS 5.x(sparc)  | > 80     |          | X      |        |          |
Ultrix 4.5        |   80     |  ?       | X      |        |          |

Meaning of the columns:

• "maximum length of shebang-line": self explanating
• "cut-off(c), error(err) or ENOEXEC ( )": see the footnote [1] below.
• "only the 1st arg": argv[1] from above contains "-1" then
• "each arg in its own argv[x]": argv[1]:"-1", argv[2]:"-2", etc.
• "invoker in argv[0]": argv[0] doesn't contain "/tmp/showargs" but "./invoker.sh"
• "not full path in argv[0]": argv[0] contains the basename of the called program instead of its full path.

Footnotes in the table:

[1]	If the shebang line is too long, three things can happen: The line gets truncated Due to a returned E2BIG/ENAMETOOLONG, you get something like "Arg list too long"/"Arg list or environment too large" or "File name too long", respectively. You get ENOEXEC (which usually results in a rather silent failure in shells).
[2]	If the shebang line exceeds 255 characters, it gets cut down to exactly 80 characters--or the first argument, if this is shorter. If the shebang line was not too long, argv[0] is the basename of the invoked program, otherwise the absolute path.
[3]	If the shebang line exceeds 128 characters, it gets cut down to exactly 80 characters--or the first argument, if this is shorter. If the shebang line was not too long, argv[0] is the invoking, otherwise the invoked program.

And why shebang? In music, '#' means sharp. So just shorten it to sharp-bang. Or it might be derived from "shell bang". All this probably under the influence of the american slang idiom "the whole shebang" (everything, the works). See also the jargon dictionary for the latter explanation.