Cooper

11.1. Job Control Commands

Certain of the following job control commands take a
"job identifier" as an argument. See the table at end of the chapter.

jobs

Lists the jobs running in the background, giving the job number.
Not as useful as ps.

It is all too easy to confuse
jobs and
processes. Certain builtins, such as
kill, disown, and
wait accept either a job number or a
process number as an argument. The fg,
bg and jobs
commands accept only a job number.

bash$ sleep 100 &[1] 1384bash $ jobs[1]+ Running sleep 100 &

"1" is the job number (jobs are
maintained by the current shell), and "1384"
is the process number (processes are maintained by
the system). To kill this job/process, either a kill
%1 or a kill 1384 works.

Thanks, S.C.

disown

Remove job(s) from the shell's table of active jobs.

fg, bg

The fg command switches a job
running in the background into the foreground. The
bg command restarts a suspended job, and
runs it in the background. If no job number is specified,
then the fg or bg
command acts upon the currently running job.

wait

Stop script execution until all jobs running in
background have terminated, or until the job number or
process ID specified as an option terminates. Returns the exit status of waited-for
command.

You may use the wait command
to prevent a script from exiting before a background
job finishes executing (this would create a dreaded
orphan process).

Example 11-24. Waiting for a process to finish before proceeding

#!/bin/bash
ROOT_UID=0 # Only users with $UID 0 have root privileges.
E_NOTROOT=65
E_NOPARAMS=66
if [ "$UID" -ne "$ROOT_UID" ]
then
echo "Must be root to run this script."
# "Run along kid, it's past your bedtime."
exit $E_NOTROOT
fi
if [ -z "$1" ]
then
echo "Usage: `basename $0` find-string"
exit $E_NOPARAMS
fi
echo "Updating 'locate' database..."
echo "This may take a while."
updatedb /usr & # Must be run as root.
wait
# Don't run the rest of the script until 'updatedb' finished.
# You want the the database updated before looking up the file name.
locate $1
# Without the 'wait' command, in the worse case scenario,
#+ the script would exit while 'updatedb' was still running,
#+ leaving it as an orphan process.
exit 0

Optionally, wait can take a job
identifier as an argument, for example,
wait%1 or wait
$PPID. See the job
id table.

Within a script, running a command in the background
with an ampersand (&) may cause the script
to hang until ENTER is hit. This
seems to occur with commands that write to
stdout. It can be a major annoyance.

kill -l lists all the
signals. A kill
-9 is a "sure kill", which will
usually terminate a process that stubbornly refuses to
die with a plain kill. Sometimes, a
kill -15 works. A "zombie
process," that is, a child process that has
terminated, but that the parent
process has not (yet) killed, cannot be killed by a
logged-on user -- you can't kill something that is already
dead -- but init will generally clean
it up sooner or later.

command

The command COMMAND directive
disables aliases and functions for the command
"COMMAND".

This is one of three shell directives that
effect script command processing. The others are
builtin and enable.

builtin

Invoking builtin
BUILTIN_COMMAND runs the command
"BUILTIN_COMMAND" as a shell builtin, temporarily disabling
both functions and external system commands with the
same name.

enable

This either enables or disables a shell
builtin command. As an example, enable -n
kill disables the shell builtin kill, so that when Bash
subsequently encounters kill, it invokes
/bin/kill.

The -a
option to enable lists all the
shell builtins, indicating whether or not they
are enabled. The -f filename
option lets enable load a builtin as a shared library
(DLL) module from a properly compiled object file.
[1].

autoload

This is a port to Bash of the
ksh autoloader. With
autoload in place, a function with
an "autoload" declaration will load from an
external file at its first invocation.
[2]
This saves system resources.

Note that autoload is not a part of the
core Bash installation. It needs to be loaded in with
enable -f (see above).

Table 11-1. Job identifiers

Notation

Meaning

%N

Job number [N]

%S

Invocation (command line) of job begins with string S

%?S

Invocation (command line) of job contains within it string S

%%

"current" job (last job stopped in
foreground or started in background)

%+

"current" job (last job stopped in
foreground or started in background)

Notes

The C source for a number of loadable builtins is
typically found in the /usr/share/doc/bash-?.??/functions
directory.

Note that the -f option to
enable is not portable to all
systems.

Options

Options are settings that change shell and/or script
behavior.

The set command
enables options within a script. At the point in the script
where you want the options to take effect, use set
-o option-name or, in short form, set
-option-abbrev. These two forms are equivalent.

#!/bin/bash
set -o verbose
# Echoes all commands before executing.

#!/bin/bash
set -v
# Exact same effect as above.

To disable an option within a script,
use set +o option-name or set
+option-abbrev.

Converting DOS Batch Files to Shell Scripts

Quite a number of programmers learned scripting on a PC running
DOS. Even the crippled DOS batch file language allowed writing some
fairly powerful scripts and applications, though they often required
extensive kludges and workarounds. Occasionally, the need still
arises to convert an old DOS batch file to a UNIX shell script. This
is generally not difficult, as DOS batch file operators are only a
limited subset of the equivalent shell scripting ones.

Batch files usually contain DOS commands. These must be
translated into their UNIX equivalents in order to convert a
batch file into a shell script.

Table L-2. DOS commands and their UNIX equivalents

DOS Command

UNIX Equivalent

Effect

ASSIGN

ln

link file or directory

ATTRIB

chmod

change file permissions

CD

cd

change directory

CHDIR

cd

change directory

CLS

clear

clear screen

COMP

diff, comm, cmp

file compare

COPY

cp

file copy

Ctl-C

Ctl-C

break (signal)

Ctl-Z

Ctl-D

EOF (end-of-file)

DEL

rm

delete file(s)

DELTREE

rm -rf

delete directory recursively

DIR

ls -l

directory listing

ERASE

rm

delete file(s)

EXIT

exit

exit current process

FC

comm, cmp

file compare

FIND

grep

find strings in files

MD

mkdir

make directory

MKDIR

mkdir

make directory

MORE

more

text file paging filter

MOVE

mv

move

PATH

$PATH

path to executables

REN

mv

rename (move)

RENAME

mv

rename (move)

RD

rmdir

remove directory

RMDIR

rmdir

remove directory

SORT

sort

sort file

TIME

date

display system time

TYPE

cat

output file to stdout

XCOPY

cp

(extended) file copy

Virtually all UNIX and shell operators and commands have
many more options and enhancements than their DOS and batch file
equivalents. Many DOS batch files rely on auxiliary utilities,
such as ask.com, a crippled counterpart to
read.

DOS supports a very limited and incompatible subset of
filename wildcard expansion,
recognizing only the * and ?
characters.

Converting a DOS batch file into a shell script is generally
straightforward, and the result ofttimes reads better than the
original.

Ted Davis' Shell
Scripts on the PC site has a set of comprehensive
tutorials on the old-fashioned art of batch file
programming. Certain of his ingenious techniques could conceivably
have relevance for shell scripts.

Starting Off With a Sha-Bang

In the simplest case, a script is nothing more than a list of system
commands stored in a file. At the very least, this saves the
effort of retyping that particular sequence of commands each time
it is invoked.

There is nothing unusual here, only a set of commands that
could just as easily be invoked one by one from the command line on
the console or in an xterm. The advantages of
placing the commands in a script go beyond not having to retype them
time and again. The script becomes a tool,
and can easily be modified or customized for a particular
application.

Since you may not wish to wipe out the entire system log,
this version of the script keeps the last section of the message
log intact. You will constantly discover ways of refining previously
written scripts for increased effectiveness.

The
sha-bang
( #!) at the head of a script
tells your system that this file is a set of commands to be fed
to the command interpreter indicated. The
#! is actually a two-byte
[1]magic number, a special marker that
designates a file type, or in this case an executable shell
script (type man magic for more
details on this fascinating topic). Immediately following
the sha-bang is a path
name. This is the path to the program that interprets
the commands in the script, whether it be a shell, a programming
language, or a utility. This command interpreter then executes
the commands in the script, starting at the top (line following
the sha-bang line), ignoring comments.
[2]

Each of the above script header lines calls a different command
interpreter, be it /bin/sh, the default shell
(bash in a Linux system) or otherwise.
[3]
Using #!/bin/sh, the default Bourne shell
in most commercial variants of UNIX, makes the script portable to non-Linux machines,
though you sacrifice Bash-specific features.
The script will, however, conform to the
POSIX[4]sh standard.

Note that the path given at the "sha-bang" must
be correct, otherwise an error message -- usually "Command
not found" -- will be the only result of running the
script.

#! can be omitted if the script consists only
of a set of generic system commands, using no internal
shell directives. The second example, above, requires the
initial #!, since the variable assignment line,
lines=50, uses a shell-specific construct.
Note again that #!/bin/sh invokes the default
shell interpreter, which defaults to /bin/bash
on a Linux machine.

This tutorial encourages a modular approach
to constructing a script. Make note of and collect
"boilerplate" code snippets that might be useful
in future scripts. Eventually you can build quite an extensive
library of nifty routines. As an example, the following script
prolog tests whether the script has been invoked with the correct
number of parameters.

Many times, you will write a script that carries out one
particular task. The first script in this chapter is an
example of this. Later, it might occur to you to generalize
the script to do other, similar tasks. Replacing the literal
("hard-wired") constants by variables is a step in
that direction, as is replacing repetitive code blocks by functions.

Notes

The #! line in a shell script
will be the first thing the command interpreter
(sh or bash)
sees. Since this line begins with a #,
it will be correctly interpreted as a comment when the
command interpreter finally executes the script. The
line has already served its purpose - calling the command
interpreter.

If, in fact, the script includes an
extra#! line, then
bash will interpret it as a comment.

#!/bin/rm
# Self-deleting script.
# Nothing much seems to happen when you run this... except that the file disappears.
WHATEVER=65
echo "This line will never print (betcha!)."
exit $WHATEVER # Doesn't matter. The script will not exit here.

Also, try starting a README file with a
#!/bin/more, and making it executable.
The result is a self-listing documentation file. (A here document using
cat is possibly a better alternative
-- see Example 17-3).

command substitution. The `command` construct makes
available the output of command
for assignment to a variable. This is also known as
backquotes or
backticks.

:

null command [colon]. This is the shell equivalent of a
"NOP" (no op, a
do-nothing operation). It may be considered a synonym for
the shell builtin true. The
":" command is a itself a
Bash builtin, and its
exit status is
"true" (0).

In combination with the >>
redirection operator, has no effect on a pre-existing
target file (: >> target_file).
If the file did not previously exist, creates it.

This applies to regular files, not pipes,
symlinks, and certain special files.

May be used to begin a comment line, although this is not
recommended. Using # for a comment turns
off error checking for the remainder of that line, so
almost anything may be appear in a comment. However,
this is not the case with
:.

: This is a comment that generates an error, ( if [ $x -eq 3] ).

The ":" also serves as a field
separator, in /etc/passwd, and in the $PATH variable.

reverse (or negate) the sense of
a test or exit status [bang]. The ! operator inverts the exit status
of the command to which it is applied (see
Example 6-2). It also inverts
the meaning of a test operator. This can, for
example, change the sense of "equal"
( =
) to "not-equal" ( != ). The
! operator is a Bash keyword.

A command may act upon a comma-separated list of file specs within
braces.
[1]
Filename expansion (globbing)
applies to the file specs between the braces.

No spaces allowed within the braces
unless the spaces are quoted or escaped.

echo {file1,file2}\ :{\ A," B",' C'}

file1 : A file1 : B file1 : C file2 : A file2 : B file2 : C

{}

Block of code [curly brackets]. Also referred to as an "inline group",
this construct, in effect, creates an anonymous
function. However, unlike a function, the variables
in a code block remain visible to the remainder of the
script.

pipe. Passes the output of previous command to the input
of the next one, or to the shell. This is a method of
chaining commands together.

echo ls -l | sh
# Passes the output of "echo ls -l" to the shell,
#+ with the same result as a simple "ls -l".
cat *.lst | sort | uniq
# Merges and sorts all ".lst" files, then deletes duplicate lines.

A pipe, as a classic method of interprocess
communication, sends the stdout
of one process to the stdin
of another. In a typical case, a command,
such as cat or echo, pipes a stream of data to a
"filter" (a command that transforms its input)
for processing.

bunzip2 linux-2.4.3.tar.bz2 | tar xvf -
# --uncompress tar file-- | --then pass it to "tar"--
# If "tar" has not been patched to handle "bunzip2",
# this needs to be done in two discrete steps, using a pipe.
# The purpose of the exercise is to unarchive "bzipped" kernel source.

Note that in this context the "-" is not
itself a Bash operator, but rather an option recognized by
certain UNIX utilities that write to
stdout, such as tar,
cat, etc.

bash$ echo "whatever" | cat -whatever

Where a filename is expected,
- redirects output to
stdout (sometimes seen with
tar cf), or accepts input from
stdin, rather than from a file. This
is a method of using a file-oriented utility as a filter
in a pipe.

Filenames beginning with
"-" may cause problems when coupled with the
"-" redirection operator. A script should
check for this and add an appropriate prefix to such
filenames, for example ./-FILENAME,
$PWD/-FILENAME, or
$PATHNAME/-FILENAME.

If the value of a variable begins with a
-, this may likewise create
problems.

var="-n"
echo $var
# Has the effect of "echo -n", and outputs nothing.

home directory [tilde]. This corresponds to the $HOME internal variable.
~bozo is bozo's home directory,
and ls ~bozo lists the contents of it.
~/ is the current user's home directory,
and ls ~/ lists the contents of it.

When typing text on the console or in an
xterm window,
Ctl-D erases the character under the
cursor. When there are no characters present,
Ctl-D logs out of the session, as
expected. In an xterm window, this has the effect of closing
the window.

Ctl-G

"BEL" (beep). On some old-time teletype
terminals, this would actually ring a bell.

When typing text on the console or in an
xterm window,
Ctl-K erases from the character
under the cursor to end of line.

Ctl-L

Formfeed (clear the terminal screen). This has
the same effect as the clear command.

Ctl-M

Carriage return.

#!/bin/bash
# Thank you, Lee Maschmeyer, for this example.
read -n 1 -s -p $'Control-M leaves cursor at beginning of this line. Press Enter. \x0d'
# Of course, '0d' is the hex equivalent of Control-M.
echo >&2 # The '-s' makes anything typed silent,
#+ so it is necessary to go to new line explicitly.
read -n 1 -s -p $'Control-J leaves cursor on next line. \x0a'
echo >&2 # Control-J is linefeed.
###
read -n 1 -s -p $'And Control-K\x0bgoes straight down.'
echo >&2 # Control-K is vertical tab.
# A better example of the effect of a vertical tab is:
var=$'\x0aThis is the bottom line\x0bThis is the top line\x0a'
echo "$var"
# This works the same way as the above example. However:
echo "$var" | col
# This causes the right end of the line to be higher than the left end.
# It also explains why we started and ended with a line feed --
#+ to avoid a garbled screen.
# As Lee Maschmeyer explains:
# --------------------------
# In the [first vertical tab example] . . . the vertical tab
#+ makes the printing go straight down without a carriage return.
# This is true only on devices, such as the Linux console,
#+ that can't go "backward."
# The real purpose of VT is to go straight UP, not down.
# It can be used to print superscripts on a printer.
# The col utility can be used to emulate the proper behavior of VT.
exit 0

Ctl-Q

Resume (XON).

This resumes stdin in a terminal.

Ctl-S

Suspend (XOFF).

This freezes stdin in a terminal.
(Use Ctl-Q to restore input.)

Ctl-U

Erase a line of input, from the cursor backward to
beginning of line. In some settings,
Ctl-U erases the entire
line of input, regardless of cursor
position.

Ctl-V

When inputting text, Ctl-V
permits inserting control characters. For example, the
following two are equivalent:

echo -e '\x0a'
echo <Ctl-V><Ctl-J>

Ctl-V is primarily useful from
within a text editor.

Ctl-W

When typing text on the console or in an xterm window,
Ctl-W erases from the character
under the cursor backwards to the first instance of
whitespace. In some settings, Ctl-W
erases backwards to first non-alphanumeric character.

Ctl-Z

Pause a foreground job.

Whitespace

functions as a separator, separating commands or variables. Whitespace consists of either spaces, tabs, blank
lines, or any combination thereof. In some contexts,
such as variable assignment,
whitespace is not permitted, and results in a syntax
error.

Blank lines have no effect on the action of a script,
and are therefore useful for visually separating functional
sections.

$IFS, the special variable
separating fields of input to certain commands, defaults
to whitespace.

Notes

Exception: a code block in braces as
part of a pipe may be run as a
subshell.

ls | { read firstline; read secondline; }
# Error. The code block in braces runs as a subshell,
# so the output of "ls" cannot be passed to variables within the block.
echo "First line is $firstline; second line is $secondline" # Will not work.
# Thanks, S.C.

Variable Substitution

The name of a variable is a placeholder for
its value, the data it holds. Referencing its
value is called variable substitution.

$

Let us carefully distinguish between the
name of a variable
and its value. If
variable1 is the name of a
variable, then $variable1
is a reference to its value,
the data item it contains. The only time a
variable appears "naked" -- without
the $ prefix -- is when declared
or assigned, when unset,
when exported,
or in the special case of a variable representing
a signal (see
Example 29-5). Assignment may be with an
= (as in var1=27),
in a read statement,
and at the head of a loop (for var2 in 1 2
3).

Enclosing a referenced value in
double quotes (" ")
does not interfere with variable substitution. This is
called partial quoting, sometimes
referred to as "weak quoting."Using single quotes (' ')
causes the variable name to be used literally, and no
substitution will take place. This is full
quoting, sometimes referred to as "strong
quoting." See Chapter 5 for a
detailed discussion.

Note that $variable is actually a
simplified alternate form of
${variable}. In contexts
where the $variable syntax
causes an error, the longer form may work (see Section 9.3, below).

Example 4-1. Variable assignment and substitution

#!/bin/bash
# Variables: assignment and substitution
a=375
hello=$a
#-------------------------------------------------------------------------
# No space permitted on either side of = sign when initializing variables.
# What happens if there is a space?
# If "VARIABLE =value",
# ^
#+ script tries to run "VARIABLE" command with one argument, "=value".
# If "VARIABLE= value",
# ^
#+ script tries to run "value" command with
#+ the environmental variable "VARIABLE" set to "".
#-------------------------------------------------------------------------
echo hello # Not a variable reference, just the string "hello".
echo $hello
echo ${hello} # Identical to above.
echo "$hello"
echo "${hello}"
echo
hello="A B C D"
echo $hello # A B C D
echo "$hello" # A B C D
# As you see, echo $hello and echo "$hello" give different results.
# ^ ^
# Quoting a variable preserves whitespace.
echo
echo '$hello' # $hello
# ^ ^
# Variable referencing disabled by single quotes,
#+ which causes the "$" to be interpreted literally.
# Notice the effect of different types of quoting.
hello= # Setting it to a null value.
echo "\$hello (null value) = $hello"
# Note that setting a variable to a null value is not the same as
#+ unsetting it, although the end result is the same (see below).
# --------------------------------------------------------------
# It is permissible to set multiple variables on the same line,
#+ if separated by white space.
# Caution, this may reduce legibility, and may not be portable.
var1=21 var2=22 var3=$V3
echo
echo "var1=$var1 var2=$var2 var3=$var3"
# May cause problems with older versions of "sh".
# --------------------------------------------------------------
echo; echo
numbers="one two three"
# ^ ^
other_numbers="1 2 3"
# ^ ^
# If there is whitespace embedded within a variable,
#+ then quotes are necessary.
echo "numbers = $numbers"
echo "other_numbers = $other_numbers" # other_numbers = 1 2 3
echo
echo "uninitialized_variable = $uninitialized_variable"
# Uninitialized variable has null value (no value at all).
uninitialized_variable= # Declaring, but not initializing it --
#+ same as setting it to a null value, as above.
echo "uninitialized_variable = $uninitialized_variable"
# It still has a null value.
uninitialized_variable=23 # Set it.
unset uninitialized_variable # Unset it.
echo "uninitialized_variable = $uninitialized_variable"
# It still has a null value.
echo
exit 0

An uninitialized variable has a
"null" value - no assigned value at all
(not zero!). Using a variable before assigning a value
to it will usually cause problems.

It is nevertheless possible to perform arithmetic operations
on an uninitialized variable.