Introduction to Shell Scripting - Part 2

Last month, we took a look at some basics of creating a shell script,
as well as a few of the underlying mechanisms that make it all work. This
time around, we'll see how loops and conditional execution let us direct
program flow in scripts, as well as looking at a few good shell-writing
practices.

Conventions

The only thing to note in this article are ellipses (...) - I use them
to indicate that the code shown is only a fragment, and not an entire script
all by itself. If it helps, think of each ellipse as one or more lines
of code that is not actually written out.

Loops and Conditional Execution

for; do; done

Often, scripts are written to automate some repetitive task; as a random
example, if you have to repeatedly edit a series of files in a specific
directory, you might have a script that looks like this:

#!/bin/bash
for n in ~/weekly/*.txt
do
ae $n
done
echo "Done."

or like this:

#!/bin/bash
for n in ~/weekly/*.txt; do ae $n; done; echo "Done."

The code in both does exactly the same thing - but the first version is
much more readable, especially if you're building large scripts with
several levels. As good general practice in writing code, you should indent
each level (the commands inside the loops); it makes troubleshooting and
following your code much easier.

The above control structure is called a 'for' loop - it looks for items
remaining in a list (e.g., 'are there any more files, beyond the ones we
have already read, that fit the "~/weekly/*.txt" template?'). If the test
returns true, it assigns the name of the current item in the list to the
loop variable ("n" in this case) and executes the loop body (the part
between "do" and "done"), then checks again. Whenever the list runs out,
'for' stops looping and passes control to the line following the 'done'
keyword - in our example, the "echo" statement.

A little trick I'd like to mention here. If you want to make the "for"
loop 'spin' a certain number of times, the shell syntax can be somewhat
tiresome:

What a pain! If you wanted it to iterate, say, 250 times, you'd have to
type all of that out! Fortunately, there's a 'shortcut' - the
'seq' command, which prints a sequence of numbers from 1 to
the given maximum, e.g.,

#!/bin/bash
for i in $(seq 15)
do
echo $i
done

This is functionally the same as the previous script. 'seq'
is part of the GNU "shellutils" package and is probably already installed
on your system. There's also the option of doing this sort of iteration by
using a "while" loop, but it's a bit more tricky.

while; do; done

Often, we need a control mechanism that acts based on a specified condition
rather than iterating through a list. The 'while' loop fills this requirement:

The general flow of this script is: we invoke 'pppd', the PPP
paenguin... I mean, daemon :), then keep looping until an actual
connection is established (if you want to use this script, replace
192.168.0.1 with your ISPs IP address). Here are the details:

1) The "ping -c 1 xxx.xxx.xxx.xxx" command sends a single
ping to the supplied IP address; note that it has to be an IP address and
not a URL - "ping" will fail immediately due to lack of DNS
otherwise. If there's no response within 10 seconds, it will print
something like

2) The only line we're interested in is the one that gives us the packet
loss percentage; with a single packet, it can only be 0% (i.e., a
successful ping) or 100%. By piping the output of "ping" through the "grep
100%" command, we narrow it down to that line, if the loss is indeed 100%;
a 0% loss will not produce any output. Note that the "100%" string isn't
anything special: we could have used "ret=-1", "unreachable", or anything
else that's unique to a failure response.

3) The square brackets that contain the statement are a synonym for the
'test' command, which returns '0' or '1' (true or false) based on the
evaluation of whatever's inside the brackets. The '-n' operator returns
'true' if the length of a given string is greater than 0. Since the string
is assumed to be contiguous (no spaces), and the line we're checking for is
not, we need to surround the output in double quotes - this is a technique
that you will use again and again in script writing. Do note that the
square brackets require spaces around them - i.e., [-n $STRING]
won't work; [ -n $STRING ] is correct. For more info on the
operators used with 'test', type "help test"; a number of very useful ones
are available.

4) As long as the above test returns "true" (i.e., as long as the "ping"
fails), the 'while' loop will continue to execute - by printing the
"Connecting..." string every ten seconds. As soon as a single ping is successful
(i.e., the test returns "false"), the 'while' loop will break and pass
control to the statement after "done".

"until; do; done"

The 'until' loop is the reverse of the 'while' - it continues to loop
as long as the test is false, and fails when it becomes true. I've rarely
had the occasion to use it; the 'while' loop and the flexibility of the
available tests usually suffice, and this construct is just "syntactic
sugar" for those who prefer to avoid logical inversions.

"if; then; [else]; fi"

There are many times when we need to check for the existence of a
condition and branch the execution based on the result. For those times, we
have the 'if' statement:

<grin> I guess it's not quite that easy... but the logic makes sense.
Anyway, if a variable called BOSS has been defined as "jerk" (C programmers
take note: '=' and '==' are equivalent in a test statement - no assignment
occurs), then the first 'echo' statement will be executed. In all other
cases, the second 'echo' statement will run (if $BOSS="idiot", you'll still
be working there. Sorry about that. :). Note that the 'else' statement is
optional, as in this script fragment:

This routine will obviously exit if the ERROR variable is anything other
than empty - but it will not affect the program flow otherwise.

"case; in; esac"

The remaining tool that we can use for conditional branching is basically
a multiple 'if' statement, based on the evaluation of a test. If, for example,
we know that the only possible outputs from an imaginary program called
'intel_cpu_test' are 4, 8, 16, 32, or 64, then we can write the following:

(Before all you folks flood me with mail about running Linux on an
8088... you can't run it on a calculator either. :)

Obviously, the "*" at the end is a catch-all: if someone at
the Intel Secret Lab runs this on their new CPU (code name "UltraSuperHyperWhizBang"),
we want the script to come back with a controlled response rather than
a failure. Note the double semicolons - they 'close' each of the "pattern/command"
sets and are (for some reason) a common error in "case/esac" constructs.
Pay extra attention to yours!

break and continue

These statements interrupt the program flow in specific ways. The
"break", once executed, immediately exits the enclosing loop; the
"continue" statement skips the current loop iteration. This is useful in a
number of situations, particularly in long loops where the existence of a
given condition makes all further tests unnecessary. Here's a long (but
hopefully understandable) pseudo-example:

A couple of key points: note that in checking the status of various
party supplies, you might be better off writing multiple "if" statements
- both potato chips and pretzels may run out at the same time (i.e.,
they are not mutually exclusive). The way it is now, the chips have
top priority; if two items do run out simultaneously, it will take two
loops to replace them.

We can keep checking the food status while trying to convince the cops
that we're actually holding a stamp-collectors' meeting (in fact, maintaining
the doughnut supply is a crucial factor at this point), but we'll skip
right past the liquor status - as it was, we got Joe down off the chandelier
just in time...

The "continue" statement skips the last part of the "while" loop as
long as the "police_on_scene" function returns 'true'; essentially, the
loop body is truncated at that point. Note that even though it is actually
inside the "if" construct, it affects the loop that surrounds it:
both "continue" and "break" apply only to loops, i.e., "for", "while",
and "until" constructs.

Interestingly enough, shortly after finishing last month's article, I was
cranking out a bit of C code on a machine that didn't have 'rcs' (the GNU
Revision Control System) installed - and this script came in very handy as
a 'micro-RCS'; I used it to take "snapshots" of the project status.
Simple, generalized scripts of this sort become very useful at odd times...

Error Checking

The above is a workable script - for you, or anyone who cares to read and
understand it. Let's face it, though: what we want from a program or a
script is to type the name and have it work, right? That, or tell us
exactly why it didn't work. In this case, though, what we get is a somewhat
cryptic message:

cp: missing destination file
Try `cp --help' for more information.

For everyone else, and for ourselves down the road when we forget exactly
how to use this tremendously complex script with innumerable options :), we
need to put in error checking - specifically, syntax/usage information.
Let's see how what we've just learned might apply:

The '-z' operator of 'test' returns '0' (true) for a zero-length string;
what we're testing for is 'bkup' being run without a filename. The very
beginning is, in my opinion, the best place to put help/usage information
in a script - if you forget what the options are, just run the script without
any, and you'll get an instant 'refresher course' in using it. You don't
even have to put in the original comments, now - note that we've basically
incorporated our earlier comments into the usage info. It's still a good
idea to put in comments at any non-obvious or tricky places in the script
- that brilliant trick you've managed to pull off may cause you to cuss
and scratch your head next year if you don't.

Before we wrap up playing with this script, let's give it a few more
capabilities. What if you wanted to be able to send different types of
files into different directories? Let's give that a shot, using what we've
learned:

1) The comment section of the help now reads "...directory tree" rather
than just "directory", indicating the change we've made.

2) The "Usage:" line has been expanded to show the optional (as shown
by the square brackets) argument; we've also added an explanation
of how to use that argument, since it might not be obvious to someone else.

3) An added "if" construct that checks to see if $2 (a second argument
to 'bkup') exists; if so, it checks for a directory with the given name
under "~/Backup", and creates one if it does not exist (the "-d" tests
if the file exists and is a directory).

4) The 'cp' command now has a 'subdir' variable tucked in between "Backup/"
and "$1".

etc., and sort everything into whatever categories you like. Plus, the
old behavior of "bkup" is still available -

bkup file.xyz

will send a backup of "file.xyz" to the "~/Backup" directory itself;
useful for files that fall outside of your sorting criteria.

By the way: why are we appending a "/" to $2 in the "if" statement instead
of right in the "cp" line? Well, if $2 doesn't exist, then
we want 'bkup' to act as it did originally, i.e., send the file to the
"Backup" directory. If we write something like

cp -i $1 ~/Backup/$subdir/$1.$a

(note the extra "/" between $subdir and $1), and $2 isn't specified,
then $subdir becomes blank, and the line above becomes

cp -i $1 ~/Backup//$1.$a

- not that it hurts anything, but we want to stick with standard shell
syntactic practice wherever possible (since shell quirks, such as a double
'/' being ignored, are not guaranteed to stick around.)

In fact, it's a really good idea to consider all the possibilities whenever
you're building variables into a string; a classic mistake of that sort
can be seen in the following script -

DO NOT USE THIS SCRIPT!
#!/bin/bash
# Written by Larry, Moe, and Shemp - the Deleshun PoWeR TeaM!!!
# Checked by Curly: "Why, soitainly it woiks! Nyuk-nyuk-nyuk!"
# All you've gotta do is enter the name of this file followed by
# whatever you want to delete - directories, dot files, multiple
# files, anything is OK!
rm -rf $1*
DO NOT USE THIS SCRIPT!

Well, at least they commented it. :)

What happens if somebody does run "three_stooges",
and doesn't enter a parameter? The active line in the script becomes

rm -rf *

Assuming that you're Joe User in your home directory, the result is
pretty horrible - it'll wipe out all of your personal files. It becomes
a catastrophe if you're the root user in the root directory - the entire
system goes away!!

Viruses seem like such friendly, harmless things about now...

Be careful with your script writing. As you have just seen, you have
the power to destroy your entire system in a blink.

Unix was never designed to keep people from doing stupid things,
because that policy would also keep them from doing clever things.
-- Doug Gwyn
Unix gives you just enough rope to hang yourself - and then a
couple more feet, just to be sure.
-- Eric Allman

The philosophy makes sense: unlimited power in the tools, restriction
by permissions - but it imposes a responsibility: you must take appropriate
care. As a corollary, whenever you're logged in as root, do not run any
shell scripts that are not provably harmless (note the Very Large assumptions
hanging off that phrase - "provably harmless"...)

Wrapping it up

Loops and conditional execution are a very important part of most scripts.
As we analyze other shell scripts in future articles, you'll see some of
the myriad ways in which they can be used - a script of even average complexity
cannot exist without them.

Next month, we'll take a look at some tools that are commonly used in
shell scripts - tools that may be very familiar to you as command-line
utilities - and explore how they may be connected together to produce desired
results. We'll also dissect a couple of scripts - mine, if no one else
is brave enough to send in the results of their keyboard concoctions. (Be
Afraid. Be Very Afraid. :)

I welcome all comments and corrections in regard to this series of articles,
as well as any interesting scripts that you may send in. All flames will
be sent to /dev/null (oh no, it's full...)

Ben is the Editor-in-Chief for Linux Gazette and a member of The Answer Gang.

Ben was born in Moscow, Russia in 1962. He became interested in electricity
at the tender age of six, promptly demonstrated it by sticking a fork into
a socket and starting a fire, and has been falling down technological
mineshafts ever since. He has been working with computers since the Elder
Days, when they had to be built by soldering parts onto printed circuit
boards and programs had to fit into 4k of memory. He would gladly pay good
money to any psychologist who can cure him of the recurrent nightmares.

His subsequent experiences include creating software in nearly a dozen
languages, network and database maintenance during the approach of a
hurricane, and writing articles for publications ranging from sailing
magazines to technological journals. After a seven-year Atlantic/Caribbean
cruise under sail and passages up and down the East coast of the US, he is
currently anchored in St. Augustine, Florida. He works as a technical
instructor for Sun Microsystems and a private Open Source consultant/Web
developer. His current set of hobbies includes flying, yoga, martial arts,
motorcycles, writing, and Roman history; his Palm Pilot is crammed full of
alarms, many of which contain exclamation points.

He has been working with Linux since 1997, and credits it with his complete
loss of interest in waging nuclear warfare on parts of the Pacific Northwest.