How to use ftp in a shell script

Sometimes I want to FTP a file from one machine to another.
Usually, I can do the transfer interactively, but every so often, I
would like to have a shell script do the file transfer. This task
has eluded me in the past, but I finally figured it out. I've not
seen this particular trick documented in the past, so I submit it
for your approval.

The Problem

The problem I always encountered in scripting ftp transfers
involved getting a password to the ftp server. Typical ftp client
programs under Unix, Linux, Solaris and NetBSD all read the ftp
password from /dev/tty.

Example (non-working) script

The above script will just hang if run in the foreground (in an
xterm), or if run in the background (from a cron job), it will fail
to perform the work of transferring file.txt.

/dev/tty names a strange, magic device. Each process
(more strictly each process group) has a different
/dev/tty, and you can not naively make ftp clients read
the password from some non-magic, yet convenient source, like a
"here document". When run in an xterm, the script above appears to
hang because it reads the password from /dev/tty. The
xterm constitutes the script's /dev/tty, so the script
waits for keyboard input.

Example Working Script

The Tricks

Getting the password to the ftp server without having the ftp
client program read the password from /dev/tty requires
two tricks:

Using the -n option on the ftp client program to
prevent the ftp client from trying to log in immediately. That way,
the ftp client does not ask for a user ID and password. No use of
/dev/tty.

Use the ftp client program command quote to send
user ID and password to the ftp server.

You must the token that ends the "here document" (END_SCRIPT in
the example above) at the beginning of a line. Even if the ftp
command line and the login and transfer script are indented, END_SCRIPT
should appear with the 'E' as the first character of the line.

Further Refinements

The above sh script will spew lots of ftp client output
to standard output. Even if everything works perfectly, the user
running the above script will see lots of incomprehensible text
scrolling by quite rapidly. One refinement would send output to
different places:

ftp -n $HOST > /tmp/ftp.worked 2> /tmp/ftp.failed <<END_SCRIPT

One could further refine error handling by acting on the ftp client
program's exit status:

Control of ftp by a shell script

One obvious improvement would have the ftp client program
controlled by the shell script. I don't think that would comprise
an impossible task, but I also don't think that it would have much
value. Scripting ftp transfer using expect might cause you less pain.

Alternative #1

It still uses the "-n" trick, but it sends user ID and password
in the same "user" command.

Alternative #2

Use a .netrc file

Linux, Unix and BSD users have the alternative of using a .netrc
file. The ftp man page documents the format of .netrc. To accomplish
the task of using ftp in a shell script you would have to fill out a
.netrc file something like this:

machine something.else.com
login myid
password mypassword

ftp demands that .netrc not have group or world read or
write permissions:

$ ls -l .netrc
-rw------- 1 bediger users 51 Dec 16 13:30 .netrc

Using a .netrc file has a few problems that may or may
not prevent you from using it.

A shell scripkt that does FTP using .netrc is no longer self-contained.
You have to keep track of two files, which means that bugs can be less
than obvious.

ftp reads it's user ID's .netrc. If you develop your script
under a given user ID, then put it in production under a second user ID,
you have to coordinate .netrc file contents between those two user IDs.

Alternative #3

Apparently, the Ckermit program from Columbia University understands
FTP. You could use Ckermit to script FTP transfers.
This looks to have advantages and disadvantages. On the "pro" side,
it appears that Ckermit can exit on various errors, like unknown user IDs,
or bad passwords. On the "con" side, you have to have Ckermit. I don't
recall that it had a too onerous install, but it doesn't come with many
Linux distros these days, and it probably doesn't come with any vendor
Unix.