The PASV, RETR, REST, and PORT verbs

The PASV verb

A PASV
request
asks the server to accept a data connection
on a new TCP port selected by the server.
PASV
parameters
are prohibited.

The server normally
accepts
PASV with code 227.
Its response is a single line
showing the IP address of the server
and the TCP port number where the server is accepting connections.

Normally the client will connect to this TCP port,
from the same IP address that the client is using for the FTP connection,
and then send a RETR request.
However, the client may send some other requests first, such as REST.
The server must continue to read and respond to requests
while it accepts connections.
Most operating systems handle this automatically.

If the client sends another PASV request,
the server normally accepts the new request with a new TCP port.
It stops listening for connections on the old port,
and drops any connections already made.

RFC 959 failed to specify details of the response format.
I recommend that servers use the format

227 =h1,h2,h3,h4,p1,p2

where
the server's IP address is h1.h2.h3.h4
and the TCP port number is p1*256+p2.
The extra character before h1 is essential;
otherwise old versions of Netscape
will lose the first digit of h1.

Many servers put different strings before h1
and after p2.
I recommend that clients use the following strategy to parse the response line:
look for the first digit after the initial space;
look for the fourth comma after that digit;
read two (possibly negative) integers, separated by a comma;
the TCP port number is p1*256+p2,
where p1 is the first integer modulo 256
and p2 is the second integer modulo 256.

All servers that support file transfers are required to support PASV.
Many clients rely on PASV,
and will give up on a file transfer if PASV is rejected.

The RETR verb

A RETR request
asks the server to send the contents of a file
over the data connection already established by the client.
The RETR parameter is an
encoded pathname
of the file.
The file is either a
binary file
or a
text file,
depending on the most recent
TYPE request.

Normally the server responds with a
mark using code 150.
It then stops accepting new connections,
attempts to send the contents of the file over the data connection,
and closes the data connection.
Finally it

accepts the RETR request with code 226
if the entire file was successfully written to the server's TCP buffers;

rejects the RETR request with code 425
if no TCP connection was established;

rejects the RETR request with code 426
if the TCP connection was established but then broken by the client
or by network failure; or

rejects the RETR request with code 451 or 551
if the server had trouble reading the file from disk.

The server is obliged to close the data connection in each of these cases.
The client is not expected to look for a response from the server
until the client sees that the data connection is closed.

The server may reject the RETR request
without first responding with a mark.
In this case the server does not touch the data connection.
RFC 959 allows code 550 for file-does-not-exist, permission-denied, etc.,
and code 450 for out-of-memory, disk-failure, etc.

The client needs to be prepared for many different ways that RETR can fail:

After sending the RETR request,
the client reads one response from the server.
If this response is anything but a mark
(for example, the server can't find the file,
or doesn't have permission to open it,
or is temporarily out of memory,
or the server crashes and the connection is closed without a response,
or the server is overloaded
and the client times out before receiving a response),
the client stops and reports temporary failure.

After sending the RETR request and reading a mark,
the client reads data from the data connection
until FIN (end of file),
or a TCP error (for example, TCP reset or network unreachable),
or timeout.
The client saves the data,
remembers whether there was a FIN,
and closes the data connection.

After sending the RETR request, reading a mark,
reading data from the data connection, and closing the data connection,
the client reads another response from the server.
If this response is acceptance
and the data connection ended with FIN,
the client knows that it has received the entire file from the server.
Otherwise the client reports temporary failure;
the file has (probably) been truncated.

Note that code 226 from the server
does not guarantee that the client has received, or will receive,
the entire file.
The client must check for TCP errors on the data connection.

Some clients do not close the data connection
until they receive the 226 response from the server.
This behavior is permitted by RFC 959.
(The intent, now obsolete,
was for clients to retrieve multiple files through one data connection,
with a self-delimiting encoding of each file.
The server could use 226 to say that it was closing the connection,
or 250 to say that it wasn't.
The most obvious client implementation
wouldn't close the connection until it received 226.)
However, I recommend that clients close the data connection
immediately after seeing the end of data.
One server, wu-ftpd 2.6.0,
waits until the client closes the connection
before it sends its 226 response;
this screws up file transfers
to clients that do not close the data connection immediately.
This also wastes a round-trip time for other clients.
(As of 1999,
various versions of wu-ftpd run about half of the Internet's FTP servers.
Many servers made an emergency switch to version 2.6.0 in October 1999
when major security holes were discovered in previous versions.)

The REST verb

The server keeps track of a start position
for the client.
The start position is a nonnegative integer.
At the beginning of the FTP connection,
the start position is 0.

The start position changes the meaning of RETR for binary files as follows:
if the start position is n,
the server will omit the first n bytes
of the data that it sends through the data connection.
(RFC 959 specified a completely different use of the start position;
that use is obsolete.)

Some servers also allow start positions for text files,
with the same semantics:
omit the first n bytes
of data transferred through the data connection.
However, clients cannot rely on this.
Most existing servers skip a different number of bytes for text files.

A REST request sets the start position.
REST has a parameter giving a number as ASCII digits.
If the server accepts the REST request (required code 350),
it has set the start position to that number.
If the server rejects the REST request,
it has left the start position alone.

The server will set the start position to 0 after a successful RETR,
but might not set the start position to 0 after an unsuccessful RETR,
so the client must be careful to send a new REST request before the next RETR.
The server might set the start position to 0
after responding to any request other than REST,
so the client must send REST immediately before RETR.

Servers are not required to implement REST.
However, many clients can take advantage of REST
to save time if a previous transfer was interrupted in the middle.
Clients beware:
the file may have changed since the time of the previous transfer.

The PORT verb

A PORT request
asks the server to use a different mechanism of creating a data connection:
the server makes a TCP connection to the client.

The PORT request has a parameter in the form

h1,h2,h3,h4,p1,p2

meaning that the client is listening for connections
on TCP port
p1*256+p2
at IP address
h1.h2.h3.h4.
(The RFC 959 formal syntax does not allow any of these numbers to be 0.
The formal syntax is wrong.)

The server normally accepts PORT with code 200.
If the server was listening for a connection,
it stops, and drops any connections already made.

The server does not connect to the client's port immediately.
After the client sends RETR
and after the server sends its initial mark,
the server attempts to connect.
It rejects the RETR request with code 425
if the connection attempt fails;
otherwise it proceeds normally.

In theory, the client can send RETR without a preceding PORT or PASV.
The server is then supposed to connect to port 20 at the client's IP address.
In practice, however, servers refuse to do this.

For
security reasons,
clients should never use PORT.
However,
some clients still rely on PORT,
and will give up on a file transfer if PORT is rejected.
My current recommendation is that servers continue to support PORT.

FTP over IPv6

RFC 959 assumes that FTP connections take place over IPv4;
this assumption is built into the syntax of PORT and PASV.
One easy way to make FTP work over IPv6 is
for IPv6 FTP servers to respond to PASV using the format

227 =127,555,555,555,p1,p2

IPv6 FTP clients
can ignore the first four numbers,
extract the port number p1*256+p2,
and make an IPv6 data connection to the server on that port.
(There is no reason for IPv6 servers to support PORT.)

The reason to use the format shown above
is to make FTP work through IPv4-to-IPv6 network address translation.
IPv4 FTP clients that ignore the first four numbers
will successfully make a data connection to the server.
IPv4 FTP clients that foolishly attempt to connect to 127.555.555.555
will not end up bothering anyone.

There are several other proposed solutions
to the problem of making FTP work over IPv6:

Use a new SPSV command,
similar to PASV but with an easy-to-parse response format
that simply shows the port number in decimal.

Use a complicated new EPRT command
that was designed to support proxy FTP over multiple protocols.