How the FTP protocol Challenges Firewall Security

1. Summary

The FTP (File Transfer Protocol) protocol is one of the most popular, but also the most misunderstood protocols. One of the questions most commonly asked when dealing with firewalls, routers and other Internet connectivity devices is why a particular FTP client or server configuration isn’t working. If network and firewall administrators understood how the FTP protocol works, they would be able to easily solve the FTP related problems they encounter.

In this article we’ll look at the following issues:

How the FTP protocol works

Challenges created by the FTP Protocol

Solving the FTP protocol problem

What about Secure FTP

It must be noted that in this article we will only look at the FTP protocol from the point of view of connectivity. In other words, we are only interested in how the FTP protocol makes connections and how those connections relate to each other. We won’t cover the specifics of FTP client or FTP server configuration.

2. How the FTP Protocol Works

2.1. Some History

The File Transfer Protocol (FTP) has been around since the 1970’s and was one of the first efforts to create a standard means of exchanging files over a TCP/IP network. FTP was designed with a great deal of flexibility so it could be used on non-TCP/IP networks and also have the capability to exchange files with a broad variety of machines.

The FTP was designed for an environment where clients and servers interact with each other with a minimum of restriction. Additionally, it was designed to operate over communication channels where packets travel directly to their destination. This is in contrast to today’s environment where there may be an intermediary (Firewall, NAT device, etc..) responsible for sending the packets to and from a host on a private network.

The base specification is RFC 959 and is dated October 1985. There are some additional RFCs relating to the FTP protocol. You can find a good overview of all technical documents relating to the FTP protocol at the RFC Sourcebook.

2.2. The FTP Model

FTP uses TCP as transport protocol to provide reliable end-to-end connections. Two connections are used: the first is the control connection and the second is the data connection that is managing the data transfer. The user who initiates the control connection assumes the client function, while the server function is provided by the remote host.

On both sides of the link the FTP application is built with a protocol interpreter (PI) and a data transfer process (DTP). On the client side of the link there exists also a user interface.

The user interface communicates with the protocol interpreter, which is in charge of the control connection. The protocol interpreter, besides its function of responding to the control protocol, has also to manage the data connection. During the file transfer, the data management is performed by the DTPs.

2.3. The FTP Protocol Modes

The FTP protocol is a complex protocol because it uses a control connection (the primary connection) and a data connection (the secondary connection). How the data connection is made depends on the FTP protocol mode. There are two FTP modes:

Normal or PORT or Active Mode

Passive or PASV Mode

Let’s look now in more detail how the different connections are made and how they are related to each other.

2.4. The Control connection

The control connection is the communication path between the USER-PI and SERVER-PI for the exchange of commands and replies. This connection follows the Telnet Protocol.

When an FTP client wants to exchange files with an FTP server, the FTP client must first set up the control connection. The client makes a TCP connection from a random unprivileged port N (N > 1023) to the FTP server’s well known command port 21 (the IANA assigned port number).

Schematically the information flow is as follows:

It is important to note that the protocol requirest the control connection to remain open while the data transfer is in progress. In other words, a data connection can not exist without an open control connection. On the other hand, the data connection doesn’t need to exist all of the time and there can be many data connections during the lifetime of a control connection. As a final note, it is the responsibility of the user to request the closing of the control connection when finished using the FTP service. However, it is the server who takes the action to close the control connection.

2.5. The Data connection

The data connection is the communication path between the USER-DTP and SERVER-DTP for the exchange of the real data, being directory lists and files. Depending on the choosen FTP mode, the data connection is initiated from the server (active mode) or the client (passive mode).

In active mode, the client sends a PORT command to the server. Basically this command tells the server to which host (IP address) and port number (unprivileged port > 1023) the server must connect back for the data connection. After accepting the Port command, the server will then establish the data connection from its local data port 20 (the IANA assigned port number) to the IP address and port number learned from the PORT command.

In passive mode, the client sends a PASV command to the server. Basically this command asks the server to “listen” on a data port (which is not its default data port 20) and to wait for a connection rather than to initiate one. If the server supports the passive mode, it will send a reply to this command including the host (IP address) and port number (unprivileged port > 1023) this server is listening on. The client will then establish the data connection from a local random unprivileged port (> 1023) to the IP address and port number learned from the PASV reply.

It is important to note that the data connection will only be established upon receipt of the reply to the Transfer Service commands such as LIST, RETR, STOR. So, the FTP mode must be selected by the client before sending the appropriate Transfer Service command. Also, those Transfer Service commands wil get more than one reply: first a ‘Positive Preliminary Reply’ indicating the data transfer may start and after ending the data transfer a ‘Positive Completion Reply’.

Schematically the information flow is as follows:

I have tried to make the above drawings as generic as possible. For our purpose, the most important information is who initiates the control and data connection and from/to which port numbers. This is clearly indicated in the above drawings. If you are interested in more indepth technical details, please consult the relevant RFC’s.

3. Challenges created by the FTP Protocol

In the following discussion I assume you have only basic packet filters at your disposal to protect you from intruders. This situation may seem to be outdated, but there are still some real situations where basic packet filtering is the only protection you have. Basic packet filters can only analyse each packet individually. No relation whatever with other packets or connection states can be taken into account.

Note: the informational document RFC1579 describes a suggested change to the behavior of the FTP client programs in order to become more Firewall-Friendly.

3.1. Active mode

The main problem with active mode FTP actually is on the client side. The FTP client doesn’t make the actual data connection to the server. It simply tells the server what port it is listening on and the server connects back to the specified port on the client. From the client side this appears to be an outside system initiating a connection to an internal client on any high-number port. Allowing new, unsolicited inbound connections to such a wide range of ports represents a definite security hazard.

One way of dealing with this problem is to allow inbound connections to the high-number ports from source port TCP port 20. In this way you limit access to what is assumed to be the FTP server data port. However, there are a number of tools available that allow you to set your source port manually. Therefore, you cannot be sure that incoming connections from TCP port 20 are actually sourcing from an FTP server.

You can improve on the situation somewhat by limiting inbound access to the high-number ports only from TCP port 20 and from a limited number of IP addresses of trusted FTP servers. The major drawback here is that you must be able to identify the trusted FTP server addresses in advance, and you still have to be concerned by someone spoofing a source port and IP address.

On the server side the situation is a bit less hazardous than the situation the client side has to handle. However, it’s still considered poor security practice to allow outbound access to such a wide array of ports just to support a single server application. Any client on the internal network may have access to network services on the Internet that use the high-number TCP ports for a primary connection.

One way to improve the situation is to allow outbound access to the high-number ports only when the source port is TCP port 20. In this way, you can assume that only the FTP servers are able to connect to these high-number ports on the Internet. You could strengthen this even more by limiting access from TCP port 20 to the high-number ports to a limited number of IP addresses on your internal network. However, you still have to deal with problems of spoofed IP addresses and manipulated port numbers.

3.2. Passive mode

At the client side the port requirements aren’t that different from those required by the PORT mode FTP client, with the exception that the PASV mode FTP client requires outbound access to all the high-number TCP ports. While this doesn’t seem like a big difference, it is in fact a tremendous difference from a security point of view. The reason why is that the direction of the connection is outbound instead of inbound and that is often considered less hazardous with a good basic packet filter.

Since you have no way to determine in advance what high-number port the FTP server will assign to the data channel, all the high-numbered ports must be opened. This might be fine if you had some way to assure that only the FTP clients would be accessing an FTP server on these ports. The problem is that you cannot easily control which applications can access which ports. And even if you did limit just FTP clients to these ports, you would be blocking other applications access to the high-number ports.

To add insult to injury, you must also allow inbound access to all high-number ports. The result is that you must allow inbound and outbound access to all high-number ports. You can improve that situation by limiting inbound access to all high-number ports to packets belonging to an already established connection (ACK-bit set in the TCP header). However, you still run into the spoofing problems such as the manipulation of the ACK-bit in the TCP header.

While passive mode FTP solves some of the problems from the client side, it opens up a whole range of problems on the server side. The biggest issue is the need to allow any remote connection to high-number ports on the server. If the FTP server allows the administrator to specify a range of ports which the FTP server will use, you can vastly improve the situation.

The second issue involves supporting and troubleshooting clients which do (or do not) support passive mode. As an example, some command line FTP utilities do not support passive mode, necessitating a third-party FTP client. Moreover, with the massive popularity of the World Wide Web, many people prefer to use their web browser as an FTP client. However, not all browsers or browser versions support both FTP modes when accessing ftp:// URLs and if they do, it is not always obvious to configure them correctly.

3.3. Network Address Translation

Due to the shortage of public routable IP addresses, it is common practice to place clients and even servers behind Network Address Translation (NAT) devices. There are two flavours of NAT devices: those who only translate the IP address portion (1:1 NAT) and those who translate the IP address and port number portion (N:1 NAT or PAT). It should be obvious that the use of PAT is probably the most implemented Network Address Translation technique because you can map a lot of unroutable private IP addresses to one public routable IP address.

Although Network Address Translation is an interesting technique to mitigate the shortage of public routable IP addresses, it creates also a whole new range of problems. The reason for that is that a lot of protocols are not designed with NAT/PAT in mind. Take as an example the FTP protocol. The IP-addresses and port numbers used for the Data connection are dynamically negotiated. Simple NAT/PAT devices who only look to the IP addresses and port numbers (layer 3 and 4 information), do not analyse the application layer where the actual negotiating process occures (PORT and PASV commands). So, without added special support, simple NAT/PAT devices will very likely break those complex protocols, including the FTP protocol.

4. Solving the FTP protocol problem

With the above knowledge, it should be relativily easy to understand how the ISA server handles the FTP protocol. Luckily for us, the ISA server has a bit more smarts than just basic packet filtering. Let us now investigate some very common scenarios.

4.1. Trihomed DMZ

ISA Server supports what is known as the “Trihomed DMZ” configuration (also called a perimeter network, demilitarized zone, or screened subnet). The Trihomed DMZ configuration is seen when the ISA Server computer has three NICs, where two of the NICs have public interfaces and one of the NICs has a private interface. This means that the perimeter network IP addresses are considered external and are never included in the LAT. If you have difficulty to fully understand what I’m talking about, you could read the following articles for clarification:

To implement access control for hosts on the perimeter network, the following basic rules apply:

Packets are routed between the public segments. No network address translation is performed.

Access to the perimeter network is controlled by static packet filters.

Inbound packet filters allow external traffic in and let the response back out.

Outbound filters are needed if traffic is originating on the perimeter network.

This means that the ISA Server acts as a packet filtering router. In other words, there is no application layer awareness. However, if you look carefully at the packet filter properties you will see that you need to specify the direction of the initial packet. So, you might say that the packet filters in ISA are more or less connection aware.

In the Packet Filter port specification you have the following choices:

local port: Any, Dynamic and Fixed

remote port: Any and Fixed

According to the documentation the choice Dynamic port means port 1025 – 5000. If this would be correct, you would not be able to use Dynamic port as a substitute for all high-number ports (> 1023). After some tests, I can tell you that the documentation is absolutely wrong. With the utility Netcat or WinsockTool you can easily find out yourself that Dynamic port means port 1025 – 65534 (> 1024 and < 65535). So, you can safely assume that the choice Dynamic port is valid as a substitute for all high-number ports.

Before you locate an FTP server on the perimeter network or allow an FTP client on the perimeter network outbound access using the trihomed configuration, carefully consider the implications as mentioned in the Challenges section of this article. Therefore it is not advised to put an FTP server or client on the DMZ segment in a trihomed ISA Server DMZ configuration, at least if you want optimal security.

However, if you must allow the FTP protocol from/to a perimeter host, use the following packet filters as starting point:

Note: which of the above Packet Filters you will actually implement, depends on which FTP mode (active or passive) you want to support.

4.2. SecureNAT client

The File Transfer Protocol filter that is provided with the ISA server forwards FTP requests from SecureNAT clients to the Firewall service. The filter dynamically opens secondary ports, which are required by the FTP protocol, and performs the necessary address translation for SecureNAT clients. Here’s how it works:

The client opens a primary connection (control connection) to the FTP server.

The ISA Server computer notifies the filter about the connection.

The filter examines the data that is flowing through the primary connection and determines which secondary connection (data connection) the client is going to use.

The filter informs the ISA Server computer to allow that particular secondary connection.

The ISA Server computer opens the specific port, as indicated by the application filter.

If you look at the predefined protocol definitions, you will find an FTP protocol definition for an FTP client and server. Both protocol definitions are of the type Application Filter. This means that ISA supports fully the FTP protocol for client access and for server publishing. To configure ISA to support FTP client access and FTP server publishing, you must first check that the FTP Application Filter is enabled (it should by default). For client access, there must also be a protocol rule that allows the FTP protocol for that client. For FTP server publishing, make sure the FTP Server protocol is selected in the server publishing rule.

Note:apparently you need to allow the FTP server to access to all sites and content. If you don’t have a Site and Content Rule, or if you require authentication to access all sites and content, then the FTP publishing rule will fail. Note that you do NOT need a protocol rule to support publishing an FTP server.

It must be obvious that this way of supporting the FTP protocol gives the highest security. However, keep in mind that the FTP filter only “listens” on the standard FTP control port (TCP port 21). Some people consider this a serious limitation. Personally, I don’t share that opinion. The goal of any protocol standard is to facilitate the communication between totally different systems and the definition of the required port numbers is an integral part of that standard. So, by using alternate port numbers you are violating the standard and that is asking for troubles. Besides, from a security point of view there is no reason to publish FTP sites on alternate ports, unless you actually believe in security through obscurity.

Because a SecureNAT client doesn’t support secondary connections without the help of an application filter, you are not able to access or publish FTP servers on alternate port numbers. If you badly need that functionality, there exists a workaround by using the Firewall client (see 4.3. Firewall client).

Another often asked question is if you can publish an FTP server running on ISA itself. Although this is not a recommended configuration (ISA is supposed to be a firewall, not a general purpose application server), it is indeed possible. How to accomplish this is explained in the following articles:

Now, if something isn’t working as expected, you should consult the ISA logfiles. They are your primary resource for debugging. To get the most information out of the logfiles, I strongly recommend to enable the logging of all fields. In the MMC, go to the node Monitoring Configuration, then select Logs. In the details pane, right-click the applicable service and then click Properties. On the Fields tab, click Select All.
A lot of people seem to have problems with interpreting the logfiles. It isn’t that difficult, but you should first understand what is logged. In the ISA helpfile there is a section called Firewall and Web Proxy log fields, a must read. Additional information can be found in the following articles:

Note: for better readability, I removed the less important fields and abbreviated some of the remaining field names.

4.3. Firewall client

A significant advantage of using the Firewall client is that it enables just about any TCP/UDP based application protocol through ISA without the application itself being firewall or proxy aware. However, the major drawback is that the Firewall client is only available for the operating systems Windows 95 OSR2 and higher. Nevertheless, it should be standard practice to always install the Firewall client on client machines whenever possible.

You have already learned that a SecureNAT client doesn’t support secondary connections without the help of an application filter. For a Firewall client this is not the case. In fact, the Firewall client is a very interesting piece of software and it works hand in hand with the Firewall service on the ISA server. You could say that it is a generic and distributed implementation of the functionality of an application filter, without being restricted to a particular application protocol.

In my article Understanding the Firewall Client Control Channel, (a must read if you want to learn more about how the Firewall client really works) you will find a detailed analysis of how the Firewall client handles a normal FTP outbound session. I will not repeat that information here, but will instead highlight some things you can do to support the FTP protocol on alternate port numbers.

To give a Firewall client user outbound FTP access, you must allow this user access to the predefined FTP protocol definition and make sure that the FTP Application Filter is enabled. Maybe you wonder why the latter is necessary because it seems to conflict with what I said before about the Firewall client functionality. Again, if you look carefully at the predefined FTP protocol definition, you will see that only the primary connection (FTP Control connection) is defined, not the secondary connection (FTP Data connection). So, without the FTP Application Filter enabled, there is no FTP Data connection possible, unless you have of course an allow ‘All IP traffic’ protocol rule enabled.

As mentioned in the section 3.2. SecureNAT client, you should be able to access an FTP server on an alternate port number if the Firewall client is installed on the client machine. The basic configuration steps are:

create a custom FTP protocol definition.

allow the Firewall clients access to this newly created protocol definition.

Remember that a protocol definition only defines which protocols and port numbers might be used and in which direction. However, it is the application itself who will, with the help of the Firewall client, ask the ISA server to open the needed ports dynamically for which protocol and in which direction.

Here is how a custom FTP protocol definition looks like to access an FTP server on the alternate TCP port 2121:

In the above custom FTP protocol definition there are two secondary connections defined. The inbound secondary connection is to support the FTP Data connection in active mode. The outbound secondary connection is to support the FTP Data connection in passive mode.

Here are some excerpts from the firewall log for successful Firewall FTP sessions with the above custom FTP protocol definition:

Note: for better readability, I removed the less important fields and abbreviated some of the remaining field names.

It is important to note that in the above custom FTP protocol definition I use for the inbound secondary connection the port range ‘0-0’. Normally you would expect the port range ‘1025-65534’, but it did not worked as expected on my test configuration. The problem was that the incoming connection was blocked by the packet filtering, although everything looks good up to the s-operation = Listen and ISA was actual listening on the agreed port (checked with the command netstat -an). Changing the port range to ‘0-0’ solved that problem. So, any information that can shed more light on that behaviour is more than welcome.

As already noted before, you cannot server publish an FTP server on an alternate port in the usual way. However, if you can install the Firewall client on that FTP server, there is a workaround possible. How to do it is explained in the following articles:

Note 1: apparently you need a Site and Content Rule that allows the FTP server access to all sites and content, because the FTP server is establishing a new connection.

Note 2: keep in mind that you cannot install the Firewall client on the ISA server itself or on an FTP server who is not running on the Windows operating system. Hence, for those particular configurations you cannot publish an FTP server on an alternate port.

4.4. Web Proxy client

How the browser Internet Explorer handles the FTP protocol depends on the version of Internet Explorer and the type of ISA client the machine is configured for. It is important to note that asingle machine can be configured as a SecureNAT, Firewall and Web Proxy client without any adverse interaction between the client configuration settings. In this section we will look at Internet Explorer version 6.0. For older versions check out the article Issues with the Internet Explorer FTP Client by Thomas Shinder.

I assume you have a client machine configured as a Web Proxy, a Firewall client and a SecureNAT client at the same time. Now, have a look at the Advanced Internet Options in IE 6.0. There you will find two important settings that influence how IE handles the FTP protocol:

Enable folder view for FTP sites.

Use Passive FTP (for firewall and DSL modem compatibility).

If the setting Enable folder view for FTP sites is not checked, then the FTP request is sent by IE to the Web Proxy service on ISA. This is done through the HTTP protocol and there is no browser method or user setting available to specify passive or active FTP mode. The Web Proxy service on ISA will then forward this as a real FTP session in active mode. However, keep in mind that the Web Proxy service on ISA only supports FTP downloads.

As an example, here is an Ethereal trace of an FTP request sent to the Web Proxy service on ISA:

If the setting Enable folder view for FTP sites is checked, then the FTP request is sent by IE to the Firewall service on ISA. In other words as a real FTP session. If the Firewall client is installed and enabled, this request is intercepted and handled by the Firewall Client. However, if the Firewall client is disabled or not installed, the request is sent as from a SecureNAT client. Which FTP mode active or passive IE will use is determined by the setting Use Passive FTP (for firewall and DSL modem compatibility).

As an example, here is an Ethereal trace of an FTP request sent as a Firewall client to the Firewall service on ISA:

As an example, here is an Ethereal trace of an FTP request sent as a SecureNAT client to the Firewall service on ISA:

As you can see in the above examples, the way Internet Explorer version 6.0 sends the FTP request to the ISA server is completely different for the three situations. This means also that some FTP requests will be found in the Web Proxy log and other ones in the Firewall log.

5. What about Secure FTP

The document RFC2228 defines Security Extensions to the FTP protocol. These extensions provide strong authentication, integrity, and confidentiality on both the control and data channels with the introduction of new optional commands, replies, and file transfer encodings.

A widely implemented security extension to the FTP protocol is the use of the SSL (Secure Sockets Layer) version 3.0 or TLS (Transport Layer Security) version 1.0 protocol. It is documented in the Internet-Draft Securing FTP with TLS. The SSL/TLS protocol runs above the transport layer (TCP/IP), but below the application layer protocol. Therefore, it is relativily easy to implement for securing all sort of application protocols such as HTTP, Telnet, POP3, IMAP4, SMTP and FTP.

In actual FTP client and server implementations there exists at least two distinct mechanisms by which the SSL security extension is initiated: explicit (active) and implicit (passive) security:

Explicit Security: In order to establish the SSL link, explicit security requires that the FTP client issues a specific command to the FTP server afterestablishing a connection. The default FTP server port is used. This formal method is documented in RFC 2228.

Implicit Security: Implicit security automatically begins with an SSL connection as soon as the FTP client connects to an FTP server. In implicit security, the FTP server defines a specific port for the client (TCP Port 990) to be used for secure connections.Note: Implicit SSL is discussed in various SSL drafts but is not formally adopted in an RFC. For strict compliance to standards, use the explicit method.

You can think of implicit security as “always on” and explicit security as “turn on”. The following nice drawing, taken from the GlobalScape Inc. website, contrasts implicit and explicit SSL connections:

When using FTP over SSL/TLS, in short FTPS, the control connection is encrypted and therefore unreadable for entities other than the FTP client and the FTP server. As a consequence, NAT/PAT devices and firewalls can no longer monitor the negotiation of the data connection and do smart things with that information. This places some serious limitations on the practical use of the FTPS protocol through all sort of firewalls and Network Address Translators.

Note: a good discussion about some of the issues with running FTP, secured with SSL/TLS, through firewalls can be found in the Internet-Draft FTP/TLS Friendly Firewalls. Keep in mind that an Internet-Draft is “work in progress”.

It must be clear that the FTP Application Filter on the ISA server cannot help you in this situation. Therefore, you will have a hard time to support FTPS through ISA for SecureNAT clients. In fact, a lot of firewalls will have a problem to support in a secure way fully the FTPS protocol (PORT and PASV mode). However, with the ISA server you should be able to fully support FTPS if the Firewall client is installed on the client machine. In that case you must follow the configuration steps outlined for FTP client access to an FTP server on an alternate port number and for server publishing an FTP server on an alternate port number.

For FTPS client access, the custom FTP protocol definition I refer to (see section 4.3. Firewall client) should be slightly adopted. For the mode Implicit Security the primary connection should be TCP port 990 Outbound. If you have to support also the mode Explicit Security then some more work has to be done. Because Explicit Security uses the standard FTP Control port TCP port 21, you need to first disable the FTP Application Filter. Otherwise, the FTP Application Filter would interfere with the FTPS Control connection. Next, in the custom FTP protocol definition use as primary connection TCP port 21 Outbound. As a consequence, SecureNAT clients will no longer be able to access FTP servers.

6. Conclusion

The FTP protocol is a somewhat messy protocol that definitely wasn’t designed with firewall security in mind. FTP clients can use one of the two modes: PORT and PASV mode. While PORT mode has been around for a longer time, PASV mode is more popular because it doesn’t require a new inbound connection to be made to the FTP client from the FTP server. Modern firewalls and NAT devices have components that are able to handle the connection request details contained in the PORT and PASV commands to improve on the security issues related to the FTP protocol.

I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=3;t=002268 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! – Stefaan.

SUBSCRIBE

Get the most recent articles straight to your inbox!

Recommended

Follow Us

How the FTP protocol Challenges Firewall Security

TECHGENIX

TechGenix reaches millions of IT Professionals every month, and has set the standard for providing free technical content through its growing family of websites, empowering them with the answers and tools that are needed to set up, configure, maintain and enhance their networks.