How to Secure Optimized Networks

Joel Snyder, Ph.D., is a senior IT consultant with 30 years of practice. An internationally recognized expert in the areas of security, messaging and networks, Dr. Snyder is a popular speaker and author and is known for his unbiased and comprehensive tests of security and networking products. His clients include major organizations on six continents.

A long-simmering feud between network and security managers is heating up over visibility and performance.

Network managers strive to deploy fast and resilient WANs for distributed organizations. The problem is that some of the best tools available to optimize networks, such as compression, protocol optimization, load balancing and dynamic routing, can wreak havoc with proxies, data loss prevention (DLP), intrusion prevention systems (IPS) and firewalls.

To keep networks and data as secure as possible, consider these four tips:

1. Order functions correctly.

In most networks, firewalling and VPN should be at the outer edge, while IPS and DLP should occur as close to users and servers as possible. WAN optimization goes between the two. Thus, user traffic should hit the IPS or DLP system first, then pass through optimization, before finally traversing the firewall and moving out onto the WAN or Internet.

The same is true of a server: Traffic should go from the server to any security devices, then optimization, load balancing and acceleration, and finally hit the firewalls.

Mixing up that order will cause gaps. For example, unified threat management (UTM) firewalls have IPS built in, but an IPS cannot properly function on traffic that has been compressed. This means that optimized networks will not get the best results from IPS functions in a UTM firewall; they need dedicated IPS devices that can see traffic before it’s encrypted and optimized.

IPS manufacturers prefer this location anyway because the IPS can give best results when it sees network traffic as if it were end system (such as a PC, notebook or server), reducing effects of load balancing, network fragmentation and reordering.

2. Try not to do things twice — or three times.

Optimization devices must decrypt traffic in order to compress and cache it, which calls for man-in-the-middle decryption of all SSL/TLS traffic on the WAN. The same is true of next-generation firewalls, which need to decrypt traffic to identify application layer information and apply controls. And IPS solutions have the same problem — without decrypted traffic, they cannot be fully effective. Decrypting and re-encrypting twice or even three times will slow traffic down and cause problems.

Network and security managers who plan to use devices that require man-in-the-middle decryption should deploy products that can work together. This can limit product selection options, but it’s better to work out interoperability early rather than having to start over.

3. Identify key monitoring and control points.

For highly optimized networks, it’s better to have multiple smaller IPS devices instead of one enormous centralized device that is partially blinded by encryption. When traffic flows through multiple IPS devices, security managers should be sure to write rules so that traffic is only scanned once at the most appropriate place. This improves performance and efficacy while reducing false positives.

For example, many application managers have used sophisticated application delivery controllers to load balance, increase reliability and scalability, and optimize application delivery. In most cases, these devices can also perform SSL/TLS offloads, handle encryption on the outside and pass unencrypted traffic to the application, speeding server performance as well. The short path between the application delivery controller and the servers is the perfect place to put IPS and DLP functionality.

4. Closely watch dynamic routing.

When building optimized networks, look out for the effects of dynamic routing. Network managers build networks to keep packets flowing, but this can cause both short-term and long-term asymmetric traffic flows. From a networking point of view, that’s fine, but from a security point of view, it can be a problem. Any good firewall will block asymmetric traffic by default, making the firewall responsible for network outages.

Security managers can work around this issue in several ways. Most firewalls will allow asymmetric traffic if they’re specifically configured to do so. They should not do this out of the box — that’s a sign of a broken firewall — but manufacturers have recognized this problem and usually have an option to allow asymmetric flows. A better option is to be aware of the potential for asymmetric flows.

Network and security staff should work together during design and upgrade planning to watch out for these potential problems. That should make it easier to place firewalls and firewall clusters so that any asymmetry is invisible to the firewalls. The same advice applies to optimization devices, which cannot do their job properly if traffic flows aren’t symmetric.