Community Area

Configuring ISA Server 2000 to Support Outlook 2003 RPC over HTTP - Part 2: Forcing SSL on the RPC Directories and Configuring IPSec Security on the Front-end and Back-End Exchange Servers

by
Thomas Shinder
[Published on 30 Dec. 2003 / Last Updated on 20 May 2013]

In the first part of this series on configuring ISA Server 2000 firewalls to support Outlook RPC over HTTP client connections we went over how to configure some of the core network infrastructure components to support the RPC over HTTP publishing solution. We also discussed how to install the RPC over HTTP proxy service on the front-end Exchange Server and how to issue a Web site certificate to the RPC over HTTP Web server. We continue the adventure by showing you how to force SSL on the RPC directory, configure the Registry entries on the front-end Exchange Server, and enforce IPSec encryption between the front-end and back-end Exchange Servers.

In the first part of this series on configuring ISA Server 2000 firewalls to support Outlook RPC over HTTP client connections we went over how to configure some of the core network infrastructure components to support the RPC over HTTP publishing solution. We also discussed how to install the RPC over HTTP proxy service on the front-end Exchange Server and how to issue a Web site certificate to the RPC over HTTP Web server.

Configure IPSec Policy to secure all communications between the front-end and back-end Exchange Server

Just as a reminder, the figure below shows the basic network configuration covered in this series on RPC over HTTP publishing.

Force SSL and Basic Authentication on the RPC Directory

The external Outlook 2003 clients will use SSL to connect to the external interface of the ISA Server firewall. In addition, the ISA Server firewall uses SSL to protect the connection between the internal interface of the firewall and the front-end Exchange Server’s Web site. The Outlook 2003 client will use basic authentication to authenticate with the ISA Server firewall’s Incoming Web Requests listener. The firewall forwards these basic authentication credentials to the front-end Exchange Server’s Web site.

You need to force basic authentication on the front-end Exchange Server’s RPC directory to insure the best performance and widest compatibility. User credentials are protected by SSL encryption so you do not need to worry about the clear text credentials being captured by an intruder. In addition, we will force an SSL connection between the firewall and the RPC directory. This prevents any host, including the firewall, from using a non-secure connection to the front-end server’s RPC directory.

Perform the following steps to force basic authentication and SSL encryption on the front-end Exchange Server’s RPC directory:

Click Start, point to Administrative Tools and click on Internet Information Services (IIS) Manager. In the Internet Information Services (IIS) Manager, expand your server name and then expand the Web Sites node. Click the Rpc node in the left pane of the console and right click an empty area in the right pane. Click on the Properties command.

Click on the Directory Security tab in the Rpc Properties dialog box. Click on the Edit button in the Secure communications frame.

You will use fully qualified domain names in this Registry entry. The internal network DNS infrastructure should already be in place because these machines all belong to an Active Directory domain. The front-end Exchange Server must be able to resolve the FQDNs of the back end Exchange and Global Catalog servers.

Note: You do not need to configure specific ports to be used on the Global Catalog server because the Global Catalog server and the Exchange Servers are ISA LAT hosts. A LAT host is any computer on the internal network that has its IP address on the LAT. LAT hosts have full access to each other and communications between them are not screened by firewall policy. Never put a front-end Exchange Server on a DMZ segment. When the front-end Exchange Server is deployed on a DMZ segment, it exposes your internal network security zone to attackers.

Perform the following steps to configure the Registry on the front-end Exchange Server:

Click Start and then click the Run command. Type regedit in the Open text box and click OK. In the Registry Editor, drill down to the following Registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\RpcProxy

Right click on the ValidPorts value and click Modify

Enter the information in the Value Data text box for the back-end Exchange Server and the Global Catalog server using the format:

Configure IPSec Policy to Secure All Communications Between the Front End and Back End Servers (optional)

The link between the Outlook 2003 client and the external interface of the ISA Server firewall is protected by SSL encryption. In addition, the link between the internal interface of the ISA Server firewall and the front-end Exchange Server is protected by SSL encryption. The problem is that the link between the front-end and back-end Exchange Server is not protected by SSL encryption. You can not use SSL between the front-end and back-end Exchange servers.

You can solve this problem by applying IPSec Policies to force a secure, encrypted link between the front-end and back-end Exchange servers. The IPSec encrypted link protects messages moving between the front-end and back-end, and provides the end to end encryption the Outlook 2003 client implicitly expects when it creates the secure link with the external interface of the ISA Server firewall.

There are several ways you can implement IPSec Policies. The two most popular methods are:

Use local security policy on the front-end and back-end Exchange servers.

There are several ways to enforce authentication between the front-end and back-end Exchange Servers when they establish their IPSec links. In this following example we assume that the front-end and back-end Exchange Servers have received machine certificates from the same CA. We will use certificate authentication as the preferred authentication method in the IPSec Policy.

Perform the following steps to create the IPSec transport mode secure connection between the front-end and back-end Exchange Servers:

Go to the front-end Exchange Server, click Start, point to Administrative Tools and click on Local Security Settings. In the Local Security Settings console, click on the IP Security Policies on Local Computer node in the left pane of the console. Right click on an empty area in the right pane of the console and click the Create IP Security Policy command.

Click Next on the Welcome to the IP Security PolicyWizard page.

On the IP Security Policy Name page, give the policy a name in the Name text box. Enter an optional description for the policy in the Description text box. Click Next.

On the Requests for Secure Communication page, remove the checkmark from the Activate the default response rule checkbox and click Next.

On the Completing the IP Security Policy Wizard page, put a checkmark in the Edit properties checkbox and click Finish.

The Properties dialog box for the policy appears. Keep in mind that a security policy consists of a set of rules. If any of the conditions we set in any of the rules matches a connection, then the settings of the rule are triggered. The only rule included in the policy at this point is the default response rule, but it is not selected and we will not select it. Instead, we will add our own rule. Make sure that there is a checkmark in the Use Add Wizard checkbox and click the Add button.

Click Next on the Welcome to the Create IP Security Rule Wizard page.

We are creating a transport mode connection between the front-end and back-end Exchange Servers. Therefore, on the Tunnel Endpoint page select the This rule does not specify a tunnel option. Click Next.

Select the All network connections option on the Network Type page. Click Next.

An IP filter list contains IP addresses, computer names, or network IDs and protocol information. When a connection matches a connection specified in the IP filter list, then a filter action is applied. In this example we will create an IP filter list that contains the source IP address being the back-end Exchange Server and the destination IP address being the front-end Exchange Server. We will also configure the IP filter list to match all protocol connections inbound from the back-end Exchange Server to the front-end Exchange Server.

On the IP Filter List page, click the Add button. In the IP Filter List dialog box, type a name for the filter list in the Name text box. Type an optional description in the Description text box. To add filters to the list, make sure there is a checkmark in the Use Add Wizard checkbox and click the Add button.

Click Next on the Welcome to the IP Filter Wizard page.

On the IP Filter Description and Mirrored property page, type in a description of the filter in the Description text box. You should enter information about the nature of the filter, such as the source and destination addresses and the protocols. In our example, we have a single front-end and back-end Exchange Server, so we’ll just call it OWA.

Put a checkmark in the Mirrored. Match packets with the exact opposite source and destination addresses option. This will allow the trigging of the filter action when packets moving in the opposite direction as specified in the filter as detected by the IPSec Policy Agent.

On the IP Traffic Source page, select the A specific IP Address option in the Source address drop down list. In the IP Address text box, type in the IP address of the back-end Exchange Server. You will have to repeat the step if you have multiple back-end Exchange Servers. Click Next.

On the IP Protocol Type page, left the default setting of Any in the Select a protocol type drop down list. We want to match all packets moving between the front-end and back-end Exchange Servers, so we match all protocols with this filter list entry. Click Next.

Do not put a checkmark in the Edit properties checkbox, then click Finish on the Completing the IP Filter Wizard page.

The IP filter that matches all protocols for communications between the front-end and back-end Exchange Server. You have completed the configuration of the IP filter list that matches the packets moving between the front-end and back-end Exchange Server. Click OK to continue create the rule that secures the connection between the front-end and back-end Exchange Servers.

On the IP Filter List page, select the IP filter list you created in the IP Filter lists box. Click Next.

On the Filter Action page, select the Require Security option from the Filter Actions list.

When a packet matches the source and destination, as well as protocol included in the filter list, then a Filter Action is triggered. We are creating a rule that includes a filter list we created that matches all protocol packets between the front-end and back-end Exchange Servers. When packets match these conditions, then the Filter Action is applied. The Require Security filter action will require that all packets moving between the front-end and back-end Exchange Server are security with IPSec encryption.

Click Next.

On the Authentication Method page, you can choose from Active Directory default (Kerberos V5 protocol), Use a certificate from this certification authority (CA) and Use this string to protect the key exchange (preshared key) options.

You and use the Active Directory default option if the front-end and back-end machines belong to the same domain. You can enhance security by using the Use a certificate from this certification authority (CA) option. The front-end and back-end Exchange Server must have machine certificates from the same CA (or in a more complex environment, trust the CAs that issued each other’s machine certificates).

In the current example, we have already deployed machine certificates to both the front-end and back-end Exchange Servers, so select the Use a certificate form this certification authority (CA) and click Browse.

On the Select Certificate dialog box, find the CA certificate for your root CA on the internal network and select it. Then click OK.

The CA information appears in the text box. Click Next.

Put a checkmark in the Completing the Security Rule Wizard checkbox and click Finish.

Click OK on the New Rule Properties dialog box.

On the Properties dialog box for the policy, click the General tab. You can change the Name and the Description for this rule on this tab. You can click the Settings button and change core characteristic of the how key exchange is performed. Click OK.

You will need to repeat the procedure on the back-end Exchange Servers. The only difference between the policy you created on the front-end Exchange Sever and the back-end Exchange Servers is few settings in the IP Filter. The figure below shows the IP Traffic Source page of the IP Filter Wizard. On the back-end Exchange Servers, make sure you use the back-end Exchange Server’s address in the source address text box.

The Destination address on the back-end Exchange Servers is set for My IP address.

The protocol type on the back-end Exchange Server’s is Any. Again, the same as what we did with the front-end Exchange Servers.

After the IPSec Policies are created on the front-end and back-end Exchange Servers, right click on the policy you created to secure communications between the front-end and back-end Exchange Servers and click the Assign command.

When the policy is assigned, you will see a Yes entry in the Policy Assigned column.

You can test the policy by opening a command prompt and typing the command:

ping –t w.x.y.z

where w.x.y.z is the IP address of another Exchange Server included in the IP filter list. You will see the ping command report that it is negotiating IP Security and then you will see ping replies. You can see details of the IPSec negotiation and packet exchange by using the IP Security Monitor standalone mmc console snap-in.

Summary

In this, part 2 of our series on RPC over HTTP publishing using ISA Server 2000 firewalls, we started with how to force SSL on the RPC over HTTP proxy directory. Then we configured the Registry entries on the front-end Exchange Server that are required to identify the back-end Exchange Servers and Global Catalog servers on the internal network. Finally, we went through the step by step procedures required to enforce IPSec transport mode security for all communications between the front-end and back-end Exchange Servers.

I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=5;t=002297 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom

If you would like us to email you when Tom Shinder releases another article on ISAserver.org, subscribe to our 'Real-Time Article Update' by clicking here. Pleasenote that we do NOT sell or rent the email addresses belonging to our subscribers; we respect your privacy!