Should You Virtualize Exchange 2007 SP1?

Topic Last Modified: 2009-03-02

With the release of Windows Server 2008 with Hyper-V technology and Microsoft Hyper-V Server 2008, a virtualized Microsoft Exchange Server 2007 Service Pack 1 (SP1) server is no longer restricted to the realm of the lab; it can be deployed in a production environment and receive full support from Microsoft. In August 2008, Microsoft published support policies and recommendations for virtualizing Exchange. This article answers the more philosophical question: Is virtualization a good idea when it comes to Exchange?

Due to the performance and business requirements of Exchange Server, most deployments benefit from deployment of Exchange server roles on physical servers. However, there are some scenarios in which running Exchange in a hardware virtualization environment may allow you to achieve real benefits in terms of space, power, and deployment flexibility. Presented in this article are:

Scenarios This section discusses four scenarios in which virtualization of Exchange 2007 SP1 may make sense.

Technical Checklists This section provides checklists to help you evaluate whether the current load on your infrastructure makes it a viable candidate for virtualization.

Some organizations are small, but they still require enterprise-class availability. For example, consider Contoso, Ltd., a fictitious company that regards e-mail as a critical service. Contoso has several small branch office sites consisting of 250 users. Contoso wants to keep their e-mail environment on premises for legal reasons, and they want to have a fully redundant e-mail system. Contoso’s users have average user messaging profiles with 2 GB mailboxes.

To achieve full redundancy for all Exchange server roles using physical hardware, Contoso would have to deploy seven servers: two servers for Active Directory and DNS, one server for file and print services, two servers running both the Client Access and Hub Transport server roles, and two servers running the Mailbox server role in a cluster continuous replication (CCR) environment. These servers are assumed to have two quad-core processors and the amount of RAM is based on the Microsoft recommendations for each installed role. Each CCR node has 4 GB of RAM, and each of the other servers has a minimum 2 GB of RAM to support the users and traffic patterns. With average user messaging profiles, the typical load should be 35%-45% of CPU on a server that has eight cores.

By using virtualization, Contoso can give the same level of redundancy and availability with only three servers. Each physical server would run multiple virtualized servers as guest machines. In this scenario, three physical servers with two quad-core processors and 16 GB of RAM would have sufficient capacity to serve Contoso’s users. Each of the virtual servers would be configured with two virtual CPUs. Along with RAM and processors, the servers also need to have multiple network adapters and redundant paths for storage. Because Contoso still has the same number of Exchange servers to manage, they haven't benefitted from an operations and maintenance perspective, but there are benefits from a space, power, and cooling perspective.

The following diagram illustrates the scenario.

Note:

The Hub Transport server hosting the file share witness (FSW) for the CCR environment is not a guest of the same hypervisor root machine as either node in the cluster. This is done intentionally to eliminate any single point of failure in the clustering solution and to provide true clustering capability.

In this scenario, when moving from seven physical servers to three physical servers and seven virtual servers, there is a potential savings of 25,754 kWh and $22,516 per year. The Microsoft Virtualization Tool was used to gather these numbers.

Sizing Exchange for a hardware virtualization environment is no different from sizing Exchange for physical servers. Whether you are using physical or virtualized servers, each CCR node needs 4 GB of RAM and storage capable of supporting 48 IOPS for the databases and 19 IOPS for the transaction logs. In this scenario, the IOPs requirements are very low and should be maintainable in a virtualized environment. Using fixed virtual hard drives would be sufficient for this solution, given the small amount of users that are hosted on virtualized Exchange servers. For higher user counts, Microsoft recommends that you use external storage.

In the past, some organizations required local Exchange servers in remote or branch offices to provide sufficient performance for the users in those offices. With improvements such as Cached Exchange Mode and Outlook Anywhere, consolidating those servers to a central datacenter became the recommended approach.

In some situations, however, poor network connectivity to remote offices still requires some organizations to have a local Exchange server in those offices. Often the user populations at these locations are so small that it doesn’t make sense to dedicate a physical server to the Exchange environment. The technical considerations in this scenario are the same as described in the “Small Office with High Availability” scenario above. For an example of this scenario where a company used Exchange 2007 SP1 in a Hyper-V environment, download this Windows Server 2008 Customer Solutions Case Study about the Caspian Pipeline Consortium.

To provide site resilience and redundancy for a production site, some organizations may require a warm site that contains a duplicate of the primary production Exchange 2007 infrastructure. The intent of this warm site is to provide as near to the same level of functionality of the primary site as possible, in the event of the loss of the primary site. However, keeping a duplicate infrastructure for standby purposes, although useful for high service level agreement (SLA) requirements, can be prohibitively expensive for some organizations.

To reduce these costs, it's possible to provide a duplicate of the entire primary site using guest machines in a Hyper-V environment. A typical warm site configuration using physical Exchange 2007 servers would include one or more servers configured together as a standby cluster, and one or more other servers configured as the Client Access and Hub Transport server roles. To achieve redundancy of just the messaging services within the warm site, four physical servers would be needed. By contrast, a virtualized solution with only three physical servers can provide an organization with a warm site that includes two Mailbox servers in a CCR environment, as well as redundant Client Access and Hub Transport servers. Thus, by virtualizing Exchange in this scenario, you can provide a higher level of services to your users while saving on space requirements and hardware, power, and cooling costs when compared to a similarly configured solution deployed on physical servers.

The following diagram illustrates this configuration.

The Primary Site uses physical hardware due to the demanding size and messaging profile of the user population. In this scenario, the Warm Site is designed to support the entire user population from the Primary Site. Even though use of the Warm Site is for a temporary period and at reduced performance, careful consideration must still be given to the configuration of the environment that will support the user population.

The diagram illustrates that the Warm Site would contain a standby cluster, with one of its nodes configured as a standby continuous replication (SCR) target for the production CCR environment. A pair of domain controllers is also deployed. The SCR target is a two-node standby cluster. In the event of a site failure, the standby cluster would be activated by using Restore-StorageGroupCopy, and then the clustered mailbox server (CMS) would be recovered by using the Setup /recoverCMS switch. The same procedures for recovering from a disaster using a standby cluster still apply despite the fact that the standby cluster is a guest machine in a hardware virtualization environment. After the standby cluster is online and hosting the CMS from the failed site, client access to messaging services and data will be restored after DNS and Active Directory replication has occurred.

The virtual Warm Site must be able to provide an adequate level of service to users in the event of the loss of the Primary Site, with the understanding that there will probably be a reduced level of service due to the WAN/Internet link(s) to the Warm Site. However, because the site is designed to provide emergency functionality, and only for a brief period, this reduced level of service should be acceptable. It would be understood, however, that while the Primary Site is down, there is no site resilience for the Warm Site.

In this scenario, when moving from eight physical servers to three physical servers and eight virtual servers, there is a potential savings of 33,005 kWh and $28,225 per year. The Microsoft Virtualization Tool was used to gather these numbers.

Sometimes a company, agency, or governmental department may need a complete network infrastructure that can be deployed to specific locations at a moment’s notice. This infrastructure is then connected to the organization’s network via satellite or other remote WAN technology. For example, a non-governmental organization (NGO) may need to respond to a disaster and set up local servers to serve an affected community. This subset of servers would need to be completely self-contained and able to provide all necessary services to the personnel located in the target location.

In such situations, the mobile LAN must be easy to transport. It must have a small and highly efficient form factor, providing all of the local users’ necessary services while taking up the smallest amount of space possible. Given the probable remoteness of the locations in which the Mobile LAN would be deployed, it must also provide fault-tolerant capabilities to ensure that there is no single point of failure in a location where spare parts may be hard to come by.

In this scenario, a Hyper-V server can be used to host Exchange services, file server services, and domain infrastructure services in a compact form factor.

Note:

Virtualization of Exchange 2007 SP1 requires that no IO-intensive applications be installed on the root server.

The following diagram illustrates a possible configuration for a Hyper-V server hosting the Exchange 2007 and domain infrastructure systems. Due to the nature of this scenario, none of the Exchange server roles have been combined on the same guest machine.

The diagram illustrates a comprehensive network solution for an organization that requires all necessary server services to be provided locally regardless of location. The solution is as small as possible while also allowing for a high degree of fault tolerance and system availability.

In the rack, you would also need enough network infrastructure equipment to support the hypervisor root servers and the workstations. The hypervisor root systems would use either an iSCSI or Fiber Channel SAN. The SAN should give enough spindles to provide the necessary performance for the guest systems. In this scenario, everything needed would fit in a single 42U rack.

In this scenario, when moving from 14 physical servers to three physical servers and 14 virtual servers, there is a potential savings of 91,012 kWh and $73,891 per year. The Microsoft Virtualization Tool was used to gather these numbers.

In each of the scenarios described above, if Exchange were deployed on physical hardware, resources would have been underutilized. You can use the following checklists to help determine if your Exchange environment is a candidate for consolidation. If these checklists reveal that your hardware is underutilized, you should consider the following possible actions:

If you are a small organization, you may be able to reduce your physical server footprint to as few as three physical servers with full redundancy of Exchange roles by using virtualization.

If the underutilized hardware is at a branch or remote office that can't be consolidated to a central datacenter, you may be able to reduce your physical server footprint at that location by using virtualization.

In other scenarios, you may want to make your Exchange environment a bit leaner by revisiting how well your server capacity corresponds to your user load. You can reduce the number of physical servers to boost utilization to the desired levels. In this scenario, using virtualization isn't required.

Keep in mind that underutilized hardware simply means that your Exchange environment has excess capacity. This situation may be by design (to accommodate usage spikes or expected growth) or by accident. Some excess capacity is desirable, and that factor has been considered in the following checklists.

The following checklist outlines performance counters to monitor that may indicate whether your Exchange environment is underutilized. These counters need to be gathered from physical Exchange servers, not virtualized Exchange servers. Because the planning for the Exchange infrastructure is based on the Mailbox server role, the checklists below use mostly Mailbox server metrics.

We recommend that you collect each of these counters at regular intervals for a minimum of one week. After the data has been gathered, you can compare the results to the expected values. If the observed values are less than the expected values below, then your server hardware is being underutilized.

We recommend that you monitor processor performance on Windows Server 2008 to verify that the operating system isn't slowing down your processors frequency. That situation could lead you to believe you have high CPU utilization, when in fact you have low CPU utilization and the processors are just throttled down to save power.

Performance counter checklist

Category

Indicator

Expected value

Present

Common Performance Counters (All Exchange Servers)

Processor% Total

Should be less than 40 % average.

[ ]

System\Processor Queue Length (all instances)

Should not be greater than 5 per processor.

[ ]

Network Interface(*)\Bytes Total/sec

For a 1000-Mbps network adapter, should be below 30-35 Mbps.

[ ]

Mailbox Server-Specific Performance Counters

MSExchangeIS Client (*)\RPC Average Latency

Should be less than 30 ms on average.

[ ]

Process(Microsoft.Exchange.Search.ExSearch)\% Processor time

Should be less than 1% of overall CPU typically and not sustained above 3%.

[ ]

MSExchange Store Interface(_Total)\RPC Latency average (msec)

Should be less than 100 ms at all times.

[ ]

MSExchange Store Interface(_Total)\RPC Requests outstanding

Should be 0 at all times.

[ ]

CCR, LCR, and SCR Mailbox Server -Specific Performance Counters

MSExchange Replication(*)\CopyQueueLength

Should be less than 10 at all times for CCR and SCR.

Should be less than 1 at all times for local continuous replication (LCR).

A less time-consuming (and less precise) way of judging whether your servers are underutilized is to compare processor cores to user profiles using the general sizing guidelines provided in Planning Processor Configurations. To determine the user profile of your organization, refer to the user profile table in that article (also reprinted below for your convenience).

User type (usage profile)

Send/receive per day approximately 50-kilobyte (KB) message size

Light

5 sent/20 received

Average

10 sent/40 received

Heavy

20 sent/80 received

Very heavy

30 sent/120 received

As noted in Planning Processor Configurations, a rule of thumb for sizing is that 1,000 active average profile mailboxes will require a one processor core. For example, a 4,000-mailbox server with an average usage profile requires four processor cores. A heavy usage profile requires more processor cycles than an average profile, thus for planning purposes, use 750 active, heavy-profile mailboxes per processor core. Using this logic, we can summarize in the following checklist how many users are needed to fully utilize a processor core:

User profile factors checklist

Category

Indicator

Expected value

Present

Light user profile

Recommended Number \ Processor Core = 2000

≤1,000

[ ]

Average user profile

Recommended Number \ Processor Core = 1000

≤500

[ ]

Heavy user profile

Recommended Number \ Processor Core = 750

≤375

[ ]

Very heavy user profile

Recommended Number \ Processor Core = 500

≤250

[ ]

Because the recommended number of average users is 1,000 per processor core, having an active user population of 500 or less average profile mailboxes would indicate that there aren't enough users to fully utilize one Mailbox server processor core. Keep in mind that, for physical Exchange servers, eight processor cores is the maximum number efficiently used by the Mailbox server role (see Planning Processor Configurations). Deploying mailboxes on servers with more than eight cores will not provide significant scalability improvements.

Using this checklist to gauge the number of Mailbox server processors core is a good starting point for reviewing other Exchange server roles because the Exchange infrastructure planning methodology for Hub Transport and Client Access servers is in part based on the processor core ratio to the Mailbox server (Mailbox:Hub and Mailbox:CAS). For example, the Mailbox:Hub server ratio is 5:1 if the Hub Transport servers are running transport antivirus software, and 7:1 if the Hub Transport servers aren't running transport antivirus software. Therefore, if a user population isn't likely to fully utilize one Mailbox server processor core, then it likely will not fully utilize a Hub Transport processor core. This finding can indicate that using virtualization for the Hub Transport and/or the Client Access servers may be warranted.

With the release of Windows Server 2008 with Hyper-V and, more recently, Microsoft Hyper-V Server 2008, you have new options for deploying Exchange Server 2007 SP1. In many situations, keeping Exchange on physical hardware provides better manageability and a lower total cost of ownership (TCO) than using virtualization. However, there are some scenarios in which a virtualized Exchange infrastructure may allow you to realize real benefits in terms of space, power, and deployment flexibility.

The degree to which your current hardware is under-utilized is a key factor in determining whether the benefits of virtualization outweigh the complexities that are introduced by adding a virtual layer beneath Exchange. The benefits of virtualization are usually reserved for environments in which the Exchange deployment is too small to fully tax the underlying servers. Small Exchange deployments, remote sites with poor connectivity, certain disaster recovery scenarios, and Mobile LAN deployments are examples of good candidates for virtualization.

In most organizations, Exchange is a mission-critical application. If you remember this as you design your virtualized environment, and follow the Microsoft support policies and recommendations for virtualized Exchange servers, you will set yourself up for success.