Understanding Back Pressure

Back pressure is a system resource monitoring feature of the Microsoft Exchange Transport service that exists on computers that are running Microsoft Exchange Server 2007 that have the Hub Transport server role or Edge Transport server role installed. Important system resources, such as available hard disk drive space and available memory, are monitored. If utilization of a system resource exceeds the specified limit, the Exchange server stops accepting new connections and messages. This prevents the system resources from being completely overwhelmed and enables the Exchange server to deliver the existing messages. When utilization of the system resource returns to a normal level, the Exchange server accepts new connections and messages.

The following system resources are monitored as part of the back pressure feature:

Free space on the hard disk drive that stores the message queue database.

The number of uncommitted message queue database transactions that exist in memory.

The memory that is used by the EdgeTransport.exe process.

The memory that is used by all processes.

For each monitored system resource on a Hub Transport server or Edge Transport server, the following three levels of resource utilization are applied:

Normal The resource is not overused. The server accepts new connections and messages.

Medium The resource is slightly overused. Back pressure is applied to the server in a limited manner. Mail from senders in the authoritative domain can flow. However, the server rejects new connections and messages from other sources.

High The resource is severely overused. Full back pressure is applied. All message flow stops, and the server rejects all new connections and messages.

We strongly discourage any modifications to the back pressure settings in the EdgeTransport.exe.config file on a production server. Modifications to the back pressure settings may result in poor performance or data loss. We recommend that you investigate and correct the root cause of any back pressure events that you may encounter before you modify the back pressure settings.

All configuration options for back pressure are available in the EdgeTransport.exe.config application configuration file that is located in the C:\Program Files\Microsoft\Exchange Server\Bin directory. The EdgeTransport.exe.config file is an XML application configuration file that is associated with the EdgeTransport.exe file. EdgeTransport.exe and MSExchangeTransport.exe are the executable files that are used by the Microsoft Exchange Transport service. This service runs on every Hub Transport server or Edge Transport server. Changes that are saved to the EdgeTransport.exe.config file are applied after the Microsoft Exchange Transport service is restarted. The default value is enforced if either of the following conditions is true:

A configuration option is missing.

A configuration option is present and contains the default value.

The following example shows the typical structure of the EdgeTransport.exe.config file:

<configuration>

<runtime>

<gcServer enabled="true" />

</runtime>

<appSettings>

<add key="Configuration Option" value="Value" />

...

</appSettings>

</configuration>

You can add new configuration options or modify existing configuration options in the <appSettings> section. Many configuration options available are completely unrelated to back pressure. Any configuration options that don't involve back pressure are outside the scope of this topic.

Note:

The parameter names in the <add key=../> section are case sensitive.

Table 1 shows the general configuration options for back pressure that are available in the EdgeTransport.exe.config file.

Table 1 General back pressure options

Parameter name

Description

EnableResourceMonitoring

This parameter enables or disables back pressure. The default value is TRUE.

ResourceMonitoringInterval

This parameter controls how frequently the system resource utilization levels are checked. To specify a value, enter the value as a time span: hh:mm:ss, where h = hours, m = minutes, and s = seconds. The default value is 00:00:02. The valid range for this parameter is 00:00:01 to 00:00:30.

By default, the message queue database is stored at C:\Program Files\Microsoft\ExchangeServer\TransportRoles\data\Queue. This location is controlled by the QueueDatabasePath parameter in the EdgeTransport.exe.config file.

The results of this formula are expressed as a percentage of the total hard disk drive space that is being used. The formula guarantees that free hard disk drive space is always available for the message queue database, regardless of the size of the hard disk drive. The results of the formula are always rounded down to the nearest integer.

By default, the medium level of hard disk drive utilization is 2 percent less than the high level. By default, the normal level of hard disk drive utilization is 4 percent less than the high level. As the available hard disk drive space decreases, the hard disk drive utilization increases.

You can override the default calculations for the high, medium, and normal levels of hard disk drive utilization by specifying the values in the EdgeTransport.exe.config file. Table 2 shows the options for monitoring the free hard disk drive space for the message queue database that are available in the EdgeTransport.exe.config file.

0. This value indicates that the actual value is 2 percent less than the value of PercentageDatabaseDiskSpaceUsedHighThreshold.

3 to 100. The value must be less than the value of PercentageDatabaseDiskSpaceUsedHighThreshold.

PercentageDatabaseDiskSpaceUsedNormalThreshold

0. This value indicates that the actual value is 2 percent less than the value of PercentageDatabaseDiskSpaceUsedMediumThreshold.

3 to 100. The value must be less than the value of PercentageDatabaseDiskSpaceUsedMediumThreshold.

Important:

When the back pressure limits for hard disk drive space utilization are set to their default levels, the hard disk drive that stores the message queue database on an Edge Transport server or Hub Transport server must always have a fixed amount of free hard disk drive space. In Exchange 2007 RTM, the amount of required free hard disk drive space is 4 GB. In Exchange 2007 SP1, the amount of required free hard disk drive space is 500 MB. If the available free space is less than the required amount of free hard disk drive space, the hard disk drive utilization level is considered high. Therefore, all message flow stops. In that case, you must follow one of these steps:
- Relocate the message queue database to a different hard disk drive that has more free space. For more information, see How to Change the Location of the Queue Database.
- Increase the values of the PercentageDatabaseDiskSpaceUsedHighThreshold, PercentageDatabaseDiskSpaceUsedMediumThreshold, or PercentageDatabaseDiskSpaceUsedNormalThreshold parameters.

By default, the message queue database transaction logs are stored at C:\Program Files\Microsoft\ExchangeServer\TransportRoles\data\Queue. This location is controlled by the QueueDatabaseLoggingPath parameter in the EdgeTransport.exe.config file.

The EdgeTransport.exe.config file contains a QueueDatabaseCheckPointDepthMax parameter that has a default value of 20 MB. This parameter controls the total allowed size of all uncommitted transaction logs that exist on the hard disk drive. This parameter is used in the formula that calculates hard disk drive utilization.

Note:

The value of the QueueDatabaseCheckPointDepthMax parameter applies to all transport-related Extensible Storage Engine (ESE) databases that exist on the Hub Transport server or Edge Transport server. This would include the message queue database and the IP filter database. If you set the value of the QueueDatabaseCheckPointDepthMax parameter too low, this can cause significant performance issues because uncommitted transactions are forcibly committed to the database instead of being written to transaction logs. We recommend that you do not modify the default value of the QueueDatabaseCheckPointDepthMax parameter.

By default, the high level of hard disk drive utilization is calculated by using the following formula:

If you use the 20 MB default value of the DatabaseCheckPointDepthMax parameter, 500 MB is always reserved for the message queue database transaction logs regardless of the size of the hard disk drive. The results of the formula are always rounded down to the nearest integer.

By default, the medium level of hard disk drive utilization is 2 percent less than the high level. The normal level of hard disk drive utilization is 4 percent less than the high level. As the available hard disk drive space decreases, the hard disk drive utilization increases.

You can override the default calculations for the high, medium, and normal levels of hard disk drive utilization by specifying the values in the EdgeTransport.exe.config file. Table 3 shows the options for monitoring the free hard disk drive space for the message queue database transaction logs that are available in the EdgeTransport.exe.config file.

A list of changes that are made to the message queue database is kept in memory until those changes can be committed to a transaction log. Then the list is committed to the message queue database itself. These outstanding message queue database transactions that are kept in memory are known as version buckets. The number of version buckets may increase to unacceptably high levels because of virus issues, problems with the message queue database integrity, or hard disk drive performance.

You can override the default values for the high, medium, and normal numbers of version buckets by specifying the values in the EdgeTransport.exe.config file. Table 4 shows the options for monitoring the version buckets that are available in the EdgeTransport.exe.config file.

Table 4 Options for monitoring the number of version buckets

Parameter name

Default value in Exchange 2007 RTM

Default value in Exchange 2007 SP1

Valid input range

VersionBucketsHighThreshold

100

200

1 to 8000

VersionBucketsMediumThreshold

60

120

1 to 8000. The value must be less than the value of VersionBucketsHighThreshold.

VersionBucketsNormalThreshold

40

80

1 to 8000. The value must be less than the value of VersionBucketsMediumThreshold.

By default, the high level of memory utilization by the EdgeTransport.exe process is calculated by using the following formula:

75 percent of the total physical memory or 1 TB, whichever is less

This calculation doesn't include virtual memory that is available on the hard disk drive in the paging file, or the memory that is used by other processes. The results of this formula are expressed as a percentage of the total memory that is used by the EdgeTransport.exe process. The results of the formula are always rounded down to the nearest integer.

By default, the medium level of memory utilization by the EdgeTransport.exe file is calculated as 73 percent of the total physical memory or 2 percent less than the value of the high level, whichever is less. By default, the normal level of memory utilization by the EdgeTransport.exe file is calculated as 71 percent of the total physical memory or 4 percent less than the value of the high level, whichever is less. As the available memory decreases, memory utilization increases.

You can override the default calculations for high, medium, and normal memory utilization by the EdgeTransport.exe process by specifying the values in the EdgeTransport.exe.config file. Table 5 shows the options for monitoring the memory that is used by the EdgeTransport.exe process that are available in the EdgeTransport.exe.config file.

Table 5 Options for monitoring the memory that is used by the EdgeTransport.exe process

Parameter name

Default value

Valid input range

PercentagePrivateBytesUsedHighThreshold

0. This value indicates that the default calculation will be used.

3 to 100

PercentagePrivateBytesUsedMediumThreshold

0. This value indicates that the actual value is 2 percent less than value PercentagePrivateBytesUsedHighThreshold.

3 to 100. The value must be less than the value of PercentagePrivateBytesUsedHighThreshold.

PercentagePrivateBytesUsedNormalThreshold

0. This value indicates that the actual value is 2 percent less than value PercentagePrivateBytesUsedMediumThreshold.

3 to 100. The value must be less than the value of PercentagePrivateBytesUsedMediumThreshold.

By default, the high level of memory utilization by all processes is 94 percent of total physical memory. This value doesn't include virtual memory that is available on the hard disk drive in the paging file. As the available memory decreases, memory utilization increases.

When the specified memory utilization level is reached, message dehydration occurs. Message dehydration is the act of removing unnecessary elements of queued messages that are cached in memory. Complete messages are cached in memory for enhanced performance. Removal of the MIME content of queued messages from memory reduces the memory that is used at the expense of higher latency because the messages are read directly from the message queue database. By default, message dehydration is enabled. You can enable or disable message dehydration in the EdgeTransport.exe.config application configuration file.

You can override the default value for high memory utilization by all processes by specifying the high value in the EdgeTransport.exe.config file. Table 6 shows the options for monitoring the memory that is used by the all process that are available in the EdgeTransport.exe.config file.

Table 6 Options for monitoring the memory that is used by all processes that are available in the EdgeTransport.exe.config file

Back pressure doesn't stop all message delivery at the first sign of trouble. Monitoring and corresponding actions are applied in several stages when there are indications that a monitored resource is overused. If these actions don't successfully restore resource utilization to a normal level, additional corrective actions are taken. The severity of the corrective actions continues to escalate until resource utilization returns to a normal level. The frequency at which the resource utilization levels are checked is controlled by the ResourceMonitoringInterval parameter. The default value is 2 seconds.

The following sections describe the stages in which back pressure in applied.

The memory utilization of the EdgeTransport.exe process is compared to the value that is specified by the PercentagePrivateBytesUsedNormalThreshold parameter. The default value is 71 percent of the total physical memory. If the memory utilization of the EdgeTransport.exe process is higher than the specified normal level, garbage collection is forced. Garbage collection is a process that checks for unused objects that exist in memory, and reclaims the memory that is used by those unused objects.

The number of uncommitted message queue database transactions that exist in memory is compared to the value specified by the VersionBucketsNormalThreshold parameter. In Exchange 2007 RTM, the default value is 40. In Exchange 2007 SP1, the default value is 80. If the number of uncommitted message queue database transactions that exist in memory is higher than the specified normal level, an attempt is made to force the outstanding message queue database transactions that exist in memory to be written to transaction logs.

The utilization levels of all monitored resources are checked against specified values for normal levels of utilization. If any of the resource levels are higher than the specified values for normal levels, the resource with the highest level of utilization is used to take action. The following sections describe the actions taken on Hub Transport servers and Edge Transport servers at different resource utilization levels.

The memory utilization of the EdgeTransport.exe process is compared to the value specified by the PercentagePrivateBytesUsedNormalThreshold parameter. The default value is 71 percent of the total physical memory. If the memory utilization of the EdgeTransport.exe process is higher than the specified normal level, the following action occurs:

If the Microsoft Exchange Transport service is restarted, messages that exist in the submission queue are not automatically processed when the service starts.

The number of transactions in the uncommitted message queue database that exist in memory is compared to the value specified by the VersionBucketsNormalThreshold parameter. In Exchange 2007 RTM, the default value is 40. In Exchange 2007 SP1, the default value is 80. If the number of transactions in the uncommitted message queue database that exist in memory is larger than the specified normal level, the following actions occur:

The transport dumpster is disabled. The transport dumpster is a feature of the Hub Transport server role that should be enabled when using cluster continuous replication (CCR) or local continuous replication (LCR). The transport dumpster submits recently delivered mail after an unscheduled outage in a CCR environment, and as part of the activation process in an LCR environment. The transport dumpster is used only in LCR and CCR environments. For more information, see How to Configure the Transport Dumpster.

Message delivery to any remote destination that uses a remote delivery queue is disabled. These destinations include remote domains or remote Active Directory directory service sites. Message delivery to mailbox servers that use mailbox delivery queues is unaffected.

If the memory utilization of the EdgeTransport.exe process is at the medium or high level, or if the memory utilization of all other processes exceeds the maximum specified value, the following actions occur:

Description: The Microsoft Exchange Transport service is rejecting messages because available disk space is below the configured threshold. Administrative action may be required to free disk space for the service to continue operations.

Event log entry for critically low available memory

Event Type: Error

Event Source: MSExchangeTransport

Event Category: Resource Manager

Event ID: 15007

Description: The Microsoft Exchange Transport service is rejecting message submissions because the service continues to consume more memory than the configured threshold. This may require that this service be restarted to continue normal operation.