Chapter 9
MTA Address Translation and Routing

Prior to Messaging Server 6 2003Q4, Messaging Server used to access all user, domain, and group data from a database that was compiled from information stored in an LDAP server. When directory information was updated in the LDAP server, the database information was synchronized with a program called dirsync. The Messaging Server MTA now accesses the LDAP directory directly. This chapter describes the flow of data through the MTA using direct LDAP data access. It contains the following sections:

The Direct LDAP Algorithm and Implementation

Domain Locality Determination

Starting with an address of the form user@domain, the address translation and routing process first checks to see if domain is local.

Rewrite Rule Machinery

A facility has been added to the MTA rewrite rule machinery to check a given string to see if it is a domain we need to handle locally. This new facility is activated by a $V or $Z metacharacter. These new metacharacters are syntactically similar to the existing $N, $M, $Q, and $C metacharacters, that is, they are followed by a pattern string. In the case of $N, $M, $Q, and $C the pattern is matched against either the source or destination channel. In the case of $V and $Z the pattern is a domain and the check is to see whether or not it is local. $V causes a rule failure for a nonlocal domain and $Z causes a rule failure for a local domain.

The handling of these metacharacters is implemented as the following procedure:

Messaging Server verifies that the current domain is local.

If the base DN determination succeeds, the attribute specified by the LDAP_DOMAIN_ATTR_ROUTING_HOSTS MTA option (default mailRoutingHosts) is retrieved. If this attribute is present, it lists the set of hosts able to handle users in this domain. This list is compared against the host specified by the local.hostname configutil parameter and the list of hosts specified by the local.imta.hostnamealiases configutil parameter. These options can be overridden by the LDAP_LOCAL_HOST and LDAP_HOST_ALIAS_LIST MTA options, respectively. If there is a match or the attribute is not present on the domain, the domain is local. If no match occurs, the domain is nonlocal.

The handling of domains considered to be nonlocal because of the mailRoutingHosts attribute depends on the setting of the ROUTE_TO_ROUTING_HOST MTA option. If the option is set to 0 (the default), the address is simply treated as nonlocal and MTA rewrite rules are used to determine routing. If the option is set to 1, a source route consisting of the first value listed in the LDAP_DOMAIN_ATTR_ROUTING_HOSTS MTA option is prepended to the address.

If the base DN determination fails, remove a component from the left hand side of the domain and go to Step 1. If no components remain continue to Step 4.

A consequence of this backtracking up the domain tree is that if domain.com is recognized as local, any subdomain of domain.com will be recognized as local. Situations may arise where this is undesirable, so an MTA option, DOMAIN_UPLEVEL, is provided to control this behavior. Specifically, bit 0 (value = 1) of DOMAIN_UPLEVEL, if clear, disables retries with domain components removed. The default value of DOMAIN_UPLEVEL is 0.

Vanity domain checking now needs to be performed. This is done by instituting an LDAP search using the LDAP URL specified by the DOMAIN_MATCH_URL MTA option. The value of this option should be set to:

ldap:///$B?msgVanityDomain?sub?(msgVanityDomain=$D)

$B substitutes the value of the local.ugldapbasedn configutil parameter; this is the base of the user tree in the directory. The LDAP_USER_ROOT MTA option can be used to override the value of this configutil option specifically for the MTA. (A new $C metacharacter has also been added to substitute in the base of the domain tree.

The actual return value from this search does not matter. What matters is if there is a value to return. If there is a return value the domain is considered to be local, if not it is considered to be nonlocal.

Domain Map Determination of Domain Locality

It is also instructive to note what steps are performed to determine domain locality. The steps are schema-level-specific. In the case of Sun LDAP Schema 1, they are:

Convert the domain to a base DN in the domain tree. This is done by converting the domain into a series of dc components and then adding a domain root suffix. The default suffix is obtained from the service.dcroot configutil parameter. The default suffix is o=internet. So a domain of the form a.b.c.d would typically be converted into dc=a,dc=b,dc=c,dc=d,o=internet. The service.dcroot configutil parameter can be overridden by setting the LDAP_DOMAIN_ROOT MTA option.

Look for an entry with the base DN found in Step 1 and an object class of either inetDomain or inetDomainAlias. The search filter used for this purpose can be overridden by setting the LDAP_DOMAIN_FILTER_SCHEMA1 MTA option which defaults to (|(objectclass=inetDomain)(objectclass=inetdomainalias)).

Exit with a failure if nothing is found.

If the object class of the entry found is inetDomain return the value of the inetDomainBaseDn attribute for the domain entry. Messaging Server checks for the presence of the inetDomainBaseDn attribute. If it is present it is used If it is not present, the entry is assumed to be a domain alias. The MTA option LDAP_DOMAIN_ATTR_BASEDN can be used to override the use of inetDomainBaseDN.

If the object class of the entry found is inetDomainAlias, look up the entry referenced by the aliasedObjectName attribute. This new entry must have an object class of the inetDomainBaseDN attribute. An alternative to the use of aliasedObjectName attribute can be specified with the MTA option LDAP_DOMAIN_ATTR_ALIAS.

If the alias entry lookup is successful return the value of the inetDomainBaseDn attribute for the new entry.

In Sun LDAP Schema 2, the action taken is much simpler: The directory is searched for an entry with the object class sunManagedOrganization where the domain appears as a value of either the sunPreferredDomain or associatedDomain attribute. If need be the use of the sunPreferredDomain and associatedDomain attributes for this purpose can be overridden with the respective MTA options LDAP_ATTR_DOMAIN1_SCHEMA2 and LDAP_ATTR_DOMAIN2_SCHEMA2. The search is done under the root specified by the service.dcroot configutil parameter. The service.dcroot configutil parameter can be overridden by setting the LDAP_DOMAIN_ROOT MTA option.

Caching Of Domain Locality Information

Due to the frequency with which domain rewrite operations are performed and the expense of the directory queries (especially the vanity domain check) both negative and positive indications about domains need to be cached. This is implemented with an in-memory open-chained dynamically-expanded hash table. The maximum size of the cache is set by the DOMAIN_MATCH_CACHE_SIZE MTA option (default 100000) and the timeout for entries in the cache is set by the DOMAIN_MATCH_CACHE_TIMEOUT MTA option (default 600 seconds).

Error Handling

Temporary server failures during this process have to be handled carefully, since when they occur, it is impossible to know whether or not a given domain is local. Basically two outcomes are possible in such a case:

Return a temporary (4xx) error to the client telling it to try the address again later.

Accept the address but queue it to the reprocessing channel so it can be retried locally later.

Neither of these options is appropriate in all cases. For example, outcome 1 is appropriate when talking to a remote SMTP relay. But outcome 2 is appropriate when dealing with an SMTP submission from a local user.

While it would be possible in theory to handle temporary failures by using multiple rules with the same pattern, the overhead of repeating such queries, even with a cache in place, is unacceptable. For these reasons, the simple success/fall-through-to-the-next-rule matching model of domain rewriting is inadequate. Instead, a special template, specified by the MTA option DOMAIN_FAILURE, is used in the event of a domain lookup failure. When a $V operation fails, this template replaces the remainder of the current rewrite rule template being processed.

In addition to $V and $Z, several other new metacharacters have been added to the rewrite rule facility:

$? accepts a numeric argument which, if present, specifies the enhanced status code to be returned by SMTP. For example, “$5001001?message” specifies an enhanced status code value of 5.1.1.

$1M checks to see whether the internal reprocessing flag was set by the source channel.

$1N checks to see whether the internal reprocessing flag was cleared by the source channel.

$1~ performs any pending channel match checks and if the checks have failed terminates processing of the current rewrite rule template successfully.

Pattern for Domain Check Rewrite Rule

This domain check needs to be performed before other rewrite rules have a chance to operate. This ordering is insured by using the special $* on the left hand side in the rule. The $* pattern is checked prior to any other rules.

Putting It All Together

Taking all the machinery described so far into account, the new rewrite rule we need in the imta.cnf is:

$* $E$F$U%$H$V$H@localhost

and the value of the DOMAIN_FAILURE MTA option in the option.dat file needs to be:

In this rewrite rule, localhost is the host name associated with the local channel. The value of the DOMAIN_FAILURE option shown here is the default value so it does not need to appear in option.dat under normal circumstances.

The ordering here is especially tricky. The MTA checks $V after the address is rebuilt but before the route is added. This lets the MTA to change the route in the event of a temporary lookup failure. Pending channel match checks are applied any time the insertion point changes, so the @ after the second $H invokes the check. If the check succeeds, the remainder of the template applies and rewrite processing concludes. If the check fails, the rewrite fails and rewriting continues with the next applicable rewrite rule. If the check cannot be performed due to a temporary failure, template processing continues with the value specified by the DOMAIN_FAILURE MTA option. The value of this template first sets the routing host to reprocess-daemon. Then the template checks to see whether or not the MTA is dealing with a reprocessing channel of some sort or tcp_local. If the MTA is dealing with such a channel, the rule continues, making the routing host illegal and specifying a temporary failure as the outcome. If the MTA is not dealing with such a channel, the rule is truncated and successfully terminated, thereby rewriting the address to the reprocess channel.

Alias expansion of local addresses

Once an address has been determined to be associated with the local channel it automatically undergoes alias expansion. The alias expansion process examines a number of sources of information, including:

The alias file (part of the compiled configuration).

The alias database.

Alias URLs.

The exact alias sources that are checked and the order in which they are checked depends on the setting of the ALIAS_MAGIC MTA option in the option.dat file. For direct LDAP, set the option to 8764. This means that the URL specified by the ALIAS_URL0 MTA option is checked first, then the URL specified by the ALIAS_URL1 MTA option, then the URL specific by the ALIAS_URL2 MTA option, and finally, the alias file. The alias database is not checked when this setting is active.

Alias Checking with LDAP URLs

Checking of aliases in LDAP is implemented by specifying two special LDAP URLs as alias URLs. The first of these handles regular users and groups; vanity domains are handled by subsequent alias URLs. The first URL is specified as ALIAS_URL0:

ALIAS_URL0=ldap:///$V?*?sub?$R

The $V Metacharacter

Metacharacter expansion occurs prior to URL lookup. The two metacharacters used in the ALIAS_URL0 value are $V and $R.

The $V metacharacter converts the domain part of the address into a base DN. This is similar to the initial steps performed by the $V rewrite rule metacharacter described previously in the section entitled “Rewrite Rule Machinery.” $V processing consists of the following steps:

Get the base DN for the current domain.

Get the canonical domain associated with the current domain. In Sun LDAP Schema 1, the canonical domain name is given by the inetCanonicalDomainName attribute of the domain entry if the attribute is present. If the attribute is absent the canonical name is the name constructed in the obvious way from the DN of the actual domain entry. This will differ from the current domain when the current domain is an alias. The name attribute used to store the canonical name may be overridden with the LDAP_DOMAIN_ATTR_CANONICAL MTA option in the option.dat file.

In Sun LDAP Schema 2, the canonical name is simply the value of the SunPreferredDomain attribute.

If the base DN exists, substitute it into the URL in place of the $V.

Any applicable hosted domain for this entry is now determined. This is done by comparing either the canonical domain (if bit 2 (value = 4) of DOMAIN_UPLEVEL is clear) or the current domain (if bit 2 (value = 4) of DOMAIN_UPLEVEL is set) with the service.defaultdomain configutil parameter. If they do not match, the entry is a member of a hosted domain. The service.defaultdomain configutil parameter can be overridden by setting the LDAP_DEFAULT_DOMAIN MTA option in the option.dat file.

If the base DN determination fails, remove a component from the left hand side of the domain and go to Step 1. The substitution fails if no components remain.

$V also accepts an optional numeric argument. If it is set to 1 (for example, $1V), a failure to resolve the domain in the domain tree will be ignored and the base of the user tree will be returned.

If the attempt to retrieve the domain's base DN succeeds, the MTA also retrieves several useful domain attributes which will be needed later. The names of the attributes retrieved are set by the following MTA options in the option.dat file:

LDAP_DOMAIN_ATTR_UID_SEPARATOR (default domainUidSeparator)

LDAP_DOMAIN_ATTR_SMARTHOST (default mailRoutingSmartHost)

LDAP_DOMAIN_ATTR_CATCHALL_ADDRESS (default mailDomainCatchallAddress)

LDAP_DOMAIN_ATTR_BLOCKLIMIT (default mailDomainMsgMaxBlocks)

LDAP_DOMAIN_ATTR_REPORT_ADDRESS (default mailDomainReportAddress)

LDAP_DOMAIN_ATTR_STATUS (default inetDomainStatus)

LDAP_DOMAIN_ATTR_MAIL_STATUS (default mailDomainStatus)

LDAP_DOMAIN_ATTR_CONVERSION_TAG (default mailDomainConversionTag)

LDAP_DOMAIN_ATTR_FILTER (default mailDomainSieveRuleSource)

LDAP_DOMAIN_ATTR_DISK_QUOTA (no default)

LDAP_DOMAIN_ATTR_MESSAGE_QUOTA (no default)

LDAP_DOMAIN_ATTR_AUTOREPLY_TIMEOUT (no default)

LDAP_DOMAIN_ATTR_OPTIN (no default)

LDAP_DOMAIN_ATTR_PRESENCE (no default)

LDAP_DOMAIN_ATTR_RECIPIENTLIMIT (no default)

LDAP_DOMAIN_ATTR_RECIPIENTCUTOFF (no default)

LDAP_DOMAIN_ATTR_SOURCEBLOCKLIMIT (no default)

Calling a Mapping from a URL

There might be unusual cases where the mapping from domain to base DN is done some other way. In order to accommodate such setups, the URL resolution process has the ability to call an MTA mapping. This is done with a metacharacter sequence of the general form:

$|/mapping-name/mapping-argument|

The double quotation (“) initiates and terminates the callout. The character immediately following the $ is the separator between the mapping name and argument; a character should be chosen that does not collide with the expected character values used in either the mapping name or argument.

The $R Metacharacter

The $R metacharacter provides an appropriate filter for the URL. The intent is to produce a filter that searches all the attributes that might contain an email address for a particular user or group. The list of attributes to search comes from the configutil parameter local.imta.mailaliases. If this parameter is not set, the local.imta.schematag configutil parameter is examined, and depending on its value, an appropriate set of default attributes is chosen as follows:

sims401 mail, rfc822mailalias

nms41 mail,mailAlternateAddress

ims50 mail,mailAlternateAddress,mailEquivalentAddress

The value of local.imta.schematag can be a comma-separated list. If more than one schema is supported, the combined list of attributes with duplicates eliminated is used. The LDAP_SCHEMATAG MTA option can be used to override the setting of local.imta.schematag specifically for the MTA.

Additionally, the filter searches not only for the address that was originally supplied, but also for an address with the same local part but the domain that was actually found in the domain tree, which was saved in Step 2 in the section entitled “The $V Metacharacter.” The iterative nature of the domain tree lookup means the two addresses may be different. This additional check is controlled by bit 1 (value = 2) of the DOMAIN_UPLEVEL MTA option in the option.dat file. Setting the bit enables the additional address check. The default value of DOMAIN_UPLEVEL is 0.

For example, suppose that the domain siroe.com appears in the domain tree. Assuming Sun LDAP Schema 1 is in force, a lookup of the address

u@host1.siroe.com

The filter that results from the expansion of $R and an ims50 schematag looks like:

The ALLOW_UNQUOTED_ADDRS_VIOLATE_RFC2798 MTA Option

Some MTAs have been exceptionally lax about address quoting and canonicalization. In particular, they would allow illegal addresses such as a..b@siroe.com. Even worse, they failed to add the necessary quotes to make the address into the legal “a..b”@siroe.com prior to searching for it in the directory.

The ALLOW_UNQUOTED_ADDRS_VIOLATE_RFC2798 MTA option in the option.dat file accommodates this particular violation of the standards. If set to 1 it will add additional filter terms to search on the syntactically invalid dequoted form of quoted addresses. For example, a search for the address "a..b"@siroe.com might produce a filter of the form:

This option does not solve the problems caused by the use of illegal addresses. The relevant email and directory standards are specific about what is allowed in the local part of an address. When various messaging components are presented with syntactically illegal addresses, they may behave in different ways. They may quote such addresses to make them legal, they may pass them through unchanged, they may reject them, or they may do something entirely unexpected. Therefore, if end users are given such an illegal address they are likely to find it does not work when it hits other messaging systems provided by other vendors.

Determining the Attributes to Fetch

If the URL specifies an * for the list of attributes to return, we replace the asterisk with the list of attributes the MTA is able to use.

Handling LDAP Errors

At this point the resulting URL is used to perform an LDAP search. If an LDAP error of some kind occurs, processing terminates with a temporary failure indication (4xx error in SMTP). If the LDAP operation succeeds but fails to produce a result, the catchall address attribute for the domain retrieved from the LDAP_DOMAIN_ATTR_CATCHALL_ADDRESS MTA option is checked. If it is set, its value replaces the current address.

If no catchall address attribute is set the smarthost attribute for the domain retrieved from the LDAP_DOMAIN_ATTR_SMARTHOST MTA option is checked. If it is set, an address of the form

@smarthost:user@domain

is created and alias processing terminates successfully with this result. Additionally, the conversion tag for the domain obtained from the LDAP_DOMAIN_ATTR_CONVERSION_TAG MTA option will, if present, be attached to the address so that conversions can be done prior to forwarding to the smarthost. If no catchall address or smarthost exists for the domain, processing of this alias URL terminates unsuccessfully.

Sanity Checks on the LDAP Result

After the LDAP search has returned a result, it is checked to verify that there is only one entry in it. If there are more than one, each entry is checked to see if it has the right object class for a user or a group, a non-deleted status, and for users, a UID. Entries that do not pass this check are ignored. If the list of multiple entries is reduced to one by this check, processing proceeds. If not, a duplicate or ambiguous directory error is returned.

Support for Vanity Domains

The ALIAS_URL0 check is for a conventional user or a user in a hosted domain. If this fails a vanity domain check is also made. This is done with the following alias URL:

ALIAS_URL1=ldap:///$B?*?sub?(&(msgVanityDomain=$D)$R)

Support for Catchall Addresses

Finally, a check for a catchall address of the form @host needs to be made in the mailAlternateAddress attribute. This form of wildcarding is allowed in both hosted and vanity domains, so the proper alias URL for it is:

ALIAS_URL2=ldap:///$1V?*?sub?(mailAlternateAddress=@$D)

Processing the LDAP Result

LDAP alias result processing is done in a number of order-dependent stages. These stages are described in the following sections.

Object Class Check

If the alias search succeeds, the object class of the entry is checked to make sure it contains an appropriate set of object classes for a user or a group. The possible sets of required object classes for users and groups is normally determined by what schemata are active. This is determined by the local.imta.schematag setting.

Table 9-1 shows the user and group object classes that result from various schematag values.

Table 9-1 Object Classes Resulting from Various schematag Values

schematag

User Object Classes

Group Object Classes

sims40

inetMailRouting+inetmailuser

inetMailRouting+inetmailgroup

nms41

mailRecipient + nsMessagingServerUser

mailGroup

ims50

inetLocalMailRecipient+inetmailuser

inetLocalMailRecipient+inetmailgroup

The information in this table, like the rest of schema tag handling, is hard coded. However, there are also two MTA options in the option.dat file, LDAP_USER_OBJECT_CLASSES and LDAP_GROUP_OBJECT_CLASSES, which can be set to specify different sets of object classes for users and groups respectively.

For example, a schema tag setting of ims50,nms41 would be equivalent to the following option settings:

The LDAP result is simply ignored if it does not have a correct set of object classes appropriate for a user or a group. The MTA also determines if it is dealing with a user or a group and saves this information. This saved information will be used repeatedly later.

Note that the object class settings described here are also used to construct an actual LDAP search filter that can be used to check to see that an entry has the right object classes for a user or a group. This filter is accessible through the $K metacharacter. It is also stored internally in the MTA's configuration for use by channel programs and is written to the MTA option file, option.dat, as the LDAP_UG_FILTER option when the command imsimta cnbuild -option is used. This option is only written to the file. The MTA never reads it from the option file.

Entry Status Checks

Next the entry's status is checked. There are two status attributes, one for the entry in general and another specifically for mail service.

Table 9-2 shows the general and mail-specific user or group attributes in the schematag entry to check against depending on what schemata are in effect

Table 9-2 Attributes to Check Against

schematag

Type

General

Mail-specific

sims40

users

inetsubscriberstatus

mailuserstatus

sims40

groups

none

inetmailgroupstatus

nms41

users

none

mailuserstatus

nms41

groups

none

none

Messaging Server 5.0

users

inetuserstatus

mailuserstatus

Messaging Server 5.0

groups

none

inetmailgroupstatus

If necessary the LDAP_USER_STATUS and LDAP_GROUP_STATUS MTA options in the option.dat file can be used to select alternate general status attributes for users and groups respectively. The mail-specific user and group status attributes are controlled by the LDAP_USER_MAIL_STATUS and LDAP_GROUP_MAIL_STATUS MTA options.

Another factor that plays into this are the statuses for the domain itself (LDAP_DOMAIN_ATTR_STATUS and LDAP_DOMAIN_ATTR_MAIL_STATUS). All in all there are four status attributes. They are combined by considering them in the following order:

Domain status

Domain mail status

User or group status

Mail user or group status

The first of these that specifies something other than “active” takes precedence over all the others. The other permissible status values are “inactive,” “deleted,” “removed,” “disabled,” “hold,” and “overquota.” “Hold,” “disabled,” and “removed” statuses may only be specified for mail domains, mail users, or mail groups. “Overquota” status can only be specified as a mail domain or mail user status.

All statuses default to “active” if a particular status attribute is not present. Unknown status values are interpreted as “inactive.”

When the four statuses are combined, the following statuses for a user or group are possible: “active,” “inactive,” “deleted,” “removed,” “disabled,” “hold,” and “overquota.” Active status causes alias processing to continue. Inactive or overquota status results in immediate rejection of the address with a 4xx (temporary) error. Deleted, removed, and disabled statuses results in immediate rejection of the address with a 5xx (permanent) error. Hold status is treated as active as far as status handling is concerned but it sets an internal flag so that when delivery options are considered later on, any options that are there are overridden with an option list containing a single “hold” entry.

UID Check

The next step is to consider the entry's UID. UIDs are used for a variety of purposes and must be part of all user entries and may be included in group entries. A user entry without a UID is ignored and processing of this alias URL terminates unsuccessfully. UIDs for entries in a hosted domain can consist of the real UID, a separator character, and then a domain. The MTA only wants the real UID so the rest is removed if present using the domain separator character obtained with the LDAP_DOMAIN_ATTR_UID_SEPARATOR MTA option in the option.dat file.

In the unlikely event that an attribute other than uid is used to store UIDs, the LDAP_UID MTA option can be used to force use of a different attribute.

Message Capture

Next the LDAP attribute used to specify one or more message capture addresses is checked. The attribute used for this purpose must be specified with the LDAP_CAPTURE MTA option. There is no default. Values of this attribute are treated as addresses and a special “capture” notification is generated and sent to these address containing the current message as an attachment. Additionally, the capture addresses are used to seed the address reversal cache in the likely event the address will subsequently appear as an envelope from: address.

Seeding the Reversal Cache

Next the primary address and any aliases attached to the user entry are considered. This information is used to seed the address reversal cache. It plays no part in the current address translation process. First, the primary address, personal name, recipient limit, recipient cutoff, and source block limit attributes are considered. The primary address is normally stored in the “mail” attribute; another attribute can be specified by setting the LDAP_PRIMARY_ADDRESS MTA option appropriately. (The primary address reverses to itself, of course.) There is no default attribute for any of the other attributes. If you want to use them, you must specify them with the LDAP_PERSONAL_NAME, LDAP_RECIPIENTLIMIT, LDAP_RECIPIENTCUTOFF, and LDAP_SOURCEBLOCKLIMIT MTA options. The corresponding domain-level recipient limit, recipient cutoff, and source block limit attributes are also considered at this point. User-level settings completely override any domain-level setting.

Next, any secondary addresses are considered and a cache entry is made for each one. There are two sorts of secondary addresses: Those that undergo address reversal and those that do not. Both must be considered in order to properly seed the address reversal cache because of the need to check for message capture requests in all cases.

Secondary addresses that undergo reversal are normally stored in the mailAlternateAddress attribute. Another attribute can be specified by setting the LDAP_ALIAS_ADDRESSES MTA option. Secondary addresses that do not undergo reversal are normally stored in the mailEquivalentAddress attribute. Another attribute can be specified with the LDAP_EQUIVALENCE_ADDRESSES MTA option.

Mail Host and Routing Address

It is now time to consider the mailhost and mailRoutingAddress attributes. The actual attributes considered can be overridden with the LDAP_MAILHOST and LDAP_ROUTING_ADDRESS MTA options, respectively. These attributes work together to determine whether or not the address should be acted on at this time or forwarded to another system.

The first step is to decide whether or not mailhost is meaningful for this entry. A preliminary check of the delivery options active for the entry is done to see whether or not the entry is mailhost-specific. If it is not, mailhost checking is omitted. See the description of Delivery Options Processing, and the # flag in particular, to understand how this check is done.

In the case of a user entry, the mailhost attribute must identify the local system in order to be acted on. The mailhost attribute is compared with the value of the local.hostname configutil parameter and against the list of values specified by the local.imta.hostnamealiases configutil parameter. The mailhost attribute is deemed to identify the local host if any of these match.

A successful match means that the alias can be acted on locally and alias processing continues. An unsuccessful match means that the message needs to be forward to the mailhost to be acted on. A new address of the form

@mailhost:user@domain

is constructed and becomes the result of the alias expansion operation.

The handling of a missing mailhost attribute is different depending on whether the entry is a user or a group. In the case of a user, a mailhost is essential, so if no mailhost attribute is present a new address of the form

@smarthost:user@domain

is constructed using the smart host for the domain determined by the LDAP_DOMAIN_ATTR_SMARTHOST MTA option. An error is reported if no smart host exists for the domain.

Groups, on the other hand, do not require a mailhost, so a missing mailhost is interpreted as meaning that the group can be expanded anywhere. So alias processing continues.

The mailRoutingAddress attribute adds one final wrinkle. If it is present, alias processing terminates with the mailRoutingAddress as the result. However, if a mailhost is present, it is added to the mailRoutingAddress as a source route.

Miscellaneous Attribute Support

Next the mailMsgMaxBlocks attribute is considered. First it is minimized with the domain block limit returned from the LDAP_DOMAIN_ATTR_BLOCKLIMIT MTA option. If the size of the current message is known to exceed the limit, alias processing terminates with a size-exceeded error. If the size is not known or does not exceed the limit, the limit is nevertheless stored and will be rechecked when the message itself is checked later. The use of mailMsgMaxBlocks can be overridden with the LDAP_BLOCKLIMIT MTA option.

Next a number of attributes are accessed and saved. Eventually these will be written into the queue file entry for use by the ims_master channel program, which will then use them to update the store's user information cache. If the attributes are not found for individual users, domain-level attributes can be used to set defaults.

This step is skipped if the LDAP entry is for a group rather than a user or if the LDAP entry came from the alias cache and not from the LDAP directory. The logic behind the latter criteria is that frequent updates of this information are unnecessary and using the alias cache offers a reasonable criteria for when updates should be done. The names of the attributes retrieved are set by various MTA options.

Table 9-3 shows the MTA options which set the retrieved disk quota and message quota attributes.

Spare slots for additional attributes are included so that you can use them to build customized address expansion facilities.

Next any values associated with the mailConversionTag attribute are added to the current set of conversion tags. The name of this attribute can be changed with the LDAP_CONVERSION_TAG MTA option. If any values were associated with the domain's mailDomainConversionTag attribute, they are attached as well.

Delivery Options Processing

Next the mailDeliveryOption attribute is checked. The name of this attribute can be changed with the LDAP_DELIVERY_OPTION MTA option. This is a multi valued option and its values determine the addresses produced by the alias translation process. Additionally, the permissible values are different for users and groups. Common permissible values are program, forward, and hold.” User-only values are mailbox, native, unix, and autoreply. The group-only values are members, members_offline, and file.

The conversion of the mailDeliveryOption attribute into appropriate addresses is controlled by the DELIVERY_OPTIONS MTA option. This option not only specifies what addresses are produced by each permissible mailDeliveryOption value, but also what the permissible mailDeliveryOption values are and whether or not each one is applicable to users, groups, or both.

The value of this option consists of a comma-separated list of deliveryoption=template pairs, each pair with one or more optional single character prefixes.

Each delivery option corresponds to a possible mailDeliveryOption attribute value and the corresponding template specifies the resulting address using the same metacharacter substitution scheme used by URL processing.

Table 9-5 shows the single character prefixes available for the DELIVERY_OPTIONS options.

Sets a flag saying that the vacation start and end times should be checked to see if this delivery option really is in effect.

#

Sets a flag saying expansion of this delivery option does not need to take place on the entry’s designated mailhost.

/

Sets a flag that causes all addresses produced by this delivery option to be held. Message files containing these recipient addresses will have a .HELD extension.

!

Sets a flag that says that autoreply operations should be handled internally by the MTA. It only makes sense to use this prefix on an autoreply delivery option. The option’s value should direct the message to the bitbucket channel

If neither * nor & are present, the delivery option is taken to apply to both users and groups.

Additional Metacharacters for Use in Delivery Options

Several additional metacharacters have been added to support this new use of the MTA's URL template facility. These include:

Table 9-6 shows additional metacharacters and their descriptions for use in delivery options.

Table 9-6 Additional Metacharacters for Use in Delivery Options

Metacharacter

Description

$\\

Force subsequent text to lower case.

$^

Force subsequent text to upper case.

$_

Perform no case conversion on subsequent text.

$nA

Insert the nth character of the address. The first character is character 0. The entire address is substituted if n is omitted. This is intended to be used to construct autoreply directory paths.

$D

Insert the domain part of the address.

$nE

Insert the value of the nth spare attribute. If n is omitted the first attribute is used.

$F

Insert the name of the delivery file (mailDeliveryFileURL attribute).

$nG

Insert the value of the nth spare attribute. If n is omitted the second attribute is used.

$nH

Insert the nth component of the domain from the original address counting from 0. The default is 0 if n is omitted.

$nI

Insert hosted domain associated with alias. This metacharacter accepts an integer parameter n whose semantics are described in Table 9-7.

$nJ

Insert the nth part of the host domain counting from 0. The default for n is 0.

$K

Insert an LDAP filter that matches the object classes for a user or group. See the description of the LDAP_UG_FILTER output-only MTA option.

$L

Insert the local part of the address.

$M

Insert the UID associated with the current alias.

$P

Insert the program name (mailProgramDeliveryInfo attribute).

$nS

Insert subaddress associated with the current address. This metacharacter accepts an integer parameter n whose semantics are described in Table 9-7.

$nU

Insert the nth character of the dequoted form of the mailbox part of the current address. The first character is character 0. The entire dequoted mailbox is substituted if n is omitted.

$nX

Insert the nth component of the mailhost. The entire mailhost is inserted if n is omitted.

Table 9-7 shows how the integer parameter modifies the behavior of the $nI and $nS metacharacters.

Insert value if one is available. Insert nothing and delete preceding character is one is not (this particular behavior is needed by the ims-ms channel).

3

Insert value if one is available. Insert nothing and ignore following character if one is not.

In addition to the metacharacters, Table 9-8 shows two special template strings.

Table 9-8 Special Template Strings

Special Template String

Description

*

Perform group expansion. This value is not valid for user entries.

**

Expand the attribute named by the LDAP_FORWARDING_ADDRESS MTA option. This defaults to mailForwardingAddress.

With group expansion, for example, if a user's mailDeliveryOption value is set to mailbox, we form a new address consisting of the stripped UID, a percent sign followed by the hosted domain if one is applicable, a plus sign followed by the subaddress if one was specified, and finally @ims-ms-daemon.

Delivery Option Defaults

If the list of active delivery options is empty at this point, the first option on the list (usually mailbox) is activated for users and the second option on the list (usually members) is activated for groups.

Start and End Date Checks

Start and end dates are checked after the delivery option list has been read. There are two attributes whose names are controlled by the LDAP_START_DATE (default vacationStartDate) and LDAP_END_DATE (default vacationEndDate) MTA options, respectively. If one or more of the active delivery options specified the ^ prefix character, the values of these options are checked against the current date. If the current date is outside the range specified by these options, the delivery options with the ^ prefix are removed from the active set.

Optin and Presence Attributes

The LDAP_OPTIN MTA option can be used to specify an LDAP attribute containing a list of spam filter opt-in values. If the option is specified and the attribute is present, it is appended to the current spam filter opt-in list. Any values set by the domain level attribute set by the LDAP_DOMAIN_ATTR_OPTIN MTA option will also be appended to the list.

The LDAP_PRESENCE MTA option can be used to specify a URL that can be resolved to return presence information about the user. If the option is specified and the attribute is present, its value is saved for possible use in conjunction with Sieve presence tests. The domain level attribute set by the LDAP_DOMAIN_ATTR_PRESENCE MTA option is used as source for this URL if no value exists for the user entry.

Sieve Filter Handling

Next the mailSieveRuleSource attribute is checked for a Sieve filter that applies to this entry. If this attribute exists, it is parsed and stored at this point. The two possible forms for the value of this attribute are a single value that contains a complete Sieve script or multiple values where each value contains a piece of a Sieve script. The latter form is produced by the web filter construction interface. Special code is used to order the values and glue them together properly.

The use of the mailSieveRuleSource attribute specifically can be overridden by using the LDAP_FILTER MTA option.

Deferred Processing Control

Next the mailDeferProcessing attribute is checked. This attribute can be changed by using the LDAP_REPROCESS MTA option. Processing continues normally if this attribute exists and is set to no. But if this attribute is set to yes and the current source channel is not the reprocess channel, expansion of this entry is aborted and the original user@domain address is simply queued to the reprocess channel. If this attribute does not exist, the setting of the deferred processing character prefix associated with delivery options processing is checked. (See the section Delivery Options Processing for examples.) Processing is deferred if it is set. If it is not set, the default for users is no. The default for groups is controlled by the MTA option DEFER_GROUP_PROCESSING, which defaults to 1 (yes). Alias processing concludes at this point for user entries.

Group Expansion Attributes

A number of additional attributes are associated with group expansion and must be dealt with at this point. The names of these attributes are all configurable via various MTA options.

Table 9-9 lists the default attribute names, the MTA option to set the attribute name, and the way the attribute is processed by the MTA. The ordering of the elements in the table shows the order in which the various group attributes are processed. This ordering is essential for correct operation.

Table 9-9 Group Expansion Attributes

Default Attribute

MTA option to Set Attribute Name

How the Attribute is Processed

mgrpMsgRejectAction

LDAP_REJECT_ACTION

Single valued attribute that controls what happens if any of the subsequent access checks fail. Only one value is defined: TOMODERATOR, which if set instructs the MTA to redirect any access failures to the moderator specified by the mgrpModerator attribute. The default (and any other value of this attribute) causes an error to be reported and the message rejected.

mailRejectText

LDAP_REJECT_TEXT

The first line of text stored in the first value of this attribute is saved. This text will be returned if any of the following authentication attributes cause the message to be rejected. This means the text can appear in SMTP responses so value has to be limited to US-ASCII to comply with current messaging standards.

mgrpBroadcasterPolicy

LDAP_AUTH_POLICY

Specifies level of authentication needed to send to the group. Possible tokens are SMTP_AUTH_REQUIRED or AUTH_REQ, both of which mean that the SMTP AUTH command must be used to identify the sender in order to send to the group; PASSWORD_REQUIRED, PASSWD_REQUIRED, or PASSWD_REQ, all of which mean the password to the list specified by the mgrpAuthPassword attribute must appear in an Approved: header field in the message; OR, which changes the OR_CLAUSES MTA option setting to 1 for this list; AND, which changes the OR_CLAUSES MTA option setting to 0 for this list; and NO_REQUIREMENTS, which is a no-op. Multiple values are allowed and each value can consist of a comma-separated list of tokens.

If SMTP AUTH is called for it also implies that any subsequent authorization checks will be done against the email address provided by the SASL layer rather than the MAIL FROM address.

mgrpAllowedDomain

LDAP_AUTH_DOMAIN

Domains allowed to submit messages to this list. Can be multi valued.

mgrpDisallowedDomain

LDAP_CANT_DOMAIN

Domains not allowed to submit messages to this list. Can be multi valued.

mgrpAllowedBroadcaster

LDAP_AUTH_URL

URL identifying mail addresses allowed to send mail to this group. Can be multi valued. Each URL is expanded into a list of addresses and each address is checked against the current envelope from address. A match means the message is allowed.

mgrpDisallowedBroadcaster

LDAP_CANT_URL

URL identifying mail addresses not allowed to send mail to this group. Can be multi valued. Each URL is expanded into a list of addresses and each address is checked against the current envelope from address. A match means the message is not allowed.

mgrpMsgMaxSize

LDAP_ATTR_MAXIMUM_MESSAGE_SIZE

Maximum message size in bytes that can be sent to the group. This attribute is obsolete but still supported for backwards compatibility; the new mailMsgMaxBlocks attribute should be used instead.

mgrpAuthPassword

LDAP_AUTH_PASSWORD

Specifies a password needed to post to the list. The presence of this attribute forces a reprocessing pass. As the message is enqueued to the reprocessing channel, the password is taken from the header and placed in the envelope. Then while reprocessing, the password is taken from the envelope and checked against this attribute. Additionally, only passwords that actually are used are removed from the header field.

mgrpModerator

LDAP_MODERATOR_URL

The list of URLs given by this attribute to be expanded into a series of addresses. The interpretation of this address list depends on the setting of the LDAP_REJECT_ACTION MTA option. If LDAP_REJECT_ACTION is set to TOMODERATOR, this attribute specifies the moderator address(es) the message is to be sent to should any of the access checks fail. If LDAP_REJECT_ACTION is missing or has any other value, the address list is compared with the envelope from address. Processing continues if there is a match. If there is no match, the message is again sent to all of the addresses specified by this attribute. Expansion of this attribute is implemented by making the value of this attribute the list of URLs for the group. Any list of RFC822 addresses or DNs associated with the group is cleared, and the delivery options for the group are set to members. Finally, subsequent group attributes listed in this table are ignored.

mgrpDeliverTo

LDAP_GROUP_URL1

List of URLs which, when expanded, provides a list of mailing list member addresses.

memberURL

LDAP_GROUP_URL2

Another list of URLs which, when expanded, provides another list of mailing list member addresses.

uniqueMember

LDAP_GROUP_DN

List of DNs of group members. DNs may specify an entire subtree. Unique member DNs are expanded by embedding them in an LDAP URL. The exact URL to use is specified by the GROUP_DN_TEMPLATE MTA option. The default value for this option is: ldap:///$A?mail?sub?(mail=*)

$A specified the point where the uniqueMember DN is inserted.

mgrpRFC822MailMember

LDAP_GROUP_RFC822

Mail addresses of members of this list.

rfc822MailMember

LDAP_GROUP_RFC822

rfc822MailMember is supported for backwards compatibility. Either rfc822MailMember or mgrpRFC822MailMember, but not both, can be used in any given group.

Turns the headers specified in the attribute into header trimming ADD options.

mgrpRemoveHeader

LDAP_REMOVE_HEADER

Turns the headers specified into header trimming MAXLINES=-1 options.

mgrpMsgPrefixText

LDAP_PREFIX_TEXT

Adds the specified text to the beginning of the message text, if any.

mgrpMsgSuffixText

LDAP_SUFFIX_TEXT

Adds the specified text to the ending of the message text, if any.

One final attribute is checked in the special case of group expansion as part of an SMTP EXPN command: mgmanMemberVisibility or expandable. The LDAP_EXPANDABLE MTA option can be used to select different attributes to check. Possible values are: anyone, which means that anyone can expand the group, all or true, which mean that the user has to successfully authenticate with SASL before expansion will be allowed, and none, which means that expansion is not allowed. Unrecognized values are interpreted as none. If the attribute is not present, the EXPANDABLE_DEFAULT MTA option controls whether the expansion is allowed.

Alias entries are cached in a fashion similar to domain entries. The MTA options controlling the alias cache are ALIAS_ENTRY_CACHE_SIZE (default 1000 entries) and ALIAS_ENTRY_CACHE_TIMEOUT (default 600 seconds). The entire LDAP return value for a given alias is retained in the cache.

Negative caching of alias entries is controlled by the ALIAS_ENTRY_CACHE_NEGATIVE MTA option. A nonzero value enables caching of alias match failures. A zero value disables it. Negative caching of alias entries is disabled by default. The theory is that repeated specification of an invalid address is unlikely to occur very often in practice. In addition, negative caching may interfere with timely recognition of new users added to the directory. However, sites should consider re enabling negative caching of aliases in situations where vanity domains are heavily used. The search performed by the URL specified in ALIAS_URL0 is less likely to be successful.

Address Reversal

Address reversal with direct LDAP starts with a USE_REVERSE_DATABASE value of 4, which disables the use of any reverse database. It then builds on the routing facilities previously discussed. In particular, in the previous version, it began with a reverse URL specification of the form:

REVERSE_URL=ldap:///$V?mail?sub?$Q

The $V metacharacter has already been described in the context of alias URLs However, the $Q metacharacter, while quite similar in function to the $R metacharacter used in alias URLs, is specifically intended for use in address reversal. Unlike $R, it produces a filter that searches attributes containing addresses that are candidates for address reversal. The list of attributes to search comes from the MTA option LDAP_MAIL_REVERSES. If this option is not set, the local.imta.schematagconfigutil parameter is examined, and depending on its value, an appropriate set of default attributes is chosen.

The use of $Q is no longer appropriate, however. In order for message capture and other facilities to work correctly, address reversal has been enhanced to pay attention to the attribute that matched in addition to the fact that a match occurred. This means that $R should be used to specify the filter instead of $Q. Additionally, the $N metacharacter has been added, which returns a list of the attributes of interest to address reversal. The resulting option value is:

REVERSE_URL=ldap:///$V?$N?sub?$R

As always, local.imta.schematag can be a comma-separated list. If more than one schema is supported, the combined list of attributes, with duplicates eliminated, is used.

Additionally, the filter searches not only for the address that was originally supplied, but also for an address with the same local part but the domain that was actually found in the domain tree (which was saved in Step 2. The iterative nature of the domain tree lookup means the two addresses may be different.

For example, suppose that the domain siroe.com appears in the domain tree and the MTA looks the address:

u@host1.siroe.com

The filter that results from the expansion of $R and an ims50 schematag will be something like:

Note that the reverse URL explicitly specifies the attribute containing the canonicalized address. Normally this will be the mail attribute.

After the URL is constructed an LDAP search is performed. If the search is successful the first attribute value returned replaces the original address. Unsuccessful searches or errors leave the original address unchanged.

Due to the frequency with which address reversal operations are performed, especially given the number of addresses that can appear in a message header, and the expense of the directory queries involved, both negative and positive results need to be cached. This is implemented with an in-memory, open-chained, dynamically-expanded hash table. The maximum size of the cache is set by the REVERSE_ADDRESS_CACHE_SIZE MTA option (default 100000) and the timeout for entries in the cache is set by the REVERSE_ADDRESS_CACHE_TIMEOUT MTA option (default 600 seconds). The cache actually stores addresses themselves, not the LDAP URLs and LDAP results.

Asynchronous LDAP Operations

Asynchronous lookups avoid the need to store an entire large LDAP result in memory, which can cause performance problems in some cases. The MTA provides the ability to perform various types of lookups done by the MTA asynchronously.

Use of asynchronous LDAP lookups is controlled by the MTA option LDAP_USE_ASYNC. This option is a bit-encoded value. Each bit, if set, enables the use of asynchronous LDAP lookups in conjunction with a specific use of LDAP within the MTA.

Table 9-11 shows the bit and value settings for the LDAP_USE_ASYNC MTA option in the option.dat file.

Note that the last of these options also handle the case of wild carded local parts in hosted as well as vanity domains. If wild carded local part support is desired but vanity domain support is not, the following option should be used instead:

ALIAS_URL1=ldap:///$V?*?sub?&(mailAlternateAddress=@$D)

The filter ssrd:$A clause needs to be removed from the ims-ms channel definition in the MTA configuration file (imta.cnf).