This document describes how amavisd-new can be integrated into the
Postfix SMTP delivery process. It lists the necessary requirements,
explains how Postfix and amavisd-new need to be configured to basically
work together and it gives filter-examples to show how amavisd-new can be
called from Postfix.

1. Requirements

The following requirements must be met before integration can
begin:

amavisd-new has already been installed and successfully
tested.

Postfix has been installed, configured for basic operations and
tested successfully.

Tip

Independently of the configuration examples shown in this
document, it is advisable to set
strict_rfc821_envelopes = yes in
/etc/postfix/main.cf. Postfix will reject any
message from envelope-senders, whose address can't be used to send a
reply to.

This avoids accepting e-mails from erroneous envelope-senders that
can't be informed of problems, which finally would result in deleting
the message - even if Postfix claimed successful delivery in the
first.

1.1. Which Postfix version is required?

Integrating amavisd-new into the Postfix delivery process requires
that Postfix is able to delegate messages to external content filters.
The minimum version that provides content filtering is Postfix
release-20010228.

1.2. Catching errors during integration

Chances are that configuration errors during implementation cause
Postfix to bounce legitimate messages. Setting the
soft_bounce parameter during integration and
reloading the Postfix configuration afterwards prevents Postfix from
bouncing legitimate mail during that time:

# postconf -e "soft_bounce = yes"
# postfix reload

As soon as soft_bounce has been activated
Postfix will treat all delivery errors as temporary errors - any client
that wants to send messages to Postfix will keep mail in the mailqueue
and it will suspend delivery until the
soft_bounce parameter has been removed or set to
no.

Once the integration of amavisd-new into the Postfix delivery
process has been completed successfully
soft_bounce must be removed or Postfix will not
generate bounce messages for legitimate mail.

2. Basic Postfix and amavisd-new configuration

There are several moments at which Postfix can hand over messages
over to amavisd-new (before it accepts a message from a client or after)
and there are different filter approaches (globally, per recipient
(domain), per network interface, etc.) that can trigger Postfix to
transport a message to amavisd-new.

The transport methods - transporting a message from Postfix to
amavisd-new and backwards - however always remain the same. They will be
described in this section first. The section that follows will deal with
different filter approaches.

Integration procedure

The following examples have been structured to cause minimum
trouble on an online mail system. The order of steps ensures that
filtering will be enabled at the very last moment. Several tests will
have been conducted to verify the delivery chain works before the filter
is enabled. Once enabled the complete system should work at once.

2.1. Configuring amavisd-new for Postfix

Configuring amavisd-new to work with Postfix answers the following
two questions:

Which port should the amavisd-new daemon listen to for
incoming connections from Postfix?

Which IP-address and port should the amavisd-new SMTP client
use to (re)inject filtered messages (and notifications about message
statuses) into the Postfix SMTP delivery system?

2.1.1. Configuring amavisd-new for incoming connections

The $inet_socket_port in
/etc/amavisd.conf parameter sets the port number
amavisd-new will listen for incoming (E)SMTP connections. The
following example explicitly configures amavisd-new to bind to port
10024 (default setting
undef):

$inet_socket_port = 10024;

2.1.2. Configuring the reinjection path

Two parameters, $forward_method and
$notify_method, need to be configured (usually
identically) to reinject messages into the Postfix mail system.

The first parameter, $forward_method,
specifies where amavisd-new should transport scanned messages to,
while the second parameter, $notify_method,
specifies where notifications about scanned messages should be
transported to.

By default amavisd uses 127.0.0.1 on port 10025 to contact a SMTP server for
reinjection of filtered messages. Unless a different IP address or
port should be used, no modifications must be applied and this section
can be skipped.

In case a different IP address or port should be used, the
parameters $notify_method and
$forward_method need to be adjusted to reflect
these requirements. The following example edits these parameters in
/etc/amavisd.conf and uses 192.0.2.1 as IP address and port
20025:

2.2. Configuring the transport from Postfix to amavisd-new

Both, amavisd-new and Postfix, are able to use either SMTP- or
LMTP-communication to transport a message from Postfix to amavisd-new.
Both variants will be described in this section.

Why configure a dedicated service?

Theoretically it's possible to transport messages from Postfix to
amavisd-new using the existing smtp-, lmtp, or even the relay-service in
/etc/postfix/master.cf.

In practice transporting messages to amavisd-new requires imposing
transport limits on the transporting service. Imposing such limits on a
globally available service would impose these limits on the complete
Postfix mail system - it would slow down the system significantly and
should be avoided.

Note

The number of Postfix clients that may connect simultaneously to
amavisd-new instances must be limited to the maximum number of daemon
child processes amavisd-new starts.

If the Postfix transport client was allowed to open more
connections amavisd-new can handle, amavisd-new would start to queue
incoming Postfix connections. Postfix in turn would interpret such
behaviour as “unresponsive remote MTA” and would itself
begin to queue mail that should be filtered. All this would possibly
throttle down the complete system and all further filtering attempts
would suffer.

2.2.1. Configuring a dedicated lmtp-client

The following example creates a new, dedicated lmtp-transport
named amavisfeed in
/etc/postfix/master.cf. Its configuration details
are explained following the listing:

A noteworthy quote from the Postfix documentation:
“...do not specify whitespace around the ‘=’. In
parameter values, either avoid whitespace altogether, ...”.
Further details on master.cf configuration
syntax can be found in master.cf or
master(5).

Here's a quick rundown on the settings that differ from other
services defaults:

maxproc

The maximum number of concurrent Postfix amavis-service
processes has been limited to 2 (default:
default_process_limit =
100). This value reflects the default of
2 amavisd-daemon children processes and is a
good setting to start from. The value may be raised later, when
the system works stable and still can take a higher load. It
should not exceed the number of simultaneous amavisd child
processes.

lmtp_data_done_timeout

Setting lmtp_data_done_timeout to
1200 (seconds) doubles the default time span a
regular Postfix client waits after message delivery for the
server to reply DONE to claim
successful delivery. It must be larger than amavisd setting
$child_timeout (default
8*60 seconds) and should add a
sufficient safety margin, for example to cater for periods of
automatic database maintenance (e.g. bayes database on non-SQL
database types) which can take a long time in some cases.

If the server does not reply within the configured time
span, the Postfix client will quit the connection, put the
message into the deferred queue, log a delivery failure and
retry later to transport the message to amavisd-new.

Note

Raising this value serves a trick amavisd uses to avoid
message loss in case of power outage etc. The trick consists
in keeping the incoming connection as long open as it takes to
filter the message and take appropriate action (reinjection,
notification, quarantine, etc.).

Only when the message (or notifications etc.) has been
reinjected amavisd will send
DONE to the client and the
client will close the connection. This way Postfix will always
keep the message in its own mail queue, where it can be
reactivated after a system failure.

lmtp_send_xforward_command

Enabling lmtp_send_xforward_command
configures the Postfix lmtp-client to forward the original
clients HELO name and IP address to amavisd-new. amavisd-new in
turn can use these informations for

logging and notifications (macro
%a)

switching policy banks (MYNETS,
@mynetworks_maps)

pen pals functionality

p0f fingerprinting

disable_dns_lookups

The transport route from Postfix to amavisd-new, it will
be configured later in Section 3, “Message filtering examples”, will probably
never change. It will - probably - only change when the whole
mail system is being reconfigured. The target host may therefore
be specified as IP address instead of using a DNS hostname. This
saves “expensive” DNS-request (3 lookups) and
improves performance.

max_use

By default Postfix reuses a service instance 100 times
(max_use = 100), before
the instance terminates. The master daemon will reinvoke such a
service if required. There's no need for the amavisfeed-service
to have such a long life-span. Best practice has it to set
max_use to 20.

2.2.2. Configuring a dedicated smtp-client

Configuring a dedicated smtp-client is almost identical to
configuring a dedicated lmtp-client. The syntax differences in detail
are that the names of parameters start with
smtp_ instead of lmtp_
and that the command at the end of the service invokes the smtp- and
not lmtp-client. The same reasons given for differing lmtp client
options apply to the dedicated smtp client configuration.

Here's an example of a dedicated smtp client given the service
name amavisfeed:

2.3. Configuring a dedicated SMTP-server for message
reinjection

The second service that needs to be added to the Postfix mail
system is a dedicated SMTP-server. It will exist only to accept filtered
messages and notifications from amavisd-new to transported them closer
to their final destination.

This dedicated smtpd server will differ in many aspects from the
default smtpd daemon. The most important difference is that it
configures an empty content_filter parameter,
thus overriding any global external content filtering settings in
Postfix.

Note

Delegating messages to an external content filter in Postfix is
done using the content_filter parameter. If the
dedicated smtpd-daemon would not override any global
content_filter settings, the reinjected message
would be sent of to the external content filter again - the mail would
end in an endless loop.

The following Postfix example uses amavisd-new default settings
taken from the $forward_method and
$notify_method parameters. These settings
configure amavisd-new to forward filtered messages and notifications to
127.0.0.1 on port 10025; the Postfix smtpd daemon will be
configured to bind to that IP address and listen on the specified port
for incoming connections:

Empty ..._maps override any other
globally set map lookups. Procedures to enforce settings specified
in such maps have already taken place when Postfix accepted the
message from the external client. Doing them again will not
produce new results but only waste resources.

..._restrictions_...

There's no need to apply any already enforced
..._restrictions_... another time. It would
also only waste resources.

mynetworks

To avoid abuse from remote hosts, the dedicated smtpd-daemon
will only allow clients from 127.0.0.0/8 to relay
messages.

local_header_rewrite_clients

By default this option would “rewrite message header
addresses in mail from these clients and update incomplete
addresses with the domain name”. If such action has already
been taken by Postfix before the message went off to amavis, it
should not be done a second time when it reenters the Postfix mail
system. Leaving this option empty disables local header rewrites
and saves resources.

remaining options

All remaining options either configure the dedicated
smtpd-daemon to be more failure tolerant or exist to avoid
unnecessary use of resources.

Running the postfix reload will activate the new transports
(Postfix will not yet send regular mail to amavisd). Combined with the
tail command problems can easily be detected:

# postfix reload && tail -f /var/log/maillog

If there are no problems reported, basic configuration can be
tested.

2.4. Testing basic configuration

Testing basic configuration consists of three separate tests,
starting at the end of the new delivery chain and working to it's
beginning. Their goal is to answer the following questions:

Will amavisd-new accept connections at the specified IP
address and port?

Will the new dedicated smtpd-daemon accept connections at the
specified IP address and port?

Will a test message, injected into amavisd-new, be filtered,
sent to Postfix and delivered into a mailbox?

2.4.1. Testing amavisd's host and port

A test, using the telnet command, serves to verify that amavisd
listens on the specified IP address and port. A successful connection
looks like this:

amavisd connects with Postfix dedicated smtpd-daemon and
hands over the e-mail that had been sent during the telnet
session. smtpd gives a queue-id of 079474CE44 that can be tracked
throughout the maillog.

amavisd notices it has checked and sent an e-mail to
<postmaster>.

Postfix' local-service logs it successfully delivered an
e-mail with queue-id 079474CE44 to the mailbox of
postmaster.

If the test fails, the following questions may help to debug the
problem:

Does amavisd-new log errors?

Does running amavisd-new in debug-mode report errors?

3. Message filtering examples

Postfix can use various criteria to decide whether a message should
be sent to amavisd-new for examination. Combinations of criteria may serve
to create different configurations. The following section describes the
following configurations:

Filtering e-mail globally

Filtering e-mail globally by service

Filtering e-mail per recipient domain

Filtering e-mail per sender domain

Filtering e-mail by content

3.1. Filtering E-mail globally

In most cases email policies require global filtering - every
inbound and every outbound e-mail must be filtered by amavisd-new -
before it may be sent closer to its final destination.

Why check outgoing mail traffic?

Some reasons for checking mail coming from internal networks or
from authenticated roaming users are:

detect an internal infected PC which is sending
viruses

detect an internal zombiized PC (or an internal open relay
or proxy) which is sending or relaying spam

let the SpamAssassin Bayes autolearning feature see a
balanced view of all mail, including useful samples of non-spam
originating from inside

make it possible for pen pals feature to function (if
enabled)

In Postfix global settings for its services are written to
main.cf. The content_filter
parameter, the parameter configuring that messages are sent to
amavisd-new, must therefore be placed in
main.cf.

The content_filter parameter requires a
triplet, consisting of the transport service's name (here: amavisfeed,
given in Section 2.2.1, “Configuring a dedicated lmtp-client”), the target
hosts IP address and the port where amavisd-new listens for incoming
connections. Following the values used in this documents examples the
content_filter configuration results in
this:

content_filter=amavisfeed:[127.0.0.1]:10024

The new external content filter will be activated once Postfix has
been reloaded. Sending a test-mail verifies the system works.

3.2. Filtering E-mail by Postfix service

Postfix is able to filter messages per service. Such configuration
requires the content_filter not to be applied
globally to all services in main.cf (see: Section 3.1, “Filtering E-mail globally”), but selectively, per service in
master.cf.

The following example presumes Postfix runs on a system offering
three IP addresses. In this example these are: 192.0.2.1 (WAN), 127.0.0.1 (localhost) and 10.0.0.254 (LAN). The goal is to filter
only e-mail that enters from the WAN interface.

This requires to create three dedicated smtpd-daemon instances,
each binding to one of the given IP addresses and deactivating the
global smtp service calling the smtpd command.

Additionally the WAN interface (here: 192.0.2.1:25) is configured
to use content_filter=amavisfeed:[127.0.0.1]:10024 - it will delegate any
message that enters the Postfix mail system at this service to the
external amavisd content filter.

3.3. Filtering E-Mails per Recipient Domain

Postfix is able to filter e-mails per recipient domain. In order
to do this the content_filter parameter must not
be set globally (see: Section 3.1, “Filtering E-mail globally”). Instead the
content_filter parameter has to be associated
with one or more recipient domains listed in a lookup table
(map).

Caution

This filter method is not selective! It will send
any mail with a recipient domain listed in the
lookup table to amavis even if the mail contains another recipient
that should not be examined by the amavis
framework.

If fully selective rules are required all mail should be sent to
amavis and amavis' own rule sets should be configured to decide
whether a message for a given recipient should be examined or
not.

When Postfix searches the lookup table and finds the recipients
domain listed as key, it will take the action associated with that
domain. The action will send the message to a FILTER
amavisfeed:[127.0.0.1]:10024.

The following map
/etc/postfix/filter_recipient_domains specifies to
send messages to the FILTER amavisfeed whenever a
message for any recipient at example.com enters the Postfix
mailqueues:

example.com FILTER amavisfeed:[127.0.0.1]:10024

Once the table has been created the postmap
command must be used to create an indexed map Postfix can read:

# postmap /etc/postfix/filter_recipient_domains

Once the map has been indexed, the postmap
command is used to test the map. In the following example the
postmap command queries for the domain example.com
and returns the associated action:

The tested map must be added to main.cf,
before Postfix can make use of the new filter policy. Setting the
check_recipient_access parameter in the list of
smtpd_recipient_restrictions triggers evaluation
of entries in the map - check_recipient_access is
triggered by the envelope-recipient(s) given by a SMTP-client in a
SMTP-session with Postfix.

The following example puts the
check_recipient_access rule before
permit_mynetworks - all clients
envelope-recipient(s) will be filtered:

3.4. Filtering E-Mails by Sender-Domain

In general it doesn't make sense to filter e-mails by
sender-domain, as anyone can fake a sender-domain during a SMTP-session.
Filtering by sender-domain will probably only make sense, if messages
are not filtered globally, but e-mails from ones own domain should be
checked for spam or viruses before they leave the network.

Most of the configuration steps are identical with the ones noted
in Section 3.3, “Filtering E-Mails per Recipient Domain”, except for the parameter that
triggers evaluation of the indexed map. In this scenario
envelope-senders should trigger map evaluation. The map, named
/etc/postfix/filter_sender_domains this time,
contains the sender domain (example.com) and associates it with the
required FILTER:

The map must be listed before
permit_mynetworks, because only then it will be
applied to all clients - even the ones Postfix trusts, which are very
likely the ones from example.com.

3.5. Filtering E-mail per Content

Postfix is able - with deliberate limitations (see:
BUILTIN_FILTER_README) - to search for strings in
headers, the body and MIME-headers. If a string matches, Postfix may
call appropriate action.

The following example configures Postfix to look for the string
offer in Subject:-headers and delegate the message to
an external content filter if if finds a matching string.

A map, consisting of the search string noted as regexp-expression,
associates the search pattern with a FILTER action:

/^Subject:.*offer/ FILTER amavisfeed:[127.0.0.1]:10024

Indexing regexp- or pcre-maps?

regexp- or pcre-maps are and must be plaintext files. They
must not and cannot be converted to an indexed
map using the postmap command. They can be tested
using the postmap command using the
-q command line option.

Once the map has been created, Postfix must be configured to use
it. The following example uses the header_checks
parameter (not body_checks or
mime_header_checks as they apply to other message
parts) to implement the map into the Postfix delivery process:

header_checks = regexp:/etc/postfix/filter_header

Once Postfix has been reloaded it will send every e-mail that
contains the word offer in the Subject:-header off to
the external amavisd content filter.

4. Advanced Postfix and amavisd-new configuration

In a post-queue content filtering setup, a mail message passes
through smtpd and
cleanup Postfix services twice, once before a
content filter, and the second time when an approved message is reinjected
from a content filter into the Postfix mail system. This is because checks
and transformations that have been configured in
main.cf are globally active and will be loaded and
run by any instance of these two services. To avoid wasting resources,
options that control runtime behavior of these services should not be
applied globally in main.cf, but selectively to
separate instances of these services in
master.cf.

Checks and transformations which are performed by a cleanup Postfix service are trickier because
in a normal Postfix setup there is only one
cleanup service, unlike
smtpd services of which there are many. Some of
the more important cleanup settings are
dynamically controllable by a smtpd service
through the use of its receive_override_options
option.

Transformations and checks

Any transformation should preferably only be performed once,
either before or after content filtering. When to transform depends on
the desired effect, for example whether a content filter should see
unchanged or modified mail messages. Typical transformations are:

rewrite addresses

add BCC recipients

modify mail header.

Most checks should also be performed only once, preferably only on
the first passage, when the mail enters the Postfix mail system the
first time. This way messages can be rejected early - if needed - and
will not tie up downstream resources. Checking early also avoids bounces
in case of negative check results on a second passage after content
filtering.

4.1. Multiple cleanup service architecture

To gain more control over a cleanup
service than offered by receive_override_options,
two (or more) cleanup services, each with its
own set of options, must be run. A Postfix setup with more than one
cleanup service is possible either with two
separate Postfix instances, or through a specification of services and
their options in master.cf file of a single Postfix
instance.

The following diagram illustrates a setup with two cleanup
services in a single Postfix instance:

Such transformations will be done later by the second
cleanup service, so that a content filter
will see original (external) recipient mail addresses. Other options
may also be used as needed.

4.2.2. Modifying the existing cleanup service

The already existing cleanup service - having the service name
cleanup - will be used to process messages
that re-enter the Postfix mail system (also for delivery notifications
and forwarding as generated internally by Postfix).

Cleanup jobs that already have been performed by the
pre-cleanup service should not be run again.
The following example disables typical checks that have been run
before or are not needed for internally generated
notifications:

The specified options disable header and body checks as
these would already be performed by a
pre-cleanup service.

always_bcc

This cleanup service would also be
the appropriate one for specifying always_bcc
option - doing it globally would apply to both
cleanup services and would result in
two copies of each message to be sent to the
specified address.

4.2.3. Configuring smtpd services

Finally existing smtpd services on
ports 25 and 587 (submission), and the
pickup service must be configured to send
messages to the new pre-cleanup service
instead of a default cleanup service:

5. Tuning

5.1. Maximum Number of Concurrent Processes

The most important settings to tune and optimize in Postfix and
amavisd workflow are the maximum number of concurrent processes. The
maximum number of concurrent processes on both sides must be chosen with
care.

If the number is too low, hardware resources aren't used
efficiently and delivery time will be unnecessarily prolonged.
Experience tells that raising the number of processes a little, will not
raise the overall throughput in the same proportion.

As the system resources are nearing saturation with each increase
of the number of processes, an increase in throughput becomes marginal,
and eventually even negative when the number of processes exceeds its
near-optimum value. E-mail throughput will decrease, because processes
need to wait for each other. At worst e-mail delivery stalls.

Best practice is to start with a (conservative) maximum number of
2 concurrent processes. Everyday use has shown that this value may be
raised to a value between 10 and 30 concurrent Postfix client and
amavisd server processes. This also depends on the overall resources the
system may provide, how amavisd has been integrated into the Postfix
delivery process and on the anti-virus and anti-spam software being
loaded and used by amavisd-new.

Regardless of the maximum number of concurrent processes, both
sides - Postfix and amavisd - should be synchronized. To synchronize
both sides edit, the $max_servers parameter for
amavisd-new (see: amavisd.conf) and the number of
processes in master.cf listed in the dedicated
transports maxproc column for Postfix.

Both values should be identical for two reasons: If amavisd-new
offers more processes than Postfix will ever use, amavisd-new wastes
resources. On the other hand, if Postfix starts more dedicated
transports than amavisd can handle simultaneously, e-mail transport will
be refused and logged as error.

Controlling the maximum number of concurrent processes in
main.cf

Instead of controlling the maximum number of concurrent
processes of Postfix' dedicated transport in
master.cf it is also possible to keep the default
setting - in master.cf and set
the following parameter and option in
main.cf:

amavisfeed_destination_concurrency_limit = 2

The name of the parameter starts with the service in
master.cf (here: amavisfeed) that should be
controlled and goes on with the suffix
_destination_concurrency_limit. Here also
2 is set as initial (conservative) value.