Editions

Based on Mandrake 10.x, but valid for all distributions.
Very thorough. Includes package description, where to get the sources and binaries,
how to build them or which RPMs to use, includes many refrences, etc etc.
Starts off with a basic working server, then advances, extends and tightens it in stages.

Research

Don't take my word for it!
Research others opinions and methods.
Look at my references,
look at Postfix.org's howtos,
read the excellent books available
(E.g. Kyle's or Hildebrandt's),
search the web or read the proper
documentation.

If you refer to this howto in your own document,
or find useful links, then
let me know.

Donate

If you found this howto very useful, spread the word and help others?

If this howto was exceptionally useful why not donate me some beer money?

Software

What software packages have/will I use and why.

OS: Ubuntu Linux

Ah, the age old distro argument...
Thankfully this set up should work on most distros.
I used to base this howto on Mandrake(now Mandriva),
and I started this new edition on a Gentoo box.
But I don't have the patience for Gentoo,
nor the money to stay with Mandriva Power editions.
Why Ubuntu? It's free, simple and slick.
As Ubuntu is derived from debian, the installations
used here will be apt-get based.
Please refer to my other editions for details on RPM
or source based installations.

Authentication: Cyrus SASL

PostGrey

Postgrey is an excellent little script to stop 99% of all spam.
All it does is on first contact for specific from-to combinations,
tells the sender server to try again in a little while,
which most spammers can't afford to do.
When legitimate servers try again after a few minutes it lets it through.

Encryption: TLS

Secure and trusted cryptography technology
for encryption of SMTP traffic.
Not too be confused with client encryption technology
like GnuPG and S/MIME. They are covered in the
extend section.
Formerly referenced as SSL.

Please see software links appendix for further information
about these software packages. In that section there are more links to
documentation or forums, and viable alternatives, downloadable packages, versions, details etc.

Installation

Distribution

This section is different for every distribution and for every version.

This howto is based on Ubuntu and its base of debian which uses apt-get.
Therefore this section uses apt packages to its fullest.

For other installation method please refer to previous editions
software links
and your own distribution
for the documention for other ways of installing.
My 2nd edition(outdated) has instructions
for Mandriva, general RPM and tarball compiling.

To follow the rest of this howto with another distribution,
you need to ensure all your
packages have been installed with the same modules,
E.g MySQL lookup on postfix and sasl, php in apache etc.

I have set up mail servers using the 32bit and 64bit x86 platforms,
and if all the packages are available then other,
E.g. Mac platforms should work too.

Base Install

With installing Ubuntu you have a choice of which base system to install.
You may choose server or desktop image or very basic setups.
I will assume a server install, but it should not differ.

Repositories

But basically for this you need main and universe, and I also throw in the other "safe" ones: restricted and multiverse (and partner, when available).
sudo vi /etc/apt/sources.list
If main and universe are already listed,
this is a quick find and replace to add the others all over:
:%s/main un/main restricted multiverse un/g

As mentioned in the previous edition
you also might want to find a repository closer to your server.

Packages

You need to install a whole bunch of packages.
We will install them bit by bit.
But first check your package sources are correctly pointing to main multiverse restricted universe
repositories of your current Ubuntu version.
sudo vi /etc/apt/sources.list
Secondly update your current system:
sudo aptitude update
sudo aptitude safe-upgrade

MySQL

First we'll install MySQL
sudo aptitude install mysql-client mysql-server
This will prompt you for a root password.
Choose something wise and remember it!
For the purpose of this tutorial I will set it to rootPASSWORD

Postfix

Then we'll install postfix
sudo aptitude install postfix postfix-mysql
This will prompt you to choose type of email server.
Select internet site
It will also suggest a server name. Correct this if needed.

ShoreWall

Amazon provides a firewall/ access control for its servers,
so not always needed then, but nice to have.
And in all others situations; a must have.

Courier

sudo aptitude install courier-base courier-authdaemon courier-authlib-mysql \
courier-imap courier-imap-ssl courier-ssl
will prompt you about webdirectories. You can say no to this.
It will also warn you about the certificate location. Ignore it.

Extras

I also install a few other packages that I personally prefer.
But these have nothing todo with the mail server.
sudo aptitude install vim mutt lynx

Package status5>

To find out which packages you may have installed,
you can use for example:
sudo dpkg --list | grep postfix
or
sudo aptitude search postfix

Simple mail server

Now let's configure a simple mail server using some of
the packages installed.

Firewall

Shorewall

Not essential for an EC2 image.
It is essential for a normal server.
UFW is bundled with recent Ubuntu distributions,
but I still prefer Shorewall for servers.

Basically at first you want to only allow SSH.
Then SMTP and IMAP from your IP only.

When you are confident that the mail server is secure,
you can open SMTP to the world.
If you prefer you can also open IMAP to the world,
unless you have a very small client IP range.

Later you may open web access to the webmail and admin gui.
This you may also restrict to specific IPs.

SSH only

By default Shorewall in Ubuntu has an empty set up.
You can find the default values for Shorewall in
/usr/share/doc/shorwall/default-config.
And examples in /usr/share/doc/shorwall/examples.
We will create a basic set up.

Then we will configure network zones
cp /usr/share/doc/shorewall/default-config/zones /etc/shorewall/
vi /etc/shorewall/zones
Add the firewall if not there and the internet as a zone.
fw firewall
# loc ipv4
net ipv4

Then if needed to specify hosts you can do it in this file.
E.g. If you wanto specify what is your home IP etc.
cp /usr/share/doc/shorewall/default-config/hosts /etc/shorewall/
vi /etc/shorewall/hosts# loc eth0:192.168.0.0/24

Then set what is the default policy for firewall access.
cp /usr/share/doc/shorewall/default-config/policy /etc/shorewall/
vi /etc/shorewall/policy$FW net ACCEPT
net $FW DROP info
net all DROP info
# The FOLLOWING POLICY MUST BE LAST
all all REJECT info

For safety in case it goes down.
cp /usr/share/doc/shorewall/default-config/routestopped /etc/shorewall/
vi /etc/shorewall/routestoppedeth0 0.0.0.0 routeback
You may put in a netmask of your ip range if you are more concerned.

Now for the main firewall rules.
You can find predetermined macro rules for Shorewall in
/usr/share/shorewall.
cp /usr/share/doc/shorewall/default-config/rules /etc/shorewall/
vi /etc/shorewall/rulesSSH/ACCEPT net $FW

Open for business

Once your server is working
come back to this step and
open up SMTP and Web access to others.

MTA

Postfix

You should put the name of your server in this file
sudo vi /etc/mailname
Could be something like smtp.domain.name,
where domain name obviously is replaced with your domain name.

Now will open the main postfix configuration file:
sudo vi /etc/postfix/main.cf
Debian and Ubuntu already puts in some sensible default values in this file.
You may need to comment some of them out if we put the same in as well.

First specify the name of your server.
# This is already done in /etc/mailname
#myhostname= mail.example.com
Next is the origin which is the domain appended to email from this machine,
this can be your full servername, or domain name.
# myorigin=/etc/mailname
myorigin=example.com

Then decide what the greeting text will be.
Enough info so it is useful,
but not divulge too much to potential hackers.
smtpd_banner = $myhostname ESMTP $mail_name

Next you need to decide whether to send
all outgoing mail via another SMTP server,
or send them yourself.
I send via my ISP's server,
so it has to worry about the queing etc.
If you send it yourself then you are not reliant
on 3rd party server.
But you may risk more exposure and
accidentally be blocked by spam blockers.
And it is more work for your server.
Also many servers block dynamic dns hosts,
so you may find your server gets rejected.
However choose whichever you are comfortable with.
# leave blank to do it yourself
relayhost =# or put it an accessible smtp server
relayhost = smtp.yourisp.com

Next is network details.
You will accept connections from anywhere,
and you only trust this machine
inet_interfaces = all
mynetworks_style = host

Next you can masquerade some outgoing addresses.
Say your machine's name is mail.domain.com.
You may not want outgoing mail to come from
username@mail.example.com, as you'd prefer
username@example.com.
You can also state which domains not to masquerade.
E.g. if you use a dynamic dns service,
then your server address will be a subdomain.
You can also specify which users not to masquerade.
# masquerade_domains = mail.example.com www.example.com !sub.dyndomain.com
# masquerade_exceptions = root

As we will be using virtual domains, these need to be empty.
local_recipient_maps =
mydestination =

Then will set a few numbers.
# how long if undelivered before sending warning update to sender
delay_warning_time = 4h
# will it be a permanent error or temporary
unknown_local_recipient_reject_code = 450
# how long to keep message on queue before return as failed.# some have 3 days, I have 16 days as I am backup server for some people# whom go on holiday with their server switched off.
maximal_queue_lifetime = 7d
# max and min time in seconds between retries if connection failed
minimal_backoff_time = 1000s
maximal_backoff_time = 8000s
# how long to wait when servers connect before receiving rest of data
smtp_helo_timeout = 60s
# how many address can be used in one message.# effective stopper to mass spammers, accidental copy in whole address list# but may restrict intentional mail shots.
smtpd_recipient_limit = 16
# how many error before back off.
smtpd_soft_error_limit = 3
# how many max errors before blocking it.
smtpd_hard_error_limit = 12

Next we need to set some maps and lookups for the virtual domains.
# not sure of the difference of the next two# but they are needed for local aliasing
alias_maps = hash:/etc/postfix/aliases
alias_database = hash:/etc/postfix/aliases
# this specifies where the virtual mailbox folders will be located
virtual_mailbox_base = /var/spool/mail/virtual
# this is for the mailbox location for each user
virtual_mailbox_maps = mysql:/etc/postfix/mysql_mailbox.cf
# and this is for aliases
virtual_alias_maps = mysql:/etc/postfix/mysql_alias.cf
# and this is for domain lookups
virtual_mailbox_domains = mysql:/etc/postfix/mysql_domains.cf
# this is how to connect to the domains (all virtual, but the option is there)# not used yet
# transport_maps = mysql:/etc/postfix/mysql_transport.cf

You can (as in my older editions) use a lookup for the uid and gid of the owner of mail files.
But I tend to have one owner(virtual), so instead add this:
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000

You need to set up an alias file.
This is only used locally,
and not by your own mail domains.
sudo cp /etc/aliases /etc/postfix/aliases
# may want to view the file to check if ok.
# especially that the final alias, eg root goes
# to a real person
sudo postalias /etc/postfix/aliases

Next you need to set up the folder
where the virtual mail will be stored.
This may have already been done by the apt-get.
And also create the user whom will own the folders.
# to add if there is not a virtual user
sudo mkdir /var/spool/mail/virtual
sudo groupadd --system virtual -g 5000
sudo useradd --system virtual -u 5000 -g 5000
sudo chown -R virtual:virtual /var/spool/mail/virtual

Note: If using Amazon ec2 you may want to move the mail spool
to /mnt or an EBS location.
You will need to symlink correctly afterwards.

If you specify an ip in hosts,
(as opposed to 'localhost')
then it will communicate over tcp
and not the mysql socket. (chroot restriction).
Ps. remember to replace the passwords with your chosen mail user password.

Database

MySQL

Now we will need to create the tables for thos lookups just specified.
First you need to create a user to use in MySQL for mail only.
Then you need to create the database,
Take note of your chosen mail username and password.
You will need the password you specified for root during MySQL package installation.

Next is to edit the MySQL's my.cnf file.
In Ubuntu/debian this is created by default.
In Mandrake I had to manually create a blank one in /etc.
But we need to configure it, so:
sudo vi /etc/mysql/my.cnf
In previous versions you needed to comment out this line
#skip-networking
However in today's file the default is to bind the
address to localhost, which is fine.
bind-address = 127.0.0.1
It is very useful at the start to log any SQL calls that makes it to MySQL.
So enable these lines:
general_log_file = /var/log/mysql/mysql.log
general_log = 1
Then in a few weeks comment it out when everything is working,
as it slows mysql down

Restart MySQL to make sure its picking up the new settings.
sudo /etc/init.d/mysql restart

Summary

You now have a basic mail server!

Before continuing to the advanced and secure mail server you must ensure the basic setup works.
This will save you from loads of pain further on.
It is very easy to make typos, miss tiny steps, unclear steps or simple actual errors in this howto.

Initially we will not enable spam or virus detection!
This is so we can get amavis set up to receive, check and pass on
emails before we go on and over-complicate it.

All of amavis' configuration files are in /etc/amavisd.
They are now spread across several files in conf.d.
Debian and Ubuntu defaults are now very sensible and spread into
seperate files.
cd /etc/amavis/conf.d

This should be it to get amavis working.
If emails are picked up by amavis and
passed back to postfix then it looks okay.
Only when finished testing do you proced to uncomment the anti-virus and anti-spam lines
in
sudo vi 15-content_filter_mode@bypass_virus_checks_maps = (
\%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);@bypass_spam_checks_maps = (
\%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
But do that after the next section (SpamAssassin).

Anti Virus

ClamAV

ClamAV does not need setting up.
Configuration files are in /etc/clamav,
but they are automatically generated, so do not edit.

By default freshclam, the daemon that
updates the virus definition database, is run 24 times a day.
That seems a little excessive, so I tend to set that to once a day.
sudo dpkg-reconfigure clamav-freshclam
It will also ask if you want it to be daemon (yes) and which server is closest to you.

If needed, the command below will redefine the configuration with a lot of questions.
Not needed unless you need to configure.
sudo dpkg-reconfigure clamav-base

Secure mail server

Stopping hackers, phishers, spammers, your boss and your neighbour from
accessing your server or the traffic in between is important,
and easily done.

Authentication

Normal email traffic between clients and servers are in open plain text.
That includes passwords and content of emails.

SASL

SASL secures the actual authentication (login),
by encoding the passwords so that it can not be easily intercepted.
The rest of the emails are however in clear plain text.

SASL can be a royal pain to set up,
especially as it does not support storing encrypted passwords by default in Ubuntu.
Therefore my previous editions described how to
configure SASL using plain text passwords in the database.

Obviously this is not ideal, so there are ways to combine SASL and storing encrypted passwords.
In the future the packages that comes with Ubuntu may support the
password_format
configuration option for SASL.
But until then you can configure SASL to ask PAM to compare the passwords:

sudo vi /etc/default/saslauthd# Toggle this to yes
START=yes
# Switch this to be under postfix's spool# And add -r so that the realm(domain) is part of the username
OPTIONS="-r -c -m /var/spool/postfix/var/run/saslauthd"

In addition to tailing var/log/mail.log and /var/log/mysql/mysql.log
it is quite usefull to tail the auth.log as well when testing SASL.

tail -f /var/log/auth.log

Restart postfix and saslauthd to enable SASL for sending emails.

sudo /etc/init.d/saslauthd restart
sudo /etc/init.d/postfix restart

Imap SASL / Courier

I tend not to have SASL for my courier authentication,
as I enforce TLS for all my clients.
However if you have a more lenient access policy which is wise if you have many users,
then you may want SASL in Courier as well:

sudo vi /etc/courier/imapd

This may already be avaiable as a commented out line.
If not replace the current line by adding UTH=CRAM-MD5 AUTH=CRAM-SHA1 so it resembles something like this:
(Again on one line)

Encryption

TLS

Encrypting the traffic stops anyone else listening in on your email communications.
And is very recommended. There are different types of communication to encrypt:
The data traffic between your email applications and the server when you read emails
or when you send emails,
and communication between other email servers and your server.

For the encryption of reading emails, it is Courier you need to configure.
For sending, and beetwen server encryption it is Postfix.

TLS in Postfix

To encrypt you need certificates. Ubuntu creates some for you for which you can use
while setting up the server. However before you go live,
it is recommended to create your own with your proper domain name etc.
Please refer to previous edition
for more detail.

Next we have a look at the master.cf file.
vi /etc/postfix/master.cf
By default only the normal smtp service is enabled,
which is fine.
But I prefer to enable submission (port 587),
so that clients can use it, and I can restrict them to TLS only.
Also enabled smtps service (port 465),
for some compatebility with some older clients (outlook express etc).

TLS in Courier

Again Ubuntu has created a certificate for you, but if you want to create your own,
especially for a properly named server, then do this.
cd /etc/courier
openssl req -x509 -newkey rsa:1024 -keyout imapd.pem \
-out imapd.pem -nodes -days 999
For more details review an earlier edition.

Then you need to edit
vi /etc/courier/imapd-ssl
By default Ubuntu already points to you certificate
TLS_CERTFILE=/etc/courier/imapd.pem
Modify this if needed.

Also you if want to restrict IMAP users to SSL/TLS only toggle this setting to 1.
IMAP_TLS_REQUIRED=1

For maximum compatability it is not wise to restrict to TLS only for the traffic between servers. As this means not all valid emails sent by others can reach your server.
However enabling them the option to encrypt is a good idea.

Be aware that the emails are not encrypted on your machine, nor on the server.
For this type of client encryption, please refer to previous edition
for more on GnuPG.

In some situations SASL and TLS do not play well together.
Those situations are in combinations of storing encrypted passwords,
using MD5 authentication over encrypted traffic.
I recommend, insisting on TLS traffic with your authenticating clients,
which then negates the need for SASL.

You know have an advanced secure mail server.
Now is another good point to test the set up so far
and to insert some data in the db.

You may accept the default apache configuration where squirrelmail is folder in all sites. But I prefer virtual hosting. But you dont need to do these next steps.
sudo vi /etc/apache2/sites-available/squirrelmail
Comment out the alias.
# alias /squirrelmail /usr/share/squirrelmail
Uncomment the virtual settings.,
and insert your servers name.
# users will prefer a simple URL like http://webmail.example.com
DocumentRoot /usr/share/squirrelmail
ServerName webmail.example.com
If you have apache SSL enabled in apache, then you can also
uncomment the mod_rewrite section for further security.

You can now go toyourdomain.com/squirrelmail/
or mail.yourdomain.com if you chose virtual host.
This should show a squirrel mail page.
Log in wont work yet though.

Start configuring squirrel mail.

sudo squirrelmail-configure

Initially change nothing. You can customize more afterwards.
You can browse, and exit sub menues by typing R.

Type 2 to edit server settings.
Type A to edit IMAP settings.

Type 8 to edit server software.
Enter courier.
courier

Now they say using TLS over localhost is a waste of time.
But I do anyway.
Type 7 to edit secure IMAP.
Type Y to enable it.

Type 5 to edit IMAP port.
Enter 993

Type S to save your changes.
Hit Enter.

Type Q to exit.

You can now go to yourdomain.com/squirrelmail/
or mail.yourdomain.com if you chose virtual host.
This should show a squirrel mail page.
Log in will now work.
(Except you may not have defined users, check data
section. And they may not have received an email which also means
you can not view any IMAP info.)

Administration

Enable web access

You may choose to restrict phpMyAdmin to a spefic virtual host.
If so you need to, edit
sudo vi /etc/apache2/conf.d/phpmyadmin.conf
and comment out the alias.
#Alias /phpmyadmin /usr/share/phpmyadmin
And insert the alias instead into a virtual host configuration in /etc/apache2/sites-available/.
For this example we are not, and for testing we keep the Alias uncommented.

External changes

Before making any changes you need to have done a few steps externally.
(Or at least before you start testing).

Domain name

You need a domain name to use with your email server.
This may be one you purchased, or a subdomain of an existing one,
or a dynamic one e.g. dyndns.org.

DNS

You will also need to configure the MX details for the DNS of this server.
This is done via your domain registrar, or sometimes an external nameserver(DNS) provider.
You can also host your own DNS via packages such as Bind.

Your provider might let you do this through a GUI, but in this is technically how a the configuration should look like:

domain.tld IN MX 10 yourmailserver.domain.tld

(Replace domain.tld with your domain name, and yourmailserver.domain.tld with the full name of your mail server).
Repeat this for each domain that you want the server to handle.

Further mx entries are possible in the same file, if there are subdomains. And also if you have backup MX servers. Refer to my backup MX section if interested.

Note: Some other mail systems will check via reverse DNS
for a match between IP and mail server name, as part of their spam scoring.

If people need suggestions for domain registrars or dns providers then let me know

You know have a finished mail server.
This is as far as the main guide goes.
Hope it was clear enough to follow.

So what does each of these lines actually do?
Well the domains are pretty straight forward.

The users are as well, it requires four fields.
ID is the email address of the user, and also its username
when loggin in, described later on.
NAME is optional description of the user.
MAILDIR is the name of the folder inside /var/spool/mail/virtual.
It must end in a /, otherwise it wont be used as a unix maildir format.
CRYPT is the encrypted text password to use.

The alises are the interesting part.
Lets start from a top down view to see how emails get delivered:

Say an email arrives addressed to "john@whopper.nu".

Postfix looks up domains and say whopper.nu is an domain it listens to.

Postfix then looks up aliases and searches for a row where the mail field matches "john@whopper.nu".

None does so it next searches for "@whopper.nu",
which is the way to specify catch all others for that domain.

It finds one row and its destination is "xandros@blobber.org".

It then searches for "xandros@blobber.org"
and finds one, which destination is the same as the mail,
therefore it is the final destination.

It then tries to deliver this mail. The look up says blobber.org
is a local mail so it looks up users for a matching id and delivers it
to its maildir.

Lets try "julian.whippit@lala.com".

Postfix looks up domains and it is an domain it listens to.

First lookup does not find this user,
but the next finds the catchall "@lala.com".
But its destination is another catchall, "@blobber.org".

This means Postfix will look for "julian.whippit@blobber.org".
This address is not found either, nor is a catchall for blobber.org.
Therefore this address is not valid and the message will be bounced.

Any mail arriving for "karl@blobber.org" or "karl@lala.com",
gets forward to an external address of "karl.vovianda@gmail.com".
So forwarding is simple. I tend to use a subdomain for all my friends
addresses as easily I forget what their real addresses
are, and I use different email clients all the time.

I also added the required aliases of postmaster and abuse to
blobber.org and whopper.nu.
The catchall for lala.com means they are not required for that domain.
Another useful alias to add is root,
as often you get admin mail from e.g cron jobs within
those domains etc.
Other often used aliases are info, sysadmin, support, sales,
webmaster, mail, contact and all.
But they are also honeypots for spam,
so just include the ones you think you will need.

Adding template

So to add a new domain to the system, You do this, replacing the italics with relevant data:

Test

Common problems

Missed a step

If you mistakenly or intentially skipped past sections,
you may have missed an important step in your configuration,
which my guide pressumes you have followed.

Typo

99% of all problems is spelling errors or typos you entered
while following this guide. Sorry, but it just happens.
Often it can be trivial, such as a space at the end
of the configuration line which was not expected etc.
Or not understanding my example where it is a multi line
entry.

Typo by me

Yes, I make a lot of mistakes. Nothing wrong in that,
but I hope I have corrected most over time.
Any new sections are however at risk... :)

Different application or configurations

It is obviously entirely up to you how you set up your system.
But the more you deviate from this guide,
the more likely incompatibilties or confusion will arrise.

Distrobution/version differences

If you run a different version or even distrobution to this guide,
then some things will be different.
Small issues, such as default values and significant things
such as path differences etc.
Some sections in this guide are not always thouroughly
tested with every new release of Ubuntu,
but these differences gets pointed out by people for me.

Walking before crawling

Gamma rays and little goblins

Test strategy

What steps to think of when testing.

Test early and frequently

It is very helpfull to test early in this set up,
to establish if the first sections are working as expected.

So when you only have your very basic postfix and mysql up: Test it!
That way you know that bit worked and can rule it out of any future problems.
Don't wait till you complicated and mudded the water after amavis, courier etc is added.

By constantly testing if you can send and receive you can tick off and black box
each section as working, and immidietly spot issues.

Isolate the problem

Testing how things work is often about isolating the problem first.
So by using the steps of testing early above, you can see which step caused the problem.

Also if you can't log into your webmail it is often nothing to do with the webmail section
that is causing the problem. Often postfix itself is broken etc.

Test in order

As part of the isolating the problem rule,
you most of the time test in order,
and test each section thus isolating the problem.
This would then quickly isolate the problem when e.g. such as above issues of reading emails via the webmail.
This would be in order:

Access: Can I get(ssh) to the box, and is there a firewall issue?

Database: Is the database up, do my application reach it?

Postfix: Can I send email by command line, do I receive emails via telnet?

Content checks: Do they cause a problem?

Courier: Can I read emails?

Webmail: And last but not least does the web integration work?

Simplify the system

Assisting in isolating the problem,
you often have to disable options and applications.
Such as turn of postgrey or content checks to make sure emails to get delivered.

Switch debug on

Shorewall

You can also switch on more messages for when the firewall is rejecting connections.
Add info to all REJECT, BOUNCE and DROP policies.
sudo vi /etc/shorewall/policy
such as:
net all DROP info

MySQL

There is no point in tailing the mysql log if query debugging is not turned one.
By default it is not.
However in this guide I do switch it on,
in case that was missed switch it on now:
sudo vi /etc/mysql/my.cnf
Make sure this is not commented out
log = /var/log/mysql/mysql.log

Courier

As mentioned in the setup , switching on debugging for Courier is easy:
sudo vi /etc/courier/authdaemonrcDEBUG_LOGIN=2

Amavis

You can also debug amavis:
sudo vi /etc/amavis/conf.d/50-user
And perhaps bump it up if already debugging:
$log_level = 2;

Can postfix receive?

If testing a fuller stack, then amavis, postgrey, clamav-daemon, spamassassin etc must also be running.

Try this locally on the server first, then try from another machine once it is working locally.

Lets try and send a message to xandros@example.org
(replace with your own user in this setup, or use postmaster@localhost)
from you@example.com
(again replace with a real email address you use that is not associated with this server.)
telnet localhost 25
# Open the hand shake with ehlo and the server name you are connecting from...# Change mail.example.com to something valid eg your servername
EHLO mail.example.com# The mail server will then dump out some details about its capabilities, e.g.>250-mail.flurdy.net>250-PIPELINING>....>....# then say who is the sender of this email
MAIL FROM: <your@example.com>> 250 Ok# then say who the mail is for
RCPT TO: <xandros@example.org>
> 250 Ok# then enter the keyword data
data
> 354 End data with <CR><LF>.<CR><LF></LF></CR></LF></CR># enter message bodyand end with a line with only a full stop.
blah blah blah
more blah
.
> 250 Ok; queued as QWKJDKASAS# end the connection with
quit
> 221 BYE

If while you were doing this you were tailing the /var/log/mail.log
you would see some activities and if any errors occured.
(You should probably get some complaints about missing headers as we skipped most...)

If while you were doing this you were tailing the /var/log/mysql/mysql.log as well
you really should have seen some activity otherwise you have a problem.

If you see any errors (or worse no activity) in these log files, this is what you need to fix!
For common problems and solutions check the previous edition.

However if no errors popped up,
and the folder /var/mail/virtual/xandros now exists
then your server can receive emails!

Can postfix send?

You need to make sure you can first receive emails as above

The services MySQL and Postfix are running.

Basically you just tested that above,
but we need double check if it can send out to other servers.
Again we will first test locally, which should work,
then remotely which introduces many possible problems.
telnet localhost 25
# Open the hand shake with ehlo and the server name you are connecting from...# This time it has to be the name of your server
EHLO mail.example.org# The mail server will then dump out some details about its capabilities, e.g.>250-mail.flurdy.net>250-PIPELINING>....>....# then say who is the sender of this email, which is a local user
MAIL FROM: <xandros@example.org>> 250 Ok# then say who the mail is for which is an external address e.g. gmail etc.
RCPT TO: <you@example.com>
> 250 Ok# then enter the keyword data
data
> 354 End data with <CR><LF>.<CR><LF></LF></CR></LF></CR># enter message bodyand end with a line with only a full stop.
blah blah blah
more blah
.
> 250 Ok; queued as QWKJDKASAS# end the connection with
quit
> 221 BYE

We have to assume receiving works above so no need to tail mysql's logs.
However if any rejection errors occured in the mail.log then you have an error.

Can courier read emails

You need to make sure you have received an email and the folder /var/mail/virtual/xandros exists

The services MySQL, courier-authdaemon and courier-imap are running.

There is not too much you can test via telnet for courier.
But you can check if it is up and you can connect to it.
telnet 127.0.0.1 143
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT
QUOTA IDLE ACL ACL2=UNION STARTTLS LOGINDISABLED] Courier-IMAP ready. Copyright 1998-2008
Double Precision, Inc. See COPYING for distribution information.
The rest you would have to test via a proper email IMAP client.

Can amavis check and pass emails along?

You need to make sure you can first receive emails as above

You need to make sure you can send emails as above

You need to make sure you have received an email and the folder /var/mail/virtual/xandros exists

You can check if the service is responding:
telnet 127.0.0.1 10024
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
220 [127.0.0.1] ESMTP amavisd-new service ready
Then just tail /var/log/mail.log for any problems.

Restrict firewall

Check what the firewall rules are.
vi /etc/shorewall/rules
Refer to the firewall settings.
Restrict to just SSH access for now.

Change passwords

Next the passwords needs to be changed.
For both the system and mysql.

System passwords

Check which users are defined on the system.
cat /etc/passwd
Apart from all the system ones,
there should probably be none (if EC2 AMI) or
just your user if it is a standard Ubuntu install.
If there are some users, you need to change
their passwords.

SSH Access

Next we check whom got SSH access.
If there was any users defined,
check their home folders for ssh keys.
cat /home/username/.ssh/auth*
Remove any you do not expect to be there.
Next check if and which specific users has been defined
for SSH access in
vi /etc/ssh/sshd
Usually this is fine.

MySQL passwords

First you need to change the root mysql user.
If none has been set do this
mysqladmin -u root password new_password
Otherwise do this and you will be prompted for the old password
mysqladmin -u root password new_password -p

Then the default mail user as well.
If you know the old password
mysqladmin -u mail password new_password -p
Otherwise log into mysql as root:
mysql -u root -p
Enter new root password specified above, then:
update mysql.user set password=password('apassword') where user='mail';
flush privileges;
You may need to revisit the top of MySQL section
to re-grant the mail use rights on the database.

If you do not know the old root password,
you have to restart mysql without grant rights. Google it... :)

If SASL is set up, then you need to update its passwords.
First in postfix SASL file:
sudo vi /etc/postfix/sasl/smtpd.confsql_passw: aPASSWORD
Then on both lines in:
sudo vi /etc/pam.d/smtppasswd=aPASSWORD

Check configurations

You should scan the postfix, courier, etc. configurations
to check if they match what you expect.

Set machine name

Now you need to define your machine name,
e.g. something like smtp.yourdomain.com.
You need to define it in
sudo vi /etc/mailname
And then your domain name in
sudo vi /etc/postfix/main.cf
under the mydomain setting
myorigin=yourdomain.com
It could also be smart to check what the unix hostname is specified
as
hostname
This can be reset by
sudo hostname smtp.yourdomain.com.
All though this does not have to be the same as your postfix mail server name.
You may want to speficiy some hosts in hosts file as well,
sudo vi /etc/hosts127.0.0.1 localhost.localdomain localhost
127.0.0.1 smtp.yourdomain.com smtp

Certificates

You could go along with the generated certificates
(if they are there, default for Ubuntu).
Or if you could create new ones with the correct machine name in them.
Especially if this a mail server used by many, and authenticiy is important.
Follow the TLS certificate instructions for
Postfix and Courier.

Some of these sections can be brief as they
are not core to this howto.

Remote MX mail backup

With MX backup loosing emails are unlikely.

Normally if someone sends an email destined for you,
their server will try and connect to your server.
If it can't reach your server for whatever reason
( it is down, dns issues, there is network problems, or just too busy ),
the other server will back off and try again in a bit.
How many and for how long it will try again is determined
by the sending server. Some of them are not very patience,
and it will report undelivered after only a few attempts.
So you would have lost that email.

If you had specified a backup MX,
this email may not have been lost.
Upon first failure to connect to your server,
the sender would see if there is any alternative server
to send to. So it connects to your backup mx server.
This server spools and queues your message
and will try at intervals to send the message to you.
It too will though eventually give up.

What is the difference?
Simple, you (or whoever controls the backup mx )
is in control how long and often to try connecting
to your machine.
So if you have a reasonable values and your server
is not down for weeks, no mail is lost.

How to implement it?
First edit the DNS records again,
and add a backup mx with a higher value.

You may choose to have this as the last line in the file,
as you may use small cron jobs to modify this ip address,
if you don't have a permanent static address.
It should contain your IP addres, hence if you do not
have a very static IP address, that you need to
automatic editing if the postfix file.

You noticed I added a transport lookup.
This is a field in both the domain and the backup tables.
In domains it is used to determine how to deliver
the email, ie either virtual (correct) or local
(not used in this howto).
When backing up servers, your also need to specify
in the transport field how to connect to the correct servers.

Say you are backiup for a friends server, mail.friend.com,
for the domains of friend1.com and friend2.com.
So you should insert this into your backup table.

The :[] tells to connect directly to this server,
not doing any more look ups for valid MX servers.

This shouls now work fine.
Further tweaking of the queue values,
review these and modify as appropiate.
Shorter warning times are good for the sender,
so that they realise the email has not arrived yet,
but may also be annoying. Tradeoffs..
Look in the first main.cf configurations
for ways to do so.

Relay recipient lookup

Unfortunetly spammers are using backup mx as a way to saturate the networks with invalid emails,
known as backscatter mail.

They simply lookup a domain's MX servers and connect directly to one of the lower priority servers
whom may be just a backup mx. This server if configured as above will not check for valid addresses aliases
but will accept and queue all emails for the domain's it is configured as a backup mx for.
These will then be delivered by the server later to the primary MX server,
whom will then maybe reject them as the aliases are not valid. However the sender addresses
are often invalid and a long trail of reject messages to and forth around the net follows...

If the relay_recipient_maps setting is set, then postfix will reject
all email addresses not specificed in this table.
As with many postfix lookups, it will recursively search for a match from the full address.

In the following examples, emails to john@example.com
are the only emails that will be accepted for the whole example.com domain.
However for @example.org all emails will be accepted for backup,
except any for support@example.org which will be rejected.
insert into relays (recipient,status) values
('john@example.com','OK'),
('support@example.org','REJECT'),
('@example.org','OK');

Local file backup

Here is rough backup script to backup your configurations
and mail folders.
You may want to backup the folders seperatly
as they can quickly grow to GBs.
Adding this to a cronjob automates this process.
Be aware that you should
stop postfix and courier while backing up the
mail folders. And that if they have grown large,
that this may take some time.

I list the mail servers and websites associated with this domain (the a and mx bit).

I then specifically list the name of a server I may send mail from applications automatically using addresses within this domain.

As you can see I also use Google Apps with this domain, thus tell SPF to also allow all mail servers associated with google mail.

Then for most of the other domains I would use this DNS TXT field:

"v=spf1 a mx include:example.com ~all"

The important elements are:

I list the mail servers and websites associated with this domain

Then I tell SPF to also allow all mail servers associated with my main domain (example.com).

And for all these I use ~all!

Ps. Some domains I have added an even stricter SPF, as these are domains that will never send an email.

SPF problem

It is worth noting about SPF, that you should leave the decision to whether to reject or allow
the email to the mail servers. Therefore using -all instead of ~all is not a good choice.
Leave it to the SPAM scoring by the receiving server, like SpamAssasin does it.
You then minimise the risk of false positives.

One of the reason I do discourage -all use, is that SPF has a distinct problem:It does not like email forwarding or use of backup MX!

Consider this: Your address of lulu@hoopa.com sends a joke email to a few friends.
One of these is trixie@bellbell.org.
Trixie's email address is actually an alias and forwards the email to her private webmail account on hotmailnot.com.

Now if your domain, hoopa.com, have a strict SPF set up, which only allows emails to be sent by its mail server.
And you/the mail admin has added -all to the SPF, which tells other server to reject emails not from your server.
This you think makes sense, spammers can not use your domain for spoof emails.

So what happens: bellbell.org receives the email from lulu, and possible checks the SPF, which is OK, and forwards it on to hotmailnot.com.
However if hotmailnot.com also checks SPF, it will receive the email from bellbell.org, check the SPF to see bellbell.org's mail server is allowed to send emails on behalf hoopa.com. SPF will say No!, and with the -all, hotmailnot.com email server will reject the email!

2nd scenario if lulu email trixie directly at hotmailnot.com,
but hotmailnot.com main mail server was down, and email was sent to the backup mx server.
When the main server came online again, and the backup spooled the email back to it,
the SPF would again fail as the hoopa.com's SPF would not mention hotmailnot.com backup mx as an allowed mail server.

Solution:
Of course you can not list all possible forwarding / backup mx email server that your domain's users might at some point email!
I simple just use the ~all option. Which simple say it is not the expected server, but probably ok.
And if this is added to a scoring by the receiver, then the accumulated spam score might be enough to reject dodgy emails.

White/Black Lists

todo

You can implement further lists inside Postfix or SpamAssassin.
Amavisd-new already has a few well known white/black listed items
in its config files.
SpamAssissin also as a feture to automaticly learn white lists.

PGP & S/MIME

This is not implemented on the postfix server side,
as this totally a client side option.

However SquirrelMail has a GnuPG option.
It is a plugin that can be downloaded from their website.
Which can then be enabled via SquirrelMail's
config script.

Here is how to create a GnuPG key pair.

# check you have not already got a key
gpg --list-keys
# then create one
gpg --gen-key

To import GnuPG into Evolution;
in your settings/preferences
edit your account settings and
add you private key under the security tab.
The private key is found via listing the GnuPG
keys as above, then it is the 8 characters
after the "sub 1024g/" bit of you key.

Throttle Output

todo

For some users with restrictions on bandwidth,
you may wish to control how much mail is sendt out.
Postfix has long refused to implement these features,
out of ideolocial beliefs that mail servers should
not be restricted.
However there are some ways around this.
More to come later.

Why

Some already have their domain's email hosted with Google.

Some people prefer Google's web based interface.

Temporary Migrations.

Include Google's security features on top of yours.

How

Options

The easiest and simples solution is not to have a domain MXed to your server,
and simply alias email to those domains.
eg All email to joeblogs.co.uk hosted on your server
are forwarded to joeblogs.com hosted with google.

You may set up your own server to simple be a mail server backup (mx)
for a domain hosted with google.
If you are the first priority in the MX details of the DNS, you still
have some control, but not all will obey the priority listing.
E.g. spammers, but some valid senders as well.

However the one I use and the option where you are most in control
is to keep you server as the only MX server in the DNS.
And only forward certain aliases onto Google after all your servers checks.
Other aliases and user can just use your mail server if you prefer.
I will explain how to do this in the next steps.

DNS

You only put your mail server as the mx for the domain in question.
Google will complain about this, as it will not be able to verify that
email is setup correctly. Ignore this as it will still accept emails.

MySQL tables

You setup you aliases as normal.
However you domain table needs tweaking.
This is because otherwise your server
will just forward the email to itself.
You can actually specify aliases in the domain table.

Example

If for example:joe@bloggs.com wants to use gmail. mary@bloggs.com does not.

Assuming there are no bloggs.com data in any tables (domain,alias,users,relays,backups):
insert into domains (domain,transport) values
('joe@bloggs.com','smtp:[aspmx.l.google.com]:587'),
('bloggs.com','virtual:');
insert into aliases (mail,destination) values
('joe@bloggs.com','joe@bloggs.com'),
('mary@bloggs.com','mary@bloggs.com');
insert into users (id,name,maildir,crypt) values
('mary@bloggs.com','mary','bloggs.com/mary',encrypt('maryspassword') );
The domains insert is the interesting one.
The transport map lookup checks recursively for an alias match and will first look
for user@domain before it looks at the general bloggs.com for which transport to use.
The square brackets around aspmx.l.google.com indicates that this server
will not lookup for mx settings for this domains DNS, but instead connect directly.
(This can avoid never ending recursive lookups/relays)

Note if you have backup mx configured and chosen to enable relay recipient lookup to avoid backscatter mail spam,
then you need to add your Google Apps users to the relays table:
insert into relays (recipient,status) values ('joe@bloggs.com','OK');
Refer to backup mx section for creation of this table.

TLS certificate

If you have set your server up to prefer TLS
then you should add Google's signing authority to your server's root certificate list.
Google used to use Thawte but now use Equifax.

Issues

There are some items you should consider when integrating Google Apps.

Privacy
First there is the privacy issue.
This is the same as if you were using Google Apps only or GMail.
Google can and will read your email.
However probably not a person, but they will use it for commercial reasons,
E.g. showing relevant ads.
Some people really hate this part and refuse to use Google's mail products.
However I trust them a little bit, and do use it.

Spam
If you forward spam, then consider your own servers reputation.
Should be okay though.

SPF
If you use SPF for your domain,
consider that both your server and google will receive and send mail
on behalf of that domin.
Adding include:_spf.google.com should cover it.

Google internally
Be aware Google think they host you domain.
So if others inside google, or using google hosted apps or GMail,
if they email you, the email may not go via your email server,
but directly to the Google Apps for your domain.
That could be an issue if not all aliases you have use Google Apps.
This needs to be tested more though.
Especially as it may only be an issue if Google's servers are part
of you domains MXs.
It may be worth adding aliases in your Google Apps admin
for the non google apps adresses
to some user whom can handle these?

Roundcube webmail client

As an alternative to SquirrelMail,
Roundcube has a more modern feel to it.
It is however not got the long testing track record of SquirrelMail,
and is not yet a 1.0 release version.
However if you prefer its very much more appealing interface then
follow these easy steps:

You may optionally want to uninstall SquirrelMail:

sudo aptitude remove squirrelmail squirrelmail-locales

Then install Roundcube

sudo aptitude install roundcube roundcube-mysql

It will ask you if you want to configure its database access,
answer yes, then select mysql.
Then it will ask for the root mysql uses password,
which it will create a roundcube mysql user and ask for its desired password.

This will create a symblink in /etc/apache2/conf.d/
to /etc/roundcube/apache.conf.
Edit this file.

sudo vi /etc/roundcube/apache.conf

Depending on your setup you may want to move those Alias
commands at the top to your virtual hosts configuration,
or for this example enable them here for all hosts.

# Uncomment them to use it or adapt them to your configuration
Alias /roundcube/program/js/tiny_mce/ /usr/share/tinymce/www/
Alias /roundcube /var/lib/roundcube

Save, exit and reload Apache to enable these aliases for Roundcube to work

sudo /etc/init.d/apache2 reload

Then go to your roundcube installation depending where and how
you modified those Aliases, e.g. at
http://mail.example.com/roundcube.
That should be it

You can obviously modify and tweak further.
One thing that may be usefull is to have
the Roundcube Apache Alias on different virtual hosts,
and configure username_domain in main.inc.php
to append different email addresses, or
configure the default_host to
different mail server depending on virtual host...
More details on the Roundcube Wiki.

fail2ban

Suggestions?

If you have any suggestions to other ways of extending
a postfix server, then fire off a mail to me via
the contact form further down.
(Or rather, Id prefer that you write down the extension, and let me know the link! :))

Elastic Compute Cloud

Impressions

Easy to use.
Anyone can use, not just big companies.
Very useful.
Tools are command line but simple.
Firefox extensions work well.
Recommended.

I find it very usefull.
Basically it is a colo hosting environment.
Some may use it as for Saas, ie single scalable application in the cloud,
but I use it as a hosting environment for complete servers.

ec2 introduction, tips and howtos

How I use it for my mail servers

Different images to launch for different needs.
Good way to scale backup MXs if needed.
Can script backup to S3 of mail dirs etc.

Using EC2 with this howto

If you plan to use EC2 to follow this howto,
then familiarise yourself with EC2 first.
Check the links further down,
e.g. my tips>.

Once competent enough on EC2, launch the latest official
Ubuntu ec2 image or one of
Eric Hammond images.
You can cheat by using my other images,
but you should really know how the whole server was built by starting
from the bottom.

When using EC2 images, be aware of security groups as they restricts
access to your server on top of the firewall.
Initially you will need SSH (22) access, quite soon you will need SMTP
and IMAP ports opened, 25,143,465,587 and 993, and
eventually webserver ports of 80 and 443.
Read here for tips on securing AMIs.

Also do not terminate your instances without backing up your machine.
This you can do by either create your own image.
Or backup certain data if you got an image to instantiate from.
Back up to S3 or your local machine.
Create images only now and then. Backup configurations, database, maildirs
more regularily.

1st note:
Spamhaus.org lists amazons ec2 ip ranges as dynamic,
thus many mail servers will reject emails from it.
(Including other people using this howto.)
But Spamhaus has a simple web page to remove ips,
which they link to in rejection messages.
Simple look in your logs, click on the link on follow the instrucions:
basically fill in your ip, email and state its for a mail server.
Then Spamhaus will remove your IP from their database.

Appendix

About author

Ivar Abrahamsen, an IT Senior Consultant from Norway.
Specialising in developing and integrating middleware application systems.
Mainly open source and Java based technology stack.
Recently moved back to UK (Hampshire) after 5 years in Oslo and before that 15 years in Manchester.

So if you have any questions,
problems with using this guide,
or any other issues with this guide,
please use the forum.
Then it is also an available archive for others to find solutions in the future.

If you find any spelling mistakes or broken links, please let me know.
Even after 6-7 years since the first version people are
still finding typos every week.
So my initial versions must have contained only typos! :0

7 years after the initial version of this document I think roughly
95% of the people that do contact me directly,
have not bothered to test properly
(command line first, not straight to GUI...)
nor read the whole guide and common solutions &
forums.

They just want me do to their work for them. For free.

The chances of me replying to their emails are miniscule,
I much prefer to spend time with my daughter instead :)

However many people contribute with really good stuff in the forums,
send me typos(frequently..), or thank you notes.
You guys are great!

Why

Why your own mail server

Main reason: Because you can.
Other good reasons:
Basically it leaves you in complete control,
to expand, customize and tweak your mail server to your needs.
You are not dependant on 3rd party providers,
limited by their technology contraints or your budgets.
With your own mail server you can add as many aliases,
users and domain as you'd like,
be as restrictive or open about security, virus, spam,
file sizes etc as you prefer.
And is it is well known, frequently updated, open source application stack,
you can also trust the software you use.

Why I wrote this howto

When I set up my first email server I used a mix
of other howtos on the net.
And they were so helpfull that I though I would
contribute back with my experience.
And it has been useful as a recipe script for
myself every time I need to install/update a server.

A less angelic reason is that back in 2003 I was setting up mail server
for a few friends and collegues.
Soon I was getting more request, and being a lazy programmer, I thought..
"Why don't I write a howto and let them do it themselves..."
Soon it was listed on postfix.org
and I was getting thousends of hits and
lots of emails. (blessing in disguise)

Todo

FAQ

But please, most of the frequent questions have been asked and answered in the forums.
Most are also unneccessary as following the test section will have solved them.

Some question that frequently get sent to me, which first of all should have been asked in the forums and has been answered there many times, which then I tend to ignore are:

Squirrelmail does not allow me to log in

This is due to many things.
Most are due to skipping too fast forward, ignoring test sections etc.

Answers:

Does postfix work?
No point trying to run before you can crawl.
Send emails to recipients on your server,
tail mail.log to see if everything is okay.
Often mysql is not configured properly, check the mysql logs for activity.

Have they ever received an email?
If not they can not log into squirrelmail
as the email folders will not yet exist.

Does Courier work?
If it doesn't then you have still got some more setup to do.

If all above is okay, then it may be a problem with your Squirrelmail setup.
Check empty spaces in squirrelmail mysql setup. More details in test section.

Email folders do not exist

Mentioned many times in this guide and forums.

Answers:

Have they received an email?
If not they you can not log into squirrelmail
as the email folders will not yet exist.
When receiving their first email,
postfix will create all the neccessary folders.
If it does not your postfix setup is broken.

There is a program that creates the folders for you.
I do not recommend it,
as basically your postfix setup is broken
if no folders are created, and you better fix it instead.

SASL authentication does not work

All lot of people have issues with SASL in certain setups.
There are quite a few messages in the forum is regarding this.

SASL works for me, but I can not tell my configuration apart
from other people's server where it does not.

Workarounds or alternatives:

Do you need external SMTP & IMAP access?
Or will webmail interface be sufficient for external dynamic ip clients?
Then you do not need SASL. Restrict by IP and TLS is enough.

Do you need SASL at all?
If you accept TLS only and restrict by IP addresses
in mynetworks it may not be neccessary.
Not an option with
road warriors and too random dynamic IPs unfortunately.