Setting Up The Mail Service in Mountain Lion Server

Mail is one of the hardest services to manage. Actually, mail is pretty simple in and of itself: there’s protocols people use to access their mail (such as IMAP and POP), protocols used to communicate between mail servers and send mail (SMTP, SMTPS) and then there’s a database of mail and user information. In Mount Lion Server, all of these are represented by a single ON button, so it really couldn’t be easier. But then there’s the ecoysystem and the evil spammers.
As a systems administrator of a large number of mail servers, I firmly believe that there is a special kind of hell where only spam is served at every meal for spammers. Here, the evil spammers must also read every piece of spam ever sent for eternity. By the end (aka Ragnarok), they should be fairly well hung, have chemically induced stamina of a 16 year old with the latest Sports Illustrated Swimsuit issue, enough pills of other types to not be able to use that stamina, plenty of African princes looking to donate large sums of money if only they can be helped out of their country (which should cost about 100,000 compared to a 5,000,000 payout, not a bad ROI, right?!?!?), have their conflicting stamina situation at the top of the search engines and of course, have lost all of the money made from their African princes due to getting their credit card hijacked by about 9,000 phishing scams. All in all, a special kind of hell…

But back to the point of the article, setting up mail… The things that mail administrators need to focus on to keep that mail server flowing mail to and from everyone else in the world:

Port Forwards. Port forwards need to be configured on the gateway for the SMTP port at a minimum and more than likely other ports used to access mail on client devices (25, 143, etc)

DNS records. An MX record and some kind of mail.domain.com type of record should definitely be configured for the DNS servers that are authoritative for the domain. There should also be reverse records for the address of the server, usually created by the Internet Services Provider, or ISP, that match that record.

Check the RBLs. If you have a new IP address you’ll be putting a DNS server on, check all the major Realtime BlackLists to make sure that some evil spammer hasn’t squatted on the IP before you got to it. This is true whether you’re in a colo, hosted on an IP you own or moving into space formerly occupied by a very standup company. A lot of IP addresses are blocked, as are blocks of IPs, so before moving mail to an IP, check it.

Mail filtration (message hygiene). OS X Server has a number of mail filters built in, including clam for viruses, the ability to leverage RBLs, block specific addresses and of course RBL checking. However, this is often not enough. Third party services such as MXLogic help to keep mail from coming into your network. You also end up with an external IP to send mail that can cache mail in the event the server is down and keep mail off your network in the event that it’s spam.

Backup. I am firmly of the belief that I’d rather not have data than not have that data backed up…

Once all of that is taken care of (I’ll add more as I think about it) then it’s time to enable the mail service. Actually, first let’s setup our SSL certificates. To do so, open the Server app and click on the name of the server in the HARDWARE section of the sidebar. Then click on the Settings tab and then the Edit button beside the SSL Certificate entry. Here, use the Certificate drop-down list for each protocol to select the appropriate certificate to be used for the service.
Click OK when they’re all configure. Now let’s enable the mail service (or outsource mail). To do so, open the Server app and click on Mail in the SERVICES list in the sidebar.
At the configuration screen is a sparse number of settings:

Provide mail for: Configures all of the domains the mail server will listen for mail for. Each account on the server has a short name and each domain name will be available for each short name. For example, an account with a shortname of charles will be available for email addresses of charles@pretendco.com and charles@krypted.com per the Domain Name listing below.

Authentication: Click Edit for a list of sources that accounts can authenticate against (e.g. Active Directory, Open Directory, Custom, Local, etc) and in some cases the specific password algorithms used for mail.

Relay outgoing mail through ISP: Provide a server that all mail will get routed through from the server. For example, this might be an account with your Internet Services Provider (ISP), an account on an appliance that you own (such as a Barracuda) or with an external filtering service (such as MXLogic).

Limit mail to: Configure the total amount of mail a user can have in the mail store, in Megabytes.

Edit Filtering Settings: Configure antivirus, spam assassin and junk mail filters. The “Enable virus filtering” checkbox enables clam. The “Enable blacklist filtering” checks the RBL (or RBLs) of your choice to check whether a given server is a “known” spammer and the “Enable junk mail filtering” option enables spam assassin on the host, configuring it to block based on a score as selected using the slider.

The client side of the mail service is straight forward enough. If you are wondering where in this article we discuss using webmail, er, that’s not installed by default any longer. But the open source project previously used, roundcube, is still available for download and easily installed (the pre-reqs are all there, already). Check out the roundcube wiki installation page here for more info on that. Also, mail groups. I hope to have a post about that soon enough. Unless, of course, I get sidetracked with having a life. Which is arguably not very likely…

Thanks for taking the time to maintain this blog. Your documentation is superior to Apple’s “advanced administration guide.”

I wonder if you would comment on the best way to implement out-of-office replies that end users could set on their own.

I’m new to OSX Server, so I’m looking for an easy method to accomplish this. I thought I might use Roundcube to add webmail, however there does not seem to be support for away messages. Another webmail client that is designed to work with Postfix and includes away message capability is Modoboa, but I am unsure if OSX Mountain Lion Server natively meets all the prerequisites.

As of right now, there’s no way to get Out of Office replies on an Apple server from the client. I think it’s funny that it works in Lion and it works for Mountain Lion clients on Exchange servers but doesn’t work from a Mountain Lion client to a Mountain Lion Server right now…

Scott Aubrey

In our organisation, we recreate an Out Of Office experience through a web-based MANAGESIEVE client since the days of 10.4. This authenticates to the mail server as a mail admin user, and masquerading as the users logged in to the webservice. We then parse and output the sieve scripts, and upload using MANAGESIEVE.

This does work very well, especially since Lion server’s upgrade allow date ranges, but setting this up to work across the different version of OS X mail server (migrating from Cyrus to Dovecot) and apple’s coming and going support of this feature has been trying, and in almost all version required going behind Apple’s admin tools and manually editing cyrus/dovecot configuration files. We also haven’t tried Mountain Lion server yet.

Knowing the open source tools behind the setup is invaluable if you’re wanting to go this route.

Dovecot is the IMAP server
Sieve is the inbox processing scripting language
MANAGESIEVE is the protocol to upload and remove the scripts.

As with Lion, you need to add your email address to the list of additional recipients. Dovecot under Mountain Lion 10.8 OS X Server has sieve still enabled and it’s just a matter of matching the config for Roundcube and Managesieve. It works well with Postgres. I’ve found your articles on Lion really useful. It’s only taken a couple of weeks to get everything up and running nicely under ML.

Thanks for the post, Gerry. That’s exactly what I needed. I’m going to attempt to follow your instructions on Roundcube/manageseieve this morning.

Brian

Thank you for this great information – but the piece I’m really hungry to find is this: what file do I modify to configure SpamAssassin? I’ve got some rules that worked just fine under Lion Server, but in Mountain Lion Server I’m putting them in /Library/Server/Mail/Config/spamassassin/local.cf and they don’t seem to have any effect.

Thanks for maintaining this excellent blog! Two weeks into Mountain Lion Server, and I’ve found you to be the best source of information on the net. Please consider authoring a book on ML Server – I’d buy it up in a heartbeat. The information you’ve graciously shared here has left me hungry for more. I could really use an itemized explanation of every setting available from the command line.

Charles, thanks again for this great site and your posts. One thing Im struggling with a bit is the loss of log file settings (debug, etc.) as a plan my migration. (I have a parallel issue where log files seem to be blank after the first turn over, but different concern.)

How can I reliably dial up the log level from the command line by service?

Is there a way to specify the auth method for the relay via serveradmin or otherwise?

Do we need to specify submission and or ssmtp ports along with the relay host? e.g. relay.domain.net:465

Brad Tombaugh

I have an issue with relaying mail through an ISP… That’s part of the reason that I’m running my own mail server. I’m very disappointed that Apple keeps removing configurable settings from the admin GUI, dropping the Server Admin Tools, and not updating their documentation!

To get postfix to be able to send outgoing mail to external clients directly, instead of by relay through an ISP, you only have to add your local subnet, like 192.168.1.0/8, to the “mynetworks” parameter in the postfix “main.cf” config file.

However, Apple has screwed this up to! Normally, you would expect to either use the “postconf -e” command, or editing the file /etc/postfix/main.cf. However, in Mtn Lion, Apple isn’t using the files in the default location, they are in /Library/Server/Mail/Config/postfix!

You can either edit the main.cf file in that directory, or use the -c option to specify the config directory, like postconf -c /Library/Server/Mail/Config/postfix …

Remember to use ‘sudo’ with either of those commands, then do ‘sudo postfix reload’ to make your changes active.

Nick C

Can you address how best to use ML Server with Outlook for Windows as a client? Secure authentication seems to be the issue – Outlook does not (in MS’ perverse way) support any of the secure authentication methods that Macintosh servers have ever supported (such as Kerberos, CRAM-MD5 or even APOP).

I have tried using SSL with “Cleartext” in the past but have never managed to get it to work using a self-signed SSL certificate.

This whole issue is certainly one that I am sure many admins need to address as Outlook is actually a pretty good client apart from its lack of support for non-MS authentication methods.

* Just to be clear, note that using the “plain” or “login” authentication mechanisms result in user/password credentials being sent in CLEAR TEXT. This is NOT a concern AS LONG AS the connection stream is encrypted with TLS/SSL, as the information will be encrypted in that stream to begin with. Out of the box, OS X disallows plain text authentications unless over TLS/SSL connection.

Is there any reason you can’t use an SSL cert? All of my Outlook clients are authenticating perfectly using plain text/SSL cert combination.

Nick C

When I’ve tried this it just didn’t work, but I will try again when I have some spare time to fiddle with it (=never!).

That’s the problem – one of the generally good things about OS X Server is that as long as you stay in the space that the Apple tools support it is straightforward enough (and great for people like me who are not dedicated sysadmins but have a hectic other job to do as well) but once you go beyond that space it comes down to trial and error and hearsay.

On many occasions I have blown a valuable half day of time trying to get something slightly non-standard to work and eventually sometimes I crack it and sometimes I just have to give up.

If there is a simple fix to (for example) the SMTP AUTH issue, why don’t Apple provide a checkbox for it (“Compatibility with Outlook 2003” or whatever)?

One can’t help but feel that sometimes Apple doesn’t live in the real real world…

I was unaware that Outlook would have a problem with self-signed SSL cert. If you have any kind of budget to solve the problem, you can get wicked cheap SSL certs here: https://www.rapidsslonline.com

I spent $18 for 1 year and it works just fine with Outlook. Now if someone would just write an add-on to allow caldav/carddav support…

Nick C

It sounds simple when you say it quickly but I have purchased an SSL cert from rapidsslonline but it is as clear as mud how to install it when now that I have got the cert.

I have generated the CSR OK in Lion server (I think – hope I got the attributes right!) and pasted it into the field on the webpage, but I now have an email containing a “Web Server Certificate” and an “Intermediate CA” and no clue what to do with them.

Any assistance would be greatly appreciated because I’m stuck at this point and the rapidsslonline help pages are no help at all because they cover every other server type than MacOSX Server…

I wish I had documented this. It was my first time installing an SSL cert in OSX, and I messed it up the first time. From memory, here’s how that went for me. Being the speed reader that I am, I totally overlooked the intermediate bit. I installed the cert on Lion without the intermediate, then upgraded to Mountain Lion. From there I couldn’t figure out how to fix it. Since I had cloned the system with Carbon Copy Cloner prior to installing the cert, I rolled back, then repeated the cert install. As I recall, installation involved dragging and dropping the cert files into the dialogue box (where it indicates to replace private cert). What I remember is that you have to drag both cert files you downloaded (one at a time) to install it properly. Do this before you click the button to apply. Once that’s done, web browsers will recognize the cert properly.

Does anyone here know how to change the port # on the webmail service in Lion Server? Also, since upgrading from 10.6 to 10.7 there is no longer option for forwarding email for a user. I understand that was added back in 10.8, but since 10.8 has no webmail I won’t be upgrading to it on our mail server any time soon. Any ideas?