Exchange 2007 Content FIlter: The Whitelist Is Here!

Messaging Hygiene features in Exchange Server 2003, including the Intelligent Message Filter (IMF), did not have a way to whitelist sending domains or SMTP addresses.

This is a follow up to a previous post, and one of the more popular ones on this blog— “IMF: Where’s the whitelist?“. (“IMF and whitelist” has for long been one of the most common search terms on this blog – Bharat).

Whitelists are common in most 3rd-party anti-spam tools. Adding domains or SMTP addresses of important senders like customers, vendors, or your CEO’s home email address (almost always an AOL address… :) for instance, ensures messages from these domains or addresses do not get filtered by the anti-spam filter.

Bypassed Senders and Sender Domains: The Whitelist

The good news is— Exchange Server 2007’s shiny new Content Filter Agent (or IMF v3 if you will) has whitelists! You can add SMTP addresses and domains to the Content Filter configuration, and have messages from these senders and domains bypass the Content Filter Agent. However, you need to resort to the Exchange shell (EMS) to manage it.

Use the following command o add sender SMTP addresses to the BypassedSenders list:

Some whitelisting considerations

Before you start using whitelists, here are a few things you should consider:

SMTP headers can be spoofed easily. If spammers spoof any of the addresses or domains you whitelist, your recipients may end up getting more spam as all of it will bypass the Content Filter.

Use SenderID Filtering to detect and protect your mail system from header spoofing.

Maintaining whitelists, just as maintaining blacklists, is a manual process that imposes its own management costs.

Checking every inbound message against a list of whitelisted recipients imposes a performance penalty – miniscule as it may be. Use the whitelists sparingly.

Nevertheless, many IMF users have repeatedly demanded this functionality and it’s great to finally have it in what some folks call IMF v3.0.

Bypassed Recipients: The Exception List

The Content Filter can also be configured with an exception list – to not apply the filter to inbound messages for particular recipients. This can be done from the console by going to Hub Transport | Anti-spam tab | Content Filtering -> properties | Exceptions. This list is limited to a 100 recipients – you can add generic recipients that you want to exempt from the Content Filter, such as [email protected], [email protected], etc.

- When you add senders to the Safe Senders list in Microsoft Outlook, Exchange doesn’t know about it in real time or by itself. You have to enable Safelist Aggregation.

- Yes, some configuration can only be done from the shell (typically these are non-repetitive tasks e.g. at transport server/connector/Org level).

- Given the number of overall options available to granularly control a whole bunch of settings, it’s probably not possible to include everything in the console UI. For instance, look at all the recipient parameters you can set with Set-Mailbox and Set-CASMailbox commands.

- There’s no denying Exchange Server 2007, as released (RTM), has some rough edges, but the issues you’ve raised have been addressed above. There’s plenty of documentation on TechNet and other resources (including this blog) to help you navigate through this new version.

- Service Pack 1 is just around the corner, which should take care of a many issues.

- If you have more such specific issues please feel free to post here. I will be happy to respond. You can also pass on feedback directly to Microsoft.

My application may help some people. I haven’t tested it with Exchange 2007 but it works with 2003. It’s still in early stages of development and looks basic but it was only intended as an internal program for my own use. Having said that, I understand how annoying it is not being able to whitelist sender addresses easily.

The trouble with Microsoft’s anti-spam solution is that it still lies in the administrators hands to manually look for the 1% of emails that are actually legitimate, in the vast sea of junk that is out there. In Exchange 2007, Microsoft has further complicated matters by putting this junk mail into an email mailbox! At least in Exchange 2003 IMF they stored it in an EML format on the gateway…

For example, because of spending 50%-60% of my day sifting through junk to catch that small percentage, I developed a Windows service using .NET 2.0 which watches the directory in which IMF puts the archived “SPAM” messages. When a message came in it opens the EML file, logs certain header information into a database (Access or SQL/SQL Express), and twice per day sends a report to all users with a clickable link to “release” those emails. Furthermore, it contains a “whitelist” AND blacklist feature that can auto-release/delete by IP, sender, receiver, SCL rating, etc. The benefit here is that users don’t have to sift through hundreds of SPAM messages rated 6 or higher (my gateway is set at 5, and user-level junk at 4) and yet not miss potentially valid email. It’s completely eliminated my SPAM administrative workload. It’s entirely up to the end-user to sift through his/her own crap and if a legit email does come through, they can release it AND create a “server-side” rule to allow it so it is never caught again. And it also cleans up after itself, never having more than x days/months stored on the server. The last part is that it’s smart; tracking those troublesome IP addresses that the RBL doesn’t catch…

It may seem to be a good idea to store the archived crap within a single mailbox, but it’s taken third party programs (such as mine)which simply had to read a ASCII EML file to now have to have an Outlook client OR use IMAP/POP3 to “fetch” the mail – further fattening up the client (my service is a 48kb executable). By chosing to store their email in a mailbox, the man-hours I’ve spent are for naught, and ensured that I won’t upgrade for a few more years as I refuse to subscribe/purchase a anti-SPAM service/product that is already provided free from Microsoft…

If you’re interested in this program (called UCEArchive), send me a message – my display name AT terminalit.com. It’s helped me out a lot.

Anyone have any idea how to list or view all the entries in the whitelist from the management shell or elsewhere? I can live with having to add them from the management shell (can hopefully script this someway to make it easy to do so remotely), but I would like to be able to view the list as well… and also how do you remove entries from the list? hmmm…

This is the issue I am having. The Exchange 2007 program only remembers the last entry in the whitelist. Can this be possible? Can anyone give me an easy way, or exact command line to Add more emails in the Powershell, without deleting the last entry?

Just worked out a couple minor tweaks to some of the script tactics discussed here and thought it might be handy for others, so posting it. This script will prompt for an SMTP address and append it to the current sender white list:

I appreciate the generosity of those providing scripts, etc, but these commands are really obtuse. MS really needs to continue to develop the GUI, and stop trying to push the command shell as a feature.

Anonymous, I am sooo with you about not having a GUI for the whitelist. I don’t need to do much on our company’s Exchange box, but editing the white lists is BY FAR the most common thing I have to do. It’s almost patently ridiculous not to have it. I’ve managed to screw up our lists twice in the last year despite doing all I can to enter in the correct info. It’s very frustrating. Thank God our consultant is nice enough to do it for me. It can’t possibly be that hard or troubling to come up with something graphical.

Yeah, this will work for server side junk filtering, but what about outlook junk mail filtering? I already had a transport rule set up to set the SCL (Spam Confidence Level) to 0 and outlook still put a SCL=0 message into the junk e-mail folder! Doh…..

In the outlook12 adm templates, I found a setting "Specify path to Safe Senders list". I pointed it to a text file I created (with entries on each line) at \\domain\netlogon\safesenders.txt. This is not all you need to do though. I also had to set the following two registry keys:[HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Outlook\Options\Mail]"JunkMailImportLists"=dword:00000001"JunkMailImportAppend"=dword:00000001

I created a custom adm template and set these two entries to Enabled as well as the specify path to safe senders list setting. This enabled me to whitelist email domains which I needed to exclude from client junk mail filtering by specifying them on a line in the text file in the form of "@domain.com". Unfortunately this will let actual spoofed spam through but in my organization this is more acceptable than the false positives on what they consider to be local email (when legitimate mail "from" our domain comes in from the outside – popular at higher education institutions).

@deb: If you mean a way to determine what part of an email causes a message to have a particular SCL score? I'm afraid not.

However, you can determine why your internal mail is being scanned.1. Is mail submitted by authenticated senders? If yes, this isn't scanned by default. Check content filter config it it's been accidentally configured to scan authenticated mail.2. If mail is being submitted by a trusted internal host such as an application server or copier/scanner, you can create a Receive Connector scoped to that host's IP address and bypass antispam.3. Any hosts that handle inbound internet mail before Exchange must be added to internal SMTP servers list. See Exchange Server 2007: Making SenderID work with non-Exchange smtp hosts and Telling Exchange about (non-Exchange) SMTP servers

I too have had enough of Exchange 2007. It completely sucks to have to look up oscure CLI commands for mundane tasks. If I wanted that I would get Linux box. MS's strategy seems clear to me; get rid of company Exchange admins and local Exchange servers and start using MS online service.If Exchange doesn't get it's act together our company will go to an online service but I will do everything in my power to make sure it is not MS.Google is loking like a good option…

@Anonymous from 3/3: No, it's not necessary to install anti-spam agents on Hub Transport if you have an Edge Transport server deployed (or if you're using a third-party anti-spam product/service). If you want to filter spam on the Hub using Exchange's built-in anti-spam features, you'll need to install the anti-spam agents.

Does anyone know if this whitelisting (in the Content Filter) works when you are using Connection Filtering? We wish to whitelist certain email addresses even if their email server IP Address appears on a real time block list (RBL). The description at http://technet.microsoft.com/en-us/library/aa997242(EXCHG.80).aspx would indicate Content Filtering never happens if the Connection Filter rejects the message. Oddly, the reverse seems also true – that if you allow a server IP address, then no Content Filtering takes place either.

Has anyone else encountered a bypassedsenderdomains list that isn’t bypassing all of the domains in it?
I have both the domain .aweber.com and all sub domains *.aweber.com listed for example but I still keep getting some emails blocked by the content filter.
550 5.2.1 Content Filter agent quarantined this message

Did you ever find a solution for this? I am having the similar issue. I have white listed a domain and email address in that domain and I am still getting the email blocked by the DNSBL. If any Microsoft tech wishes to chime in at this point I would greatly appreciate it!

I have a spam server at the gateway and route all our smtp mail through it however domains like gmail, hotmail and yahoo get stuck in the queue viewer unless i route emai lvia a smarthost.
I’ve tried to whitelist these addresses etc but still no joy.

In fact i actually want to disable completely the spam filter on exchange and just let our spam filter on the gateway drop them.

Does outbound mail get stuck in an Exchange queue? Whitelisting doesn’t help with outbound mail. Check the event logs and SMTP logs to determine why this happens.

Here’s how you can disable antispam features on Exchange:Exchange 2007/2010: If you’re not using an Edge Transport server, antispam filters aren’t installed on Hub Transport servers. To disable, you can set the following to disabled:

It should be noted when I add an additional [email protected] the previous ones are knocked out according to the get config command. Additionally, although I’ve added a wildcat domain.com example this simply doesn’t work for me. I have to enter the specific [email protected] on Exchange 2007.

What a pain for a low level tech simply trying admin SBS2008 for my small business. Did I say shame on MS yet?

When I originally commented I seem to have clicked on the -Notify me when new
comments are added- checkbox and from now on whenever a comment is added I recieve four emails with
the exact same comment. Is there an easy method you
can remove me from that service? Many thanks!