We can deal with the pros and cons of greylisting in another question :)
– neu242Oct 9 '12 at 13:03

1

I changed the title slightly. Does it look better now?
– neu242Oct 9 '12 at 13:09

2

@MichaelHampton Uh, point of interest... what spam prevention measures do you use that don't cause the CEO to complain? Or maybe, if more appropriate, what kind of CEO do you have that isn't a spoiled, whiny $#^&*@ who'll find something to complain about in absolutely anything?
– HopelessN00bOct 9 '12 at 15:22

1

@HopelessN00b Our CEO is not like that. I wouldn't be judging someone's personality or behaviour based solely on their profession.
– darvids0nOct 10 '12 at 2:23

5 Answers
5

It is legal to use as a service provider in Germany (unlike deleting spam mails after reception)

It is simple and effective.

It adds load to the spammer and not to your receiving mailserver. So even though the spammers may make it through your greylisting, you forced their machine to work harder and thus they can send less spam in total.

It blocks almost no legit mail, unlike IP-based RBLs etc.

It introduces delays, but you can whitelist clients (sending servers) of frequent contacts and whitelist recipients that really need email with minimum delay. Remember that using a spam filter like Spamassasin directly on all your mail (without greylisting) can introduce delays to legit mail as well: Some spammer sends so many mails to your server that the spam filter gets overloaded. Thus, it will send a temporary failure (e.g. 451) to the sending server of further incoming mails. This causes the same effects as greylisting, i.e. mails get delayed, with the exception that whitelisting is not that easy. Of course, you can use a cloud spam filter that scales to whatever power the spammer has, but that may be more expensive.

Limited or no maintenance required. No blacklist that need to be updated and change over the time. No pattern-based rules that need to be updated.

But unfortunately, in my stats I see that in this year, greylisting becomes less and less effective. The amount of delayed messages really approaches the amount of greylisted messages rather fast, which means the amount of blocked spam is reducing.

In the last year (365 days), 55% of the greylisted messages made it eventually through greylisting, i.e. 45% got blocked.

mailgraph stats year

Note that this chart included a timeframe in which greylisted messages were not counted due to a configuration error of mailgraph, only delayed ones. This means this calculation overestimates delayed messages a little, in fact a little more mails got blocked.

In the last month, 64% got delayed and only 36% got blocked.

mailgraph stats month

In the last week, 75% got delayed and only 25% blocked.

mailgraph stats week

Moreover, looking at the total amount of blocked messages:
This month, greylisting blocked 4 411 messages, but Amavisd (spamassasin) blocked 22 763 messages. This means only 16% of the spam gets blocked by greylisting, all the rest by amavisd.

Moreover, more and more cloud sending providers send from a bunch of several hundred IP-addresses. They attempt each transmission attempt from another IP. Thus, greylisting may block these mails for even days. Therefore, you need to whitelist all the "good" mail providers. This introduces new maintenance effort.

I have always been a great fan of greylisting, but sadly, I see that it is becoming less and less effective, and I think that I will disable it soon, as it starts to only delay 14% of my mails unnecessarily without blocking much spam.

The missleading stats

The amount of blocked mails in my (and your) stats may also be largely misleading. Let's take one email that is coming from a big cloud mail provider (like Microsoft's *.outbound.protection.outlook.com) that is not yet whitelisted. The first attempt fails. The second and third transmission attempts come from two other servers (IPs), so it again it fails, as the triplet does not match. Now the fourth attempt comes from the first server again and succeeds. This will be counted as one delayed transmission and four greylisted messages. My calculations above would indicate that 1/4=25% greylisted messages was delayed and 3/4=75% were blocked. But in fact, not a single message was blocked.
Now we whitelist the servers of these mail providers, so they will not be greylisted anymore. What will happen is that the amount of greylisted messages will go down more than the amount of delayed messages. This means that the amount of blocked messages we calculate will go down. But it is not true that less messages were blocked.

In fact, what I did since February 2017, is adding more and more cloud mail providers to the whitelist to fight the problem of long delays due to greylisting. This may explain (partly?), why the amount of blocked mails that I calculate is going down rapidly. So maybe, I just thought all the time that greylisting is blocking lots of spam, but the amount of spam blocked was a lot less all the time, it was just calculated incorrectly.
So be careful when interpreting your stats.

+1 from me - time to re-examine my data. I agree that these bloody annoying people who bounce mail around their internal server estate, so that each attempt comes from a different server, will skew the data. I'm not sure I buy your last section, which seems to be arguing that all or most apparent benefits of greylisting are caused by over-counting inbound emails thereby.
– MadHatterAug 29 '18 at 15:44

I last looked at this quantitatively in July of this year (2012). In July, my mailserver received about 46,000 attempts to deliver mail; of those, about 1,750 returned and were permitted through by the greylisting (and passed valid sender domain, SPF and some other non-content-based tests). Of those, about another 1,500 were filtered by my content-based filtering..

Assuming that those 44,250 emails were spam (since they couldn't pass greylisting, I think that's a fair assumption), if it were not for the greylisting my content-based filtering would have had to deal with 46,000 mails instead of 1,750.

A twenty-five-fold increase in load on my content-based filtering would require me to have much beefier CPUs and more memory. That would in turn increase my monthly hosting costs, because of the extra power consumption (and, probably, the size of the server).

So in short, the last time I counted, yes, greylisting still made very, very good sense as part of a complete spam-filtering system. I have activated it for clients in the past few weeks, and all are extremely happy with the decrease in load on their content-based filtering systems also.

Edit: I note that I haven't answered the question about whether it's becoming less effective over time. When I turned it on, in late 2006, my estimate at that time was that it was filtering out about 95% of the spam. 1,750 as a proportion of 46,000 is about 4%, so my data suggest that it's not become less effective over that time period.

Exactly the kind of answer I was looking for. Thanks!
– neu242Oct 9 '12 at 11:55

3

I think it makes a lot of sense to look at this quantitatively in your particular situation. I just checked, and my mail server sees very different figures: total for August and September, 460214 5xx rejects, 12331 4xx rejects and 22665 accepts. Thus, 4.6% accepted and only 2.6% of spam (at best) blocked by greylisting. The 5xx rejects are dominated by 8.4% unknown user and >90% RBL. (And I don't even run extremely aggressive RBLs. A completely overwhelming majority of the RBL blocks are XBL.) Then again, traffic caught by RBLs never make it to greylisting.
– a CVnOct 9 '12 at 14:36

7

Interesting, but I can't make a direct comparison because I won't, as a matter of principle, use any RBL as a brightline test for receipt; I only use them as contributors to a spamassassin score. I've been on RBLs myself too often, for completely bogus reasons, to entrust the operation of my own mail to someone else's rationale. If, however, we were to assume that all those XBL rejections are from fire-and-forget botnets, then if you greylisted first as I do, you'd see comparable percentages to me.
– MadHatterOct 9 '12 at 14:47

1

Yes, I have strongly considered changing it to being only a spam score contributor and relying on greylisting, for precisely the reason that you mention. However, that does not negate the point I was making, that different servers may see very different traffic patterns and the only way to really know if greylisting is effective is to look at it from the point of view of your particular setup.
– a CVnOct 10 '12 at 12:20

1

I was going to disagree again, but really I completely agree with you. For all users, the best way to find out if it's an effective technique in your mail flow is to try it in your mail flow and measure - Michael speaks wisely!
– MadHatterOct 10 '12 at 12:25

spambots usually still don't do message queueing , but some of them just send the spam twice to every recipient with a few minutes delay to defeat greylisting. also, nowadays, spam from spambots isn't the real problem anymore, spam from compromised yahoo accounts etc is much harder to catch.

From that point of view, greylisting is not as effective as it used to be. In combination with other anti-spam techniques it can still help, for example if your domain is often in the "first batch" of spam campaigns, greylisting can help delay the message long enough for domain/ip blacklists to catch up, so if the spam would have slipped through your filters on the first connection attempt, it maybe gets detected on the second attempt.

The vast majority of the Spam delivery attempts I receive do come from Spambots. I use other techniques to discourage Spambots and most give up before they can be be greylisted. On my server Greylisting still blocks about half the senders, it processes. I do exempt senders which can be determined to be extremely likely to pass greylisting.
– BillThor Oct 10 '12 at 0:35

As a tangential issue, i don't like being in the position of having deployed a technique like greylisting without being able to measure its effectiveness. On Debian, with postfix as the MTA and postgrey as the greylisting policy engine, you can just apt-get install mailgraph to get a simple graph of accepted vs. rejected mail. Mailgraph is a bit old school and completely standalone, but it works, and its data or techniques could easily be integrated into a more complex modern monitoring system.

Get a reputation-based mail filter. Greylisting is a bit old-school and isn't a comprehensive solution. There are workarounds (from the spammer's perspective), and unpredictable mail delivery times for your users...

Either outsource the filtering to a cloud service or buy an appliance that has access to such a list and has other methods of validating spam. My recommendation is usually Barracuda for their appliance or for their cloud filtering solution. Both options have economies of scale and mature heuristics that provide a cleaner overall solution.

Looking at one of my client's Barracuda Spam Filter's report for September 2012, out of 98,457 messages, 1,623 were cut-off before even hitting the mail server because of bad recipients... 34,488 were blocked as SPAM. Only 96 questionable messages made it through. Those rated as SPAM were a combination of reputation, score, intent, three RBL's, Bayesian filtering and custom rulesets. All in one unit... All processed before hitting the relatively small mail server.

@neu242 The point is that 1). Greylisting has known-disadvantages, 2). cannot be considered a whole solution and 3). there are better ways of detecting spam as the processes have evolved over the past few years.
– ewwhiteOct 9 '12 at 12:57

4

Greylisting is of course just a part of my spam prevention toolkit. My setup is much like @MadHatter's. But since I asked specifically about greylisting, I sort of expected greylist specific answers.
– neu242Oct 9 '12 at 13:05

1

@ewwhite: actually, I don't see how it wasn't rather clear. For your reference: I did check the question history. Didn't see a change that affected this in any way.
– Jürgen A. ErhardDec 2 '12 at 1:31

2

@JürgenA.Erhard You're a little late to this. And your post is rude. Any professional spam filtering solution implemented today should not rely solely on greylisting. If you have any other concerns, see the canonical Server Fault spam question here.
– ewwhiteDec 2 '12 at 1:35