I've been looking for a suitable replacement for DSPAM, a project that was good but died and became unmaintained then totally removed from Zimbra. In recent searches I came across Rspamd which does all that DSPAM did and more, in addition, it can completely replace spamassassin - it's lighter, more configurable and more performant than spamassassin. These notes are a brief explanation of how to easily integrate rspamd into a ZCS server and disable spamassassin completely.

You should make these changes on a test server and backup any ZCS & rspamd configuration files that you are going to modify before you modify them - when you're happy that it works you can try it on a live server but again, make sure you backup any modified configuration files.

These are notes for installing rspamd on ZCS 8.7.x (my current version is 8.7.11), I use CentOS and these instructions are for that distribution and the equivalent version of RHEL although installing rspamd on Ubuntu should not be that different.

NB: Rmilter from the Rspamd project and documented on their web site is deprecated and won't be available as a separate package in future, do not use it.

[EDIT] As of today (2017-09-06) I have updated my server to the latest ZCS 8.8.3GA release and these instructions work and are valid for that version as well.

I've been running various versions of Rspamd for the last eight months and have not had any ill effects, although I only run this on my home server there has been a huge reduction in CPU usage from 15-20% to less than 5% for the same levels of inbound/outbound email. I have also removed all DNS & Protocol checks from within the ZCS configuration and the Postscreen settings have been set to their default values – this leads to rspamd processing all inbound email and also further CPU usage reductions.

The current rspamd install now uses an inbuilt milter compatible process for the mail that's sent to it, the anti-spam etc. components of rspamd for processing the mail through it's various modules and pass it back to postfix via the milter process. Do note that this configuration still uses the ZCS inbuilt amavisd & clamav for anti-virus processing.

The multiple-milter function in Postfix is broken by ZCS and only allows one milter to be functional (the bugzilla reference is 97706), although that bug seems to require a trivial fix it’s unfortunately still languishing without any attention for the past two-plus years - please add your comments and vote if you feel it should be implemented.

Install the rspamd & epel repositories, details on the rspamd wiki: https://rspamd.com/downloads.html - once that's done install rspamd & redis then modify the ZCS and rspamd config files as shown below: If you are going to use redis then a more recent version than the one in the EPEL repo should be used. Further details on the install of the REMI & Rspamd repositories are in the wiki article .

1 - Make a copy of the ZCS /opt/zimbra/common/conf/main.cf & /opt/zimbra/conf/zmconfigd.cf files (just in case).

2 - We need to set the zimbraMtaSmtpdMilters parameter in LDAP (this change will survive upgrades):

The antispam service should not now be shown in the list of enabled services.

The LDAP entry for smtpd_milters that we modified earlier survives an upgrade but the following one using postconf does not survive an upgrade nor do the changes to the zmconfigd.cf file (from item 3 above) and they need to be set after after every upgrade. The postconf setting is this :

To summarise, you should now have a working rspamd daemon running with your ZCS server and mail passing between them both for processing. The only modification you should need to make after each upgrade are the ones to the zmconfigd.cf file and the postconf command for setting the smtpd_milters socket, in addition you'll also need to modify the nightly scheduled cron job that trains the anti-spam system.

I'd be interested to hear feedback on how this went for you and if anyone with a large install wants to try Rspamd it would be great hear if it's as good as I think it is

NB: because of the high number of RBL (and other) DNS lookups, it's advisable to have a caching namesever on you lan (either the ZCS dnscache or your own DNS server, I use PowerDNS), if you don't do this you may get your own IP blacklisted with the RBL sites.

The zmtrainsa script has been modified to remove (some of) the DSPAM & SA training, I've added a section for training the rspamd system from the same spam/ham accounts in ZCS so users continue using the 'Spam' & Not Spam' function in the ZCS Web UI. You can take a look at the processing of this script by installing the following (make a copy of your original file first) modified zmtrainsa files and running the following command:

sh -x /opt/zimbra/bin/zmtrainsa_test2 &>>/opt/zimbra/log/zmtrainsa_test.log <-- check the output to see if it's working correctly.

The attachment (see my comment at the end of this post for the script) is a modified zmtrainsa file, you can replace the ZCS version but you'll have to replace it after each upgrade or you can add it as a separate cron job and run that then you can disable the ZCS version. FWIW, I make a copy of the original file then replace zmtrainsa with my modified version, I just replace it when I update ZCS.

Do not modify any of the config files in the /etc/rspamd directory, read the Rspamd documentation on the web site for details on how to add config override files and make further modifications to the rspamd config.

What I haven't mentioned is the current features in Rspamd and the fact that some of it's modules could also replace their ZCS equivalents, take a look at the web site here: https://rspamd.com/features.html

If you have any comments, suggestions, improvements for what I've posted then feel free to add to this thread. I hope you find this post of some useful.

PS To any moderators that pass by, I've made this a sticky but if you think that should be removed then feel free to do so.

PPS For some strange reason the forum software won't let me add an attachment to this post so I've pasted the modified zmtrainsa script below.

#!/bin/bash # # ***** BEGIN LICENSE BLOCK ***** # Zimbra Collaboration Suite Server # Copyright (C) 2005, 2006, 2007, 2008, 2009, 2010, 2012, 2013, 2014, 2015, 2016 Synacor, Inc. # # This program is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software Foundation, # version 2 of the License. # # This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; # without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # See the GNU General Public License for more details. # You should have received a copy of the GNU General Public License along with this program. # If not, see < gnu dot org /licenses/>. # ***** END LICENSE BLOCK ***** #

The hyperscan of regexp and modern PCRE lookahead for perl because our version with centos 6 does limit our ability - resulting in excessive false positives without proper scoping. We got wrapped around the axle trying to build a simple SA rule such as the body contains an img src that does not have an extension of jpg/gif/png/etc for what is best described as tracking - so out of desperation, I wrote a plugin which was simple enough but now I am on the hook for more manual updates with my zimbra updates, etc. Crazy.

I am still reading about rspamd but what I am seeing looks extremely positive.

Your howto recipe is much appreciated Bill.

I like that they have replaced the html parser and the AST optimizer sounds interesting. I don't know what a Bayes window of 5 vs Bayes window of 1 means but if more is better than I welcome it. LOL

After the amount of perl code I have been through the past few week with SA, I have serious doubts that the SA plugin code can be brought up to current html standards and the adhoc nature of those rules/plugin's is becoming a problem when scoring is basically broken from my perspective without rule/score updates. While I don't like change, I welcome something simpler that we can customize and that it's c/lua is bonus. My perl isn't what it use to be.

I'm not a great expert on managing linux systems or, as you can probably guess, a documentation writer but those notes got me through installing it a few times on test machines. I've always had problems with spamassassin and writing rules just ain't my thing, this sort of product is for the more technically adept than me so I hope that it's useful to others, I'll be interested to see if what it's like in a larger system than my server - don't forget my RFE if you think it's a worth while addition to ZCS. There is also the possibility that rspamd can replace a couple of the other modules such as DKIM signing. SPF checking, DMARC, greylisting etc. As I mentioned, I've removed all the policy, DNS restrictions, RBLs and postscreen configurations from my server and let rspamd handle it all and that provided another drop in CPU usage.

phoenix wrote:There is also the possibility that rspamd can replace a couple of the other modules such as DKIM signing. SPF checking, DMARC, greylisting etc. As I mentioned, I've removed all the policy, DNS restrictions, RBLs and postscreen configurations from my server and let rspamd handle it all and that provided another drop in CPU usage.

That is a good point on the other modules... From what I can tell, It's not possible to filter on the Authentication-Results header with Zimbra SA. I wanted to augment a meta rule and the best explanation I could come up with is that it was added after SA ran. I could match it when I ran SA from the command line and with a filter but not with the same simple SA header rule for a live message. It's still on my todo list but my goto solution always seems to be add a milter to the perimeter MX to test there and pass this header to Zimbra/SA to score. I need a better way.

I often wonder how many zimbra sites go 3rd party for spam filtering given the time consuming nature involved with this or outsource to gmail, etc in frustration. It's funny what many get caught up on. They won't use cloud storage solutions like dropbox, etc but they will allow all incoming email to pass through a 3rd party MX host for spam scoring or block all incoming traffic based on a tiny blacklist. Not saying there are not mitigation strategies but running zimbra in house should be a base for licence growth that could use some enhanced/advanced spam infrastructure IMHO. I also believe letsencrypt should work out of the box but you probably knew I was going to say that.

Maybe rspamd is that start. I think we are at a crossroads myself with spam,security,html 5+, etc and hope the SA and Zimbra communities can rise with it. I view spam more from the security side these days vs a nuisance issue given how many threats exist against browsers/companies/users. As I mention in that other post https://forums.zimbra.org/viewtopic.php?f=15&t=62447#p277751, It was harder than it should be and we are concerned. For me, I am going to try this on a test server because I need a hedge and develop some rspamd operational experience with their html parser should things not get better with the current solution. Bill your timing is pretty good because I had been looking the past few weeks but nothing was a comprehensive as what this appears to be.

I'm glad that was a relatively easy install for you, it's always a concern when I post a set of instructions as I know they work for me but......

I'll be interested in hearing your comments on it's ease of configuration and how it compares to your current ZCS server with SA, not being a 'professional' admin the finer points of configuring rspamd are a bit hard for me to grasp but I'll keep plugging away at it. Have a good week-end.

copied the resulting password from rspamadm to the /etc/rspamd/local.d/worker-controller.inc

Ran your test case of training and it worked perfectly.

Ok... so now I just need to figure out how all this works next. Then I have some tests to run through the parser.

Thanks for the very good directions in getting this working with Zimbra. I don't think I spent more than 5 mins. I have no idea how it works at this point but it trains spam and it appears to work from your test method. Next, I will probably just send email to both zimbra instances at the same time and see what can be learned about spam detection over a few weeks in comparison. For me performance isn't as big an issue vs accurate and better detection with the html parser given ways that html can be obfuscated by current best practices in spam delivery.

ccelis5215 wrote:J has a very good point, nowadays performance it's not the issue.

I'd slightly disagree that performance is not the issue. Yes, we all want it to be as accurate as possible but the side effect of reducing CPU usage is to be able to process more email. Don't forget that my simple home server that gets a very small amount of legitimate of mail per day plus the usual load of spam, the reduction of CPU usage from 15-17% to 3% is quite startling and shows that SA is too cumbersome. If you look at the rspamd site at the bottom of the first page there's a list of users and one of those is the Russian search engine rambler, they claim to process 100 million emails a day - I think CPU usage and performance might be a major factor for them.

HelloI will not add any comment on the subject, but I use RSPAMD on another server and I can tell you that, the software works perfectly.My question is, is it possible to go back to previous configuration by having activate postscreen?Yours sincerelyMz