Hi, wondering if this is happening to anyone. Last couple of weeks I been having users complaining about Yahoo emails not getting through. I have narrowed it down to SFDC, if I disable it(0) Yahoo emails will get through just fine. But once enabled(5), it will allow some but will block most emails from Yahoo. I am running registered version 4.2.4.844, and prior version 4.2.4834. Let me know, thanks.

The SFDC filter blocks emails based on their contents, not based on the sender's IPs/domains. The content match is performed by analyzing, in realtime, the fingerprint (or hash) of all spam emails that are being stopped at that time. If the content fingerprint matches the fingerprint of similar content being stopped at that time, the SFDC filter blocks that email. This is again absolutely not done based on the domains/IPs.

If you can zip us the original source of 2-3 such emails at support @ logsat.com we'd be glad to look over them however to ensure there are no other issues.

Roberto, it's actually any emails from Yahoo. A simple plain text test message would be blocked. I am away from the office for the next couple of days, I will get them to you when I get back. BTW, is there a way to reset SFDC on the spamfilter? Thanks for the reply.

If it's *all* emails from yahoo, are you certain that there isn't another filter (maybe blacklisted senders/domains/IPs that belong to Yahoo) that are blocking the emails?

I say this because the SFDC is a centralized database that all SpamFilter servers in the world use to block content. If an email you receive is blocked by the SFDC filter, then that email would be blocked by all other SpamFilters as well. If the SFDC was to block all emails from Yahoo, then all the companies using SpamFilter would not be receiving Yahoo emails as well (which is not occurring..!).

As I mentioned, if you can zip us some sample, we'd be happy to see what could be happening in your case.

There is no way to "reset" the SFDC filter as it is a filter we host and control. You could however increase the threshold setting for it to make it less sensitive.

Roberto, let me corret myself. Not all messages from Yahoo are getting blocked. All Yahoo messages are taking a long time to receive tho, it takes about an hour before it's delivered. Gmail, after the greylisting period will deliver instantly. I did forwarded you a email with subject(RE: Failure Notice) for you to exam. Thank you.

I was emailing you about the logs, when I saw this post bout SFDC not being the filter that blocks some of the emails, but rather being the greylist filter. This is actually why I wanted to see the activity logfile - to see which filter caused the block. If it was indeed the greylist filter, please do note that we ship SpamFilter with that filter disabled, as it can, in some cases, cause the delivery of some emails to be initially delayed. This can occur (rarely) with large providers who use dozens of non-RFC compliant SMTP servers to send their emails. Per RFC a mail server must retry to send an email if the delivery temporarily fails. Some providers violate this rule by having a different SMTP server try delivering that email instead of retrying with the same one. This does increase the delays for the delivery. Yahoo appears to be using these non-RFC compliant servers.

Disabling the greylist filter will of course stop this issue with Yahoo, but you will allow more spam to get thru. Alternatively, you could try manually adding to the greylist filter configuration file the list of IPs that Yahoo uses to send mail to the file:

\SpamFilter\Domains\GreyListAllowed.txt

I'm attaching the list to this post. To proceed you would need to stop SpamFilter, copy/paste the entries below to the above GreyListAllowed.txt file, and restart SpamFilter.

Please let me know if this does not help, so we can look into this further.

Yesterday I did a test and sent a email over Yahoo. Today I got this "unable to deliver" report back from them.

This also after we implemented all the whitelisted Yahoo IP's you suggested. How do you create this Yahoo IP list?

Is there a way to find out which
IP Yahoo used to deliver this email? Is this "mailer-deamon"
comming always from the same mailserver from which they tried to deliver the
Email? So in this header it was the 77.238.189.231 and the
212.82.108.238 or the 192.168.2.100 server?

From the the bounce email I can't tell with absolute certainty which IP address made to the connection to SpamFilter. However, based on the email headers, usually the first "Received:" in the list will report the server that will deliver the email. In this case, it should be nm21.bullet.mail.ird.yahoo.com. That hostname resolves to 212.82.108.136.

This IP should already be listed in the GreyListAllowed.txt file that we are updating nightly. This is the subnet that contains it (taken fro that file):

212.82.108.*~47795.0

The updating instructions are on the posting I mentioned, which I'm including below.

Did you try going thru SpamFilter's activity logfile for the 16th to see if the above IP made a connection at around 22:45 GMT?

Yes I checked the IP and the log already, there is not one connection from this IP.I'm also in the Yahoo Postmaster group where they publish periodically the current outgoing mail server address list and any planned
changes in the list for whitelisting, but yeah, as in the current example, often it's not actual enough.

Unfortunately not. A spammer could take his IP, and configure the PTR record for that IP to read "bullet.mail.ird.yahoo.com" (they can assign any name they want to the PTR...) when sending an email pretending to be Yahoo. That would very easily then bypass the greylist filter.

What's odd is that you did not see that IP (212.82.108.136) in the log, as from the bounce message it does appear that the delivery failed for that host. Just to confirm, did you check SpamFilter's activity logfile on the 17th (not the 16th), if I read the headers correctly and your server is located in the same timezone as the sender of the email (GMT +2)?

I checked the 15th/16th and 17th But I sent you the logs now as well, looks like you don't trust me

Oh and btw, I blamed Yahoo now many times coz I thought they are really violating the RFC 2821 and how you also wrote in your thread here.

The greylist filter is a great tool that can be used to fight spam, as
long as SMTP server follow the RFC guidelines. Per RFC, if an SMTP
server is unable to deliver an email because the receiving SMTP server
rejects the connection attempt with a 4xx error code (which indicates a temporary unavailability), they are supposed to retry the delivery after a few minutes.

I spent yesterday a lot of time to read through all the SMTP relevant RFC documents to find where such providers are violating the RFC, but honestly, nowhere I could find something where is written, that it has to be the same SMTP server who retries to send the mail again after a 4xx.

Anyway, I consider the greylisting is an abuse of SMTP standards. The server isn't really busy at all so sending a 4xx status code isn't the way the protocol was really designed.

I received the logs - could you also email me the original headers of that email above in the post, as I think you modified the to/from email addresses in there?

For the RFC, it's RFC2821. Here are the relevant sections, where they define "client" and "servers", which are the two specific SMTP processes (servers) involved in the email, the "should" for retries, and the last one stating that 421 error codes must be treated like 451 error codes,

Section 2.3 provides definitions of terms specific to this document.
Except when the historical terminology is necessary for clarity, this
document uses the current 'client' and 'server' terminology to
identify the sending and receiving SMTP processes, respectively.

=========================================

4.5.4.1 Sending Strategy
The general model for an SMTP client is one or more processes that
periodically attempt to transmit outgoing mail. In a typical system,
the program that composes a message has some method for requesting
immediate attention for a new piece of outgoing mail, while mail that
cannot be transmitted immediately MUST be queued and periodically
retried by the sender. A mail queue entry will include not only the
message itself but also the envelope information.
The sender MUST delay retrying a particular destination after one
attempt has failed. In general, the retry interval SHOULD be at
least 30 minutes; however, more sophisticated and variable strategies
will be beneficial when the SMTP client can determine the reason for
non-delivery.
Retries continue until the message is transmitted or the sender gives
up; the give-up time generally needs to be at least 4-5 days. The
parameters to the retry algorithm MUST be configurable.

==============================

3.9 Terminating Sessions and Connections
An SMTP connection is terminated when the client sends a QUIT
command. The server responds with a positive reply code, after which
it closes the connection.
An SMTP server MUST NOT intentionally close the connection except:
- After receiving a QUIT command and responding with a 221 reply.
- After detecting the need to shut down the SMTP service and
returning a 421 response code. This response code can be issued
after the server receives any command or, if necessary,
asynchronously from command receipt (on the assumption that the
client will receive it after the next command is issued).
In particular, a server that closes connections in response to
commands that are not understood is in violation of this
specification. Servers are expected to be tolerant of unknown
commands, issuing a 500 reply and awaiting further instructions from
the client.
Klensin Standards Track [Page 27]
RFC 2821 Simple Mail Transfer Protocol April 2001
An SMTP server which is forcibly shut down via external means SHOULD
attempt to send a line containing a 421 response code to the SMTP
client before exiting. The SMTP client will normally read the 421
response code after sending its next command.
SMTP clients that experience a connection close, reset, or other
communications failure due to circumstances not under their control
(in violation of the intent of this specification but sometimes
unavoidable) SHOULD, to maintain the robustness of the mail system,
treat the mail transaction as if a 451 response had been received and
act accordingly.

====================================

4yz Transient Negative Completion reply
The command was not accepted, and the requested action did not
occur. However, the error condition is temporary and the action
may be requested again. The sender should return to the beginning
of the command sequence (if any). It is difficult to assign a
meaning to "transient" when two different sites (receiver- and
Klensin Standards Track [Page 42]
RFC 2821 Simple Mail Transfer Protocol April 2001
sender-SMTP agents) must agree on the interpretation. Each reply
in this category might have a different time value, but the SMTP
client is encouraged to try again. A rule of thumb to determine
whether a reply fits into the 4yz or the 5yz category (see below)
is that replies are 4yz if they can be successful if repeated
without any change in command form or in properties of the sender
or receiver (that is, the command is repeated identically and the
receiver does not put up a new implementation.)

10.17.11 00:45:03:749 -- (2608) IP is in not in GreyList Allowed. Disconnecting: 212.82.109.245

and if you had the entry "212.82.109.245~40834.0315338194" in your greylist file, then that connection should have been allowed thru. If you are positive that entry was present on the 17th, it looks like SpamFilter is having an issue with the greylist file.

Please remember that the IP may have been added to your greylist file *after* 00:45 on the 17th if that IP connected again within 12 hours. This did not occur on the 17th (the IP connected again after 13 hours so didn't make the cut), but it could have happened on another day. You can find out by looking in the logs for the entry:

IP Greylist - Added 212.82.109.245 to list

If you don't find it, meaning that the IP was likely present before the 17th, could you please zip and email me your SpamFilter.ini and your GreyListAllowed.txt files so I can have a look?

I think there is an issue with the Greylist. I added all the IPs from that list last night, stopped and restarted SF. This morning I see a bunch of IPs that are not passing thru that are listed in the GreyListAllowed.txt file

Based on what Wayne had originally reported above, yes, it's very possible. Just as I asked Wayne, could you please zip and email us (support at logsat dot com) your SpamFilter.ini and your GreyListAllowed.txt files so we can have a look?

Checked now the logs from the 16th to the 26th and I can't find a entry like "IP Greylist - Added 212.82.109.245 to list". So it must been earlier, but I don't keep this logs that long so I can't really check it. But for me it sounds logical that the entry must be in the GreyListAllowed.txt before the 17th.

Uhm... We were finally able to duplicate this thanks to both of your greylist files. This is very odd and are trying to figure out what is happening. Almost certainly this is a bug with SpamFilter. I will update this thread within the next 12/36 hours with more info.

The problem can occur intermittently when the GreyListAllowed.txt file contains an entry with a wildcard, followed by one or more entries that contain an IP within the above subnet. For example with this sequence:

216.27.93.*~47795.0

216.27.93.111~40822.871038831

216.27.93.112~40822.871038831

if the IP address 216.27.93.122 makes a connection, the 2 extra redundant lines with the specific IPs following the wildcarded entry may cause a mismatch. We are testing right now a beta release that automatically cleans up the GreyListAllowed.txt file upon startup to remove the redundant IPs that already fall into a wildcarded range, and should pre-release it to the public within 24/48 hours if there are no issues found with it.

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot delete your posts in this forumYou cannot edit your posts in this forumYou cannot create polls in this forumYou cannot vote in polls in this forum