Details

Description

Some emails keep hanging in my sendmail queue. When they are delivered through James I receive a crippled version (no subject, no body) and an error is sent back to Sendmail that the message could not be delivered. The message keeps hanging in the queue and blocks delivery of all messages that are behind it.
We can accurately reproduce the problem by filling in an address in the 'From' header that can not be resolved by DNS.

This problem might by hard to reproduce and I apologise for the lengthy description that follows. I don't know what's going on exactly but I think it would be best to give you plenty of information.

And Hes Siemelink reports that the initial problem reported in this issue appears to be related to sendmail disconnecting pre-maturely, although they aren't sure why.

"What we finally figured out was that Sendmail tried to resolve the From: address as a sort of anti-spam measure. The DNS result was an error so I suppose Sendmail interrupted the connection. This caused the 'partial message delivery'."

Noel J. Bergman
added a comment - 24/May/04 19:14 Fixed.
And Hes Siemelink reports that the initial problem reported in this issue appears to be related to sendmail disconnecting pre-maturely, although they aren't sure why.
"What we finally figured out was that Sendmail tried to resolve the From: address as a sort of anti-spam measure. The DNS result was an error so I suppose Sendmail interrupted the connection. This caused the 'partial message delivery'."

OK, this partial message problem is reproducible and comes from the fact that when the socket prematurely closes (as opposed to times out, which does generate an exception), the associated stream just returns -1 when asked for more data. This gets passed back through the chain until it reaches James, which couldn't tell the difference between the pre-mature end of stream and the legitimate end of the "Character Terminated" stream.

A fix will be committed. This does not, however, explain the other error for which this issue was opened.

Noel J. Bergman
added a comment - 22/May/04 00:20 OK, this partial message problem is reproducible and comes from the fact that when the socket prematurely closes (as opposed to times out, which does generate an exception), the associated stream just returns -1 when asked for more data. This gets passed back through the chain until it reaches James, which couldn't tell the difference between the pre-mature end of stream and the legitimate end of the "Character Terminated" stream.
A fix will be committed. This does not, however, explain the other error for which this issue was opened.

Please try to avoid using JIRA as a discussion tool. Use the mailing list(s) to discuss potential problems.

The DEBUG level for the the smtphandler will log all of the protocol requests/responses. It will not log the data, but will log other things. If necessary, additional logging could be added in a private build to isolate where a problem is happening in your reproducible environment.

> perhaps a graver issue is that a partial message is delivered by
> James, while Sendmail keeps the original message in its queue
> because it thinks delivery failed.

If there is an exception during processing, James should not deliver the message. The fact that it attempts delivery indicates that the code has entered processMail (beyond processMailHeaders) without any prior Exception. What happens next is this:

processMail creates a new MailImpl with an InputStream.

MailImpl creates a MimeMessageInputStreamSource.

MimeMessageInputStreamSource reads the entire InputStream
into a temporary file.

The InputStream returns -1 when the <CRLF>.<CRLF> is found.

MimeMessageInputStreamSource throws a MessagingException
if there is an IOException processing the InputStream.

If we get to the sendMail call in processMail, James believes that it has properly received an entire message, whose contents would be temporarily located in a .m64 file under $PHOENIX_HOME/temp/.

Noel J. Bergman
added a comment - 12/May/04 17:09 Please try to avoid using JIRA as a discussion tool. Use the mailing list(s) to discuss potential problems.
The DEBUG level for the the smtphandler will log all of the protocol requests/responses. It will not log the data, but will log other things. If necessary, additional logging could be added in a private build to isolate where a problem is happening in your reproducible environment.
> perhaps a graver issue is that a partial message is delivered by
> James, while Sendmail keeps the original message in its queue
> because it thinks delivery failed.
If there is an exception during processing, James should not deliver the message. The fact that it attempts delivery indicates that the code has entered processMail (beyond processMailHeaders) without any prior Exception. What happens next is this:
processMail creates a new MailImpl with an InputStream.
MailImpl creates a MimeMessageInputStreamSource.
MimeMessageInputStreamSource reads the entire InputStream
into a temporary file.
The InputStream returns -1 when the <CRLF>.<CRLF> is found.
MimeMessageInputStreamSource throws a MessagingException
if there is an IOException processing the InputStream.
If we get to the sendMail call in processMail, James believes that it has properly received an entire message, whose contents would be temporarily located in a .m64 file under $PHOENIX_HOME/temp/.

Thanks, will do that. I saw no exceptions in the James log on the INFO level (I assume that exceptions are never logged on the DEBUG level?).

We're a bit puzzled by this behavior – it might by a Sendmail thing but it seems only to occur in combination with James.

In the meantime the domain that caused the behavior is up again. I have to figure out how to configure our DNS so a certain domain times out before I can test again.

Apart from the From header that triggers this, perhaps a graver issue is that a partial message is delivered by James, while Sendmail keeps the original message in its queue because it thinks delivery failed.

We noticed that James delivers mail when the connection is interrupted in the DATA part of the SMTP protocol. So half-sent messages do arrive. Is this according to spec? We did some quick tests on Exchange, Sendmail and Exim and those servers do not deliver mail in this case.

Hes Siemelink
added a comment - 12/May/04 09:46 Thanks, will do that. I saw no exceptions in the James log on the INFO level (I assume that exceptions are never logged on the DEBUG level?).
We're a bit puzzled by this behavior – it might by a Sendmail thing but it seems only to occur in combination with James.
In the meantime the domain that caused the behavior is up again. I have to figure out how to configure our DNS so a certain domain times out before I can test again.
Apart from the From header that triggers this, perhaps a graver issue is that a partial message is delivered by James, while Sendmail keeps the original message in its queue because it thinks delivery failed.
We noticed that James delivers mail when the connection is interrupted in the DATA part of the SMTP protocol. So half-sent messages do arrive. Is this according to spec? We did some quick tests on Exchange, Sendmail and Exim and those servers do not deliver mail in this case.

Noel J. Bergman
added a comment - 12/May/04 04:29 Please turn on DEBUG for the smtpserver (in environment.xml), and look to see what is happening in the smtpserver log. Also let us know if there are any exceptions.
There should not be any dependence on the From: header, nor does anything appear wrong in the code.