Is it with HTTP connector or AJP connector?
Is it standalone Tomcat, or fronted by some other HTTP server?
What connector implementations are you observing this with (BIO/NIO/APR)? It is shown in the logs when Tomcat starts up.
Do you have RemoteIpValve configured?
Is it reproducible with a simple web application?

I've double checked the Tomcat 7 code and remote host and remote address are recycled at exactly the same point. If one failed to recycle, so would the other and both would be wrong. I am therefore doubtful that this is a Tomcat bug. To add to Konstantin's questions, how confident are you that the IP address in the auth header is correct?

I don't think that is it.
The remoteHost MessageBytes aren't recycled either in Tomcat 7 and that doesn't have the same issue. Also, I don't see how the remoteAddr MessageBytes could be accessed without first having been set.
I am extremely reluctant to start changing code without a clear explanation of why making the change will fix the observed problem. I don't mind if I can't reproduce the problem but there does need to be a logical explanation of why the change fixes the observed issue.

If the application is retaining a reference to the request object there is a possible race condition that could lead to the behaviour you see here.
If during the call to AbstractHttp11Processor.recycle() - triggered at the end of request processing - if there is a call to Request.getRemoteAddr() after AbstractHttp11Processor.remoteAddr has been set to null but before recycleInternal() is called then remoteAddr will end up being populated with the IP address of the previous connection.
The way to test this is to set set the following system property:
org.apache.catalina.connector.RECYCLE_FACADES=true
Please let us know how you get on with testing using this setting.

Looking at this again this morning, if the problem is in a Valve RECYCLE_FACADES won't help since Valves have direct access to the Request object rather than using the Facade.
The patch below can be used to see if a Valve (or similar) is accessing the request after it has been recycled. In addition to logging an error, it can either return null or throw an exception if this happens. Just comment out the one you don't want.
Index: java/org/apache/catalina/connector/Request.java
===================================================================
--- java/org/apache/catalina/connector/Request.java (revision 1174344)
+++ java/org/apache/catalina/connector/Request.java (working copy)
@@ -508,7 +508,8 @@
}
mappingData.recycle();
-
+ remoteAddr = null;
+
if (Globals.IS_SECURITY_ENABLED || Connector.RECYCLE_FACADES) {
if (facade != null) {
facade.clear();
@@ -1263,6 +1264,12 @@
*/
@Override
public String getRemoteAddr() {
+ if (mappingData.host == null) {
+ String msg = "Access to Request at invalid point in lifecycle";
+ log.error(msg);
+ // throw new IllegalStateException(msg);
+ return null;
+ }
if (remoteAddr == null) {
coyoteRequest.action
(ActionCode.REQ_HOST_ADDR_ATTRIBUTE, coyoteRequest);

Mark,
After applying the patch from comment 8, we get these in our logs (quite a alot). But we no longer have our problem of getting the previous IP address.
03:17:04,436 ERROR [Request] THREADID:292###PATCH###: Access to Request at invalid point in lifecycle. Returning null
for getRemoteAddr().

Fixed in trunk and 7.0.x and will be included in 7.0.22 onwards.
I also fixed a related issue discovered during testing that meant requests with multiple errors could trigger multiple entries in the access log.

We are seeing the same issue when Tomcat 7.0.22 is under heavier load (in our case servicing >30 incoming POST requests per second).
request.getRemoteAddr() is periodically (though not consistently) returning an IP address from the previous request.

(In reply to comment #15)
> Mark,
>
> we upgraded from 6.0.29 to 7.0.23 and have exactly the same problem. I can
> certainly exclude an application bug. This method is fundamental!
Again. *This* issue has been fixed. There are multiple possible causes of the symptom described here and the vast majority of them are application bugs. Please follow-up on the users-mailing list.
Feel free to create a new bug (since this specific issue is fixed) once you have one of the following:
- an explanation based on an analysis of the source code of why requests are getting mixed up
- a reproducible test case that demonstrates the issue

This is ASF Bugzilla: the Apache Software Foundation bug system. In case
of problems with the functioning of ASF Bugzilla, please contact
bugzilla-admin@apache.org.
Please Note: this e-mail address is only for reporting problems
with ASF Bugzilla. Mail about any other subject will be silently
ignored.