I've got a SLES machine that accumulates TCP connections in a CLOSE_WAIT state for what appears to be forever. These descriptors eventually suck up all available memory. At the moment, I've got 3037 of them, but it was much higher before a hurry-up reboot recently.

What's interesting is that they're not from connections to local ports that I expect to have listening processes. They have no associated PIDs, and their timers seem to have expired.

When doing the netstats, I had full privs, yes. I'll go check out the kernel processes angle -- that's a good idea. I'm really stumped, because there aren't supposed to be any listening sockets at all, except for two or three well-known privileged ports. Maybe it's a wierd iptables problem. I'll check that out too.
–
pboinMar 26 '11 at 11:33

CLOSE_WAIT indicates that the client is closing the connection but the application hasn't closed it yet, or the client is not. You should identify which program or programs are having this problem. Try using netstat -tonp 2>&1 | grep CLOSE to determine which programs as holding the connections.

If there are no programs listed, then the service is being provided by the kernel. These are likely RPC services such as nfs or rpc.lockd. Listening kernel services can be listed with netstat -lntp 2>&1 | grep -- -.

Unless the RPC services have been bound to fixed ports, they will bind to ephemeral ports as your connections appear to show. You may also want to check the processes and mounts on the other server.

You can may be able to bind your NFS services to fixed ports by doing the following: