PR> 1) I've seen in production that some sockets get large very
PR> quickly during periods of high latency (eg: when sending to a user
PR> downloading from a cablemodem and their headend gets temporarily
PR> saturated and has large buffers, which raises the RTT under
PR> saturation, which increases the bandwidth delay product), but then
PR> as there isn't any code to shrink the buffers. This would probably
PR> need to be in the timers to notice the case of the sender
PR> temporarily stopping sending - eg in a keepalive http socket (a
PR> separate, but related issue).

The send buffer increasing because of buffering in the network (like
a cable headend) is expected. We don't have any way to distinguish
this from a normal high latency link. Is is really the job of the
router operator (ISP) to configure the buffer memory and interface
queues properly and to enable things like RED.

Shrinking the send buffer can only be done when it is idle and empty.
But then it doesn't consume any kernel memory anyway. The next time
you send something it is very likely that the network conditions are
roughly the same as before and a large socket buffer makes sense and
increases throughput. In the case where the socket buffer is large
and filled and the receiver becomes unreachable the socket has to
hold on to the data unless the application closes it. There is no
way to undo an application write() and to drop data from the send
buffer. That would violate all TCP specs and assumptions about the
behavior of sockets and reliable data transport.

You have to keep in mind that TCP sees the network only as black box
and doesn't have any information on delays, bandwidths, network buffering
or any other parameter of it. It can only try to discover certain
characteristics after a flow of data has been established. Even then
there is a lot of variance and uncertainty still going on. It's essentially
impossible behave perfectly for all possible network conditions. The
automatic send buffer is not perfect either and has some cases where
it may allocate too much resources of the host to a particular connection.
OTOH it does much better than the small fixed sized buffer we had before.

Re: strange select() behavior...sharing descriptors, might fill that interface buffer.... What do you do when a blocking socket operation returns ENOBUFS? ... because it's a semantic change....(comp.unix.programmer)

Re: Socket stuck with puts over ADSL line... gets stuck with the puts command within the filevent writeable ... Is the socket configured as -blocking 1? ...local buffer would fill rapidly, ... buffered portion across the WAN as its own TCP packet,...(comp.lang.tcl)