TCP handshaking as application handshaking

Some protocols use the TCP handshaking as the application handshaking,
violating the OSI layers. Many of them, handmade. Others, official, as HP 9100.
For instance, an application may need only control events as Start of
connection and End of connection, and the programmer may think
that the TCP similar events will be useful instead of thinking application
control messages for those events.

If someone follows the
usual guidelines on TCP connection termination, he may found unexpected problems
related to data loss.

Usual connection termination, as in the connection state diagram

One side active closes, and sends a FIN. The other side sends ACK
(enters the CLOSE_WAIT state), and then when needed actively closes the
connection sending a FIN to the termination initiator, waiting its ACK, and then
considering the connection closed in both sides. On BSD you may think
of the call close() for active close.

First consideration: the state diagram doesn't show the state
changes related to data transmissions.

Key sentence in RFC 1122

From 4.2.2.13: If such a host issues a
CLOSE call while received data is still pending in TCP, or
if new data is received after CLOSE is called, its TCP
SHOULD send a RST to show that data was
lost.

This, in fact, means that you will NOT receive a FIN, if the
other side applicationactively closes the connection
without having received the incoming data. The stack will send a RST.
This potentially introduces data loss.

You can consider these two possible additional meanings for these sent
messages:

RST - Forced connection drop AND the application has not read all
the data the stack has received.

FIN - No more data from sender AND the application has read all
the data the stack has received.

A program which may find the problem

Let's assume the programmer wants to rely on the TCP handshake for sending
some data. He doesn't know if the other side will send anything:

mysocket = connect(otherhost);

send(mysocket, mydata);

close();

The programmer made a big error between the send() and the close() calls: if
the other side sends some data, and this side doesn't recv() it, it will send a
RST instead of a FIN. If the other side TCP/IP stack receives that RST, it
will throw away all the data the application still hasn't recv().. Data
loss.

A better program? Not specially.

mysocket = connect(otherhost);

send(mysocket, mydata);

do r = recv(mysocket, tmpbuffer) until r == 0;

close();

Regardless of some other error control, here we assure that we received a FIN as
an indicator that the other side application has received all the data we
sent. But how will the other side program look like?

mysocket = connect(otherhost);

recv(mysocket, buffer_for_recv_data);

do r = recv(mysocket, tmpbuffer) until r == 0; /* ?????? */

close();

You see that the third step has a problem - the sending side will not send a FIN
until we send a FIN. So, we can only solve this situation using shutdown() and
half close.

Lesson

Don't violate the OSI layers, and think a handshake for your application
communication.