My network stack has been around for some time but recently I discovered a problem. When trying to download a relatively large file (1.4M) from a server via HTTP, it is unacceptably slow. So I ran wireshark to analyse the traffic. Obviously it's a big mess, but what I think I'm seeing is the following:

The conclusion that I'm getting from this is that the server isn't expecting every single message to be ACKed, and treats the fast ACKs as indication that data has been dropped. I don't see this mentioned anywhere in RFCs or on wikipedia (i might be blind).

So my question is, with TCP, am I supposed to be waiting some time before sending an ACK, and if more data arrives, am I supposed to "reset" the timer, and effectively only send an ACK if I have not received more data for some time ???

_________________Glidix: An x86_64 POSIX-compliant operating system, aiming to be as optimized as possible, especially in graphics.https://glidix.madd-games.org/

TCP is stream oriented. It doesn't consist of messages. Why doesn't your stack immediately acknowledge all the data it has received so far?

It DOES axknowledge all the data it has received so far. But it takes some tiny bit of time for the ACK to be delivered to the server (obviously). And it appears that in that time, the server sends MORE data, before it even receoved the ACK. The server runs Ubuntu.

And by messages, i meant the packets. I know TCP is stream oriented, but data over IP is still sent using packets.

_________________Glidix: An x86_64 POSIX-compliant operating system, aiming to be as optimized as possible, especially in graphics.https://glidix.madd-games.org/

But it takes some tiny bit of time for the ACK to be delivered to the server (obviously). And it appears that in that time, the server sends MORE data, before it even receoved the ACK. The server runs Ubuntu.

This is perfectly normal for TCP. If it was waiting for your ACK before sending more data, performances would be very slow. There is a sliding windows used to limit how much data can be sent without an ack. IIRC that window size can change dynamically.

But it takes some tiny bit of time for the ACK to be delivered to the server (obviously). And it appears that in that time, the server sends MORE data, before it even receoved the ACK. The server runs Ubuntu.

This is perfectly normal for TCP. If it was waiting for your ACK before sending more data, performances would be very slow. There is a sliding windows used to limit how much data can be sent without an ack. IIRC that window size can change dynamically.

So, if I understand correctly:

1. My OS receives data over TCP.2. It should wait some small amount of time (50ms??), or until window_size bytes are received, before it sends an ACK.3. This way, if the server sends more data, the client ACKs the entire thing instead of acknowledging each individual packet.

Is that what the process is supposed to be like?

_________________Glidix: An x86_64 POSIX-compliant operating system, aiming to be as optimized as possible, especially in graphics.https://glidix.madd-games.org/

I haven’t looked at TCP for a while, so I don’t remember much from my short experience with it. But you should pay attention to the fact that packets may be coming out of order (there are TCP implementations that mess up the order before the packets even hit the wire!) making it look like you have lost some. And if you start complaining to the sender about the loss too soon, you’re going to have bad performance. In reality you should keep a number of out of order packets, so you don’t end up rerequesting or receiving them again. I can’t direct you to the right place in the right RFC, but look around SACK.

1. My OS receives data over TCP.2. It should wait some small amount of time (50ms??), or until window_size bytes are received, before it sends an ACK.3. This way, if the server sends more data, the client ACKs the entire thing instead of acknowledging each individual packet.

Is that what the process is supposed to be like?

I am not sure why you want to wait at all... Just ACK each time you receive a packet of data. Waiting won't help with anything: there will always be another packet of data coming in. If you wait for window_size bytes, you defeat the point of having a window in the first place.

My network stack has been around for some time but recently I discovered a problem. When trying to download a relatively large file (1.4M) from a server via HTTP, it is unacceptably slow. So I ran wireshark to analyse the traffic. Obviously it's a big mess, but what I think I'm seeing is the following:

1. Server sends some data.2. Server sends more data.3. Server sends even more data.4. Glidix ACKs #15. Server didn't get an ACK for #2 in time and assumes packet was lost, so server re-sends #26. Server didn't get an ACK for #3 in time and assumes packet was lost, so server re-sends #37. Server sends #48. Glidix ACKs #29. Server didn't get an ACK for #3 in time and assumes packet was lost, so server re-sends #310. etc etc etc

If that's the case, then the server is working correctly and the client has major performance problems (not sending ACKs fast enough).

Alternatively it could be like this:

1. Server sends some data.2. Server sends more data.3. Server sends even more data.4. Glidix ACKs #15. Server re-sends #2 even though receiver hasn't had time to receive the first copy6. Server re-sends #3 even though receiver hasn't had time to receive the first copy7. Server sends #48. Glidix ACKs #29. Server re-sends #3 even though receiver hasn't had time to receive the previous copy10. etc etc etc

In that case there's a problem with the server (bad/misconfigured or missing time-outs and/or inappropriate window size) and the client is working correctly.

Note: I am assuming that this is on a nice "quiet" LAN, and network congestion is not involved.

mariuszp wrote:

The conclusion that I'm getting from this is that the server isn't expecting every single message to be ACKed, and treats the fast ACKs as indication that data has been dropped. I don't see this mentioned anywhere in RFCs or on wikipedia (i might be blind).

As far as I know; the client can coalesce ACKs (e.g. instead of sending "ACK #1", "ACK #2", "ACK #3" it can just send "ACK #3" to tell server it received everything up to #3); but (even though it's a good idea - e.g. improve performance/reduce bandwidth consumption by reducing the number of packets sent by client) this is a purely optional optimisation, and server can not/must not assume that a packet won't be received simply because it wasn't included in the latest ACK from client.

Cheers,

Brendan

_________________For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.

You don't acknowledge that you have received a packet, you acknowledge that you have received all bytes up to a certain point. So if you really have received all bytes up to the end of packet 3 then you can acknowledge them all at once (assuming there are no lost packets). But why are your acknowledgements so slow? The last thing you need to do is to deliberately wait before acknowding received data; that's just going to throttle the transfer. The only reason not to immediately acknowding received bytes if some preceding bytes in the stream have not yet arrived.

In the case that a packet has been dropped, the acknowledgement mechanism offers a significant optimization. If a dozen packets have been received since the dropped packet then (when that packet is eventually received) you can miss out a dozen acks. That might well be enough to prevent some of the received packets from being retransmitted.

(Note that you can't acknowledge the bytes in those later packets untill the dropped one is received. Acknowledgement means "I have received every byte up to [the one specified in the ack packet]".)

I send an ACK every time a packet arrives. My code simply receives a TCP segment, and ACKs all the bytes it received so far, including the newly-arrived segment. In case I misdiagnosed the problem, a attached a Wireshark capture showing what is going on if anyone wants to take a look.

Well, you seem to have a lot of erroneous packets there. I'd start at the beginning, with packets 22 and 23. Why is a duplicate SYN packet sent?

Then packets 25 and 26 again seem to be duplicates. What's going on here?

Packet 27 is again duplicated almost immediately.

Later on you are getting a host of duplicate ACK packets, so I have to think there is something wrong with your acknowledgement mechanism. I get the impression that every packet you send is sent twice, which is causing problems.

I would try first of all to get rid of that duplicate SYN packet, then work your way through the other duplicated packets to see what is causing the problem. What happens if you just do a very simple connection and request, that involves just a single packet of data. Do you still get the duplicates?

wireshark is showing me all packets duplicated when i sniff on eth0 with VirtualBox brisged to it. I do not know if 2 packets are really sent or if its a sniffing problem caused by the bridging...

If i try 2 different drivers in my OS, and thdn debug each one to see what packets are sent, it tells me that only one of each is sent as expected. So it seems to be wireshark getting confused by sniffing on a bridges network.

EDIT: It now appears that if I sniff on a "host-only" network, all packets are also duplicated... perhaps it is in fact some obscure problem at the OS level. I'll have to try a few more things...

EDIT 2: If I run Ubuntu in a VM, it ALSO shows all packets duplicated... so it must be wireshark/virtualbox doing something

_________________Glidix: An x86_64 POSIX-compliant operating system, aiming to be as optimized as possible, especially in graphics.https://glidix.madd-games.org/

Well, when i disable monitoring mode on the VM (and so the guest was receiving less unnecessary packets) the download now goes up to 14% almost instantly (previously took a few minutes for 3%) but is also less reliable and sometimes fails to make connections... that's interesting

_________________Glidix: An x86_64 POSIX-compliant operating system, aiming to be as optimized as possible, especially in graphics.https://glidix.madd-games.org/

Who is online

Users browsing this forum: No registered users and 13 guests

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum