Network bandwidth: packet headers

Network bandwidth refers to how much data you have room to send over the wire. As you approach the limit, you will see increased amounts of packet loss. If you go over the limit, you will eventually get disconnected from your session.

The XNA Framework measures bandwidth in bytes per second. Confusingly, network vendors like to measure it in bits per second (I suspect they do this to make their numbers look bigger). Even more confusingly, both units have the same acronym: bps, or kps/kbps if you are talking about thousands of them. Be wary, and always check which you are dealing with.

In order to be playable by the majority of home Internet users, Xbox games are expected to work with as little as 8 kilobytes per second of both upstream and downstream bandwidth.

You might think you could figure out your bandwidth usage by adding up the size of all the data you are sending, multiplying by the number of machines you are sending it to, then multiplying again by the number of sends per second.

But you’d be wrong.

You forgot to include the packet headers. Every time you send a packet over the network, in addition to the data inside that packet, you must factor in:

20 bytes IP header

8 bytes UDP header

~22 bytes for LIVE and the XNA Framework (used for NAT traversal, encryption, and reliable/ordered delivery)

That is ~50 bytes of header data per packet.

Consider an example where we are sending a single boolean value to a single other player, 60 times per second:

1 byte of payload data

+ 50 bytes of header = 51 bytes

* 60 packets per second = 3060 bytes

Whoah! We are using 3 kilobytes per second (remember we only have 8 in total) even for so little real data. 98% of our bandwidth is being wasted on packet headers.

How can you survive this deadly attack of the killer packet header gremlins?

Send data less often

Games typically only send 10 to 20 packets per second, rather than every frame.

This could be dynamic: send packets less often to players who are further away from you.

MotoGP supported 16 players over Xbox LIVE. With that many players, we could only afford 4 packets per second. We needed good prediction and smoothing algorithms to get away with such a low send rate!

Merge packets

If you replace many small packets with one big one, they can all share a single header.

Rather than sending notifications as soon as things happen, consider holding onto them for a few frames until they can be merged with other packet data.

If you send many packets to the same machine within a single frame, the next call to NetworkSession.Update will automatically merge them for you.

The XNA Framework automatically merges voice data and reliable UDP acknowledgements into your existing packets, so this data does not usually require any extra packet headers.

The 8k limit is separate for upstream and downstream. So you can be dealing with 16k in total: 8k going out from you to other machines, and another 8k coming in from them to you.

For peer-to-peer games this usually balances out pretty well, but for client/server games it can be a pain when the clients have very little data to send to the server, but the server has way too much data to send to each client, so the clients end up running out of downstream bandwidth, while the server runs out of upstream bandwidth, even though each had plenty of bandwidth going unused in the other direction.

I don't have any more recent data on this, but at least for commercial Xbox games the TCR about bandwidth usage has not changed.

My totally unscientific estimate is that, while bandwidth has certainly improved dramatically for some users over the last few years, it is unchanged for many others, so if you want your game to run well for a large number of players, this 90th percentile figure has probably not changed much if at all.

For instance I'm still using the same home net connection as I was in 2005.