I keep hearing PPP argument for many years. At some point it was true, for sure, but does still so many people play online games through dial-up connection ? IMHO most gamers these days have ADSL/cable. I'm not sure - does TCP header compression also apply to ADSL/HDSL/etc ?

Broadband is still in the minority for home users. We tech-heads tend to forget that because we represent a large part of thge existing broadband market. And even in broadband, many providers are using PPPoE.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

I live in Poland, which is certainly not very advanced technically. Looking at my friends which have computers, 2 of them use dial up, 2-3 does not use net at all from home and rest (at least 10) have broadband. By broadband, I do not mean it is really broad - in many cases it is per-flat internal ethernet which is later connected through ADSL/radio to provider.

Of course, this data means nothing statistically, but it is enough for me to ask question if modem-centric world is still true. Are there any recent tests about it ? Remember that we are talking about gaming community, not generic housewife-with-aol.

I'm sure that significant percentage of gamers still use modems. But question is, if this 'significant' is 20%, 50% or 80% at this moment?

Remember, that by using UDP, we are not putting modem users away - just taking a bit more of their bandwidth, which, while limited, is still large enough to play most UDP-based games.

I should qualify that I was talking about the US market, where it is DEFINITELY still true that Boradband is a minority and a luxury.(I pay $55 a month for my cable modem. Many people can't afford to plunk down an extra $600+ a year just to get faster net access.)

Every country will be different. But at least for US developers and destributors North America is the primary market target. Japan is the first foreign target. Europe is a lagging third.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

Do most people live in areas of the US where broadband costs $600 per year??

It's pretty common in the US for broadband to be anywhere from $40-$60 for basic service. I've had three different ADSL providers so far and while the cost is dropping (some) so are the data rates. I started with 1.5M/768K and slowly dropped down to 512K/368K every time I moved. Oh, and PPPoE sucks. Not to mention that the SBC managers apparently have no qualms about lying through their teeth about it. "Oh no, sir! We don't use PPPoE!"

Broadband is available to approx 80% of Canadians. Most of those without access are living in small towns and villages. There's a federal government program to push broadband access out to every rural and northern community.

(Edit: I should also add that I'm paying $37.95 CAD, which works out to about $28.20 USD.)

Broadband is available to approx 80% of Canadians. Most of those without access are living in small towns and villages. There's a federal government program to push broadband access out to every rural and northern community.

(Edit: I should also add that I'm paying $37.95 CAD, which works out to about $28.20 USD.)

Broadband is available to approx 80% of Canadians. Most of those without access are living in small towns and villages. There's a federal government program to push broadband access out to every rural and northern community.

(Edit: I should also add that I'm paying $37.95 CAD, which works out to about $28.20 USD.)

The Joys of Socialism. In general the more socialist the country the faster infra-strcuture changes like moving to broadband will happen.

And actually my cable modem went UP this year from $45/mo to $55/mo. (Frickin Comcast, but thats a long story.)

JK

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

Well, I'm in a very un-techie location in south-central-eastern US, and I'm paying $44/mo for a pretty damn fast cable connection, which is less than the cost of a phone line and dialup access (runs about $20/mo in these parts for dialup and $32/mo per phone line). Where are you getting this $600 figure from Jeff?

The average price that I know of in Ontario is $39.95 to $44.95 Candian per month for ADSL or Cable.. around $30 US.

I'm not sure of the 80% quoted above, but most people I know (geeks) have broadband.. so I wouldn't be suprized if it is acccurate.

My provider is Shaw Cable. I'm not sure how far their reach extends outside of Alberta.

I'm pretty happy with my cable access, but I have considered going with Telus and ADSL. They have a promotion on now offering 1 years service for $24.95 CAD per month (about $18.51 USD for comparison's sake). After that, its $34.95 CAD per month. But the main reason I'm considering the switch is the fact that they encourage their customers to set up servers at home (HTTP, etc), while Shaw forbids it.

Unfortunately, a good amount of DSL is controlled by SBC. And for the first few months, they only charge $30 or so, but then they start charging the full price, which comes to about $600/year. And AT&T recently switched all their consumer DSL customers to SBC (at least in the Los Angeles area).

I don't think it affects compression at all. What it does is add unnecessary overhead to every packet. From your side, your router wraps the packet in a PPPoE packet, send it to the other side, the other side unwraps it, then sends it. Here are some of the fine points of what this means:

1. Greater transmission latency2. Slower upload/download rates due to wasted bandwidth3. More difficult setup for the end user4. Pushes the MTU higher than is routable thus requiring you to reduce the MTU in order to use the connection. This *again* reduces performance.

All in all, I hate PPPoE. It has all kinds of problems and no advantages. I really don't understand why SBC insists on using it.

Well, I'm in a very un-techie location in south-central-eastern US, and I'm paying $44/mo for a pretty damn fast cable connection, which is less than the cost of a phone line and dialup access (runs about $20/mo in these parts for dialup and $32/mo per phone line). Where are you getting this $600 figure from Jeff?

Well, to start with... 12 * 44 = $528 so you're pretty close, and you have a good deal.

My cable modem started at $45/mo (12 * 45 = $540) and Comcast just hiked it to $55/mo (12 * 55 = $660). My wife's SBC DSL is $50/mo I believe (we have a second house in Capitola shes using right now) (12 *50 = $600). So all in all $600/yr seems about average FWIK.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

Assuming PPPoE is still PPP protocol (which I assume it is) it actually saves you bandwidth on TCP/IP headers across that link. Its smart enough to move the static header data across the PPP link once and then just use a tag to tell it that future packets are going the same place.

This is why over PPP TCP/IP header costs drop to 8 bytes/packet as opposed to the 30 for UDP.

Quote

All in all, I hate PPPoE. It has all kinds of problems and no advantages. I really don't understand why SBC insists on using it.

AIUI the big advantage for them is that its easy to control/administer. (Why its easier then DHCP though I don't know.)

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

Well, to start with... 12 * 44 = $528 so you're pretty close, and you have a good deal.

My cable modem started at $45/mo (12 * 45 = $540) and Comcast just hiked it to $55/mo (12 * 55 = $660). My wife's SBC DSL is $50/mo I believe (we have a second house in Capitola shes using right now) (12 *50 = $600). So all in all $600/yr seems about average FWIK.

AIUI the big advantage for them is that its easy to control/administer. (Why its easier then DHCP though I don't know.)

For many providers, PPPoE primarily provides authentication without being tied to the consumer's MAC address. I have had a BB provider who did the "tied to MAC address" and it meant replacing a broken/faulty modem was a complete nightmare.

Most ISP's are accustomed to using PPP for their millions of existing customers, and have billing systems and staff expertise that work OK with PPP. I'm sure to many of them that PPPoE is a convenient upgrade path at relatively low cost.

AFAICS, PPPoE is actually rather good at making the billing process easy. IP billing is notoriously tricky as an alternative.

It seems to me there is no header compression, but I could well be wrong. Since PPPoE is primarily about billing and ISP's, there's little reason why header compression would be included (the ISP doesn't gain anything out of it!). I can't seem to find is any mention of whether PPPoE does header compression of IP packets...but then, I didn't look too deeply, and I've never worked with anyone who'd developed with PPPoE. Note, in the last source below: "PPPoE adds another six bytes of overhead, and the PPP protocol field consumes two bytes".

quote:Some key advantages of PPPoE and how they differ from PPPoA include:

Per session authentication based on Password Authentication Protocol (PAP) or Challenge Handshake Authentication Protocol (CHAP). This is the greatest advantage of PPPoE as authentication overcomes the security hole in a bridging architecture.

Per session accounting is possible, which allows the service provider to charge the subscriber based on session time for various services offered. The service provider may also require a minimal access charge.

PPPoE can be used on existing CPE installations that cannot be upgraded to PPP or that do not have the ability to run PPPoA, extending the PPP session over the bridged Ethernet LAN to the PC.

PPPoE preserves the point-to-point session used by Internet Service Providers (ISPs) in the current dialup model. PPPoE is the only protocol capable of running point-to-point over Ethernet without requiring an intermediate IP stack.

quote:PPPoE has many advantages for DSL service providers, and practically none for DSL consumers.

Because PPPoE sessions are really just PPP sessions, IP addresses can be very dynamic. There is no possibility to hold on to a fixed IP addresses by renewing a DHCP lease frequently. Service providers can ensure that your assigned IP address is changed each time you connect.

Because PPPoE creates the concept of a ``session'' over Ethernet, service providers can charge based on connect time. This allows them to discourage permanent connections and over-subscribe their IP address pool.

Because PPP sessions almost always require authentication, DSL service providers can bill the correct client regardless of where he connects from (as dial-up ISP's can now.)

In theory, PPPoE offers the following advantages to end users. In practice, these advantages are either negligible or not implemented by the service provider.

PPPoE can encapsulate non-IP protocols. Any protocol which can be encapsulated by PPP can be sent via PPPoE.

Service providers can enter into agreements with large organizations to authenticate users and provide dedicated sessions behind the organizations' firewall (for employees who need remote access, for example.)

No ISP that I'm aware of supports non-IP protocols over PPPoE, and access through a firewall is better achieved with SSH or IPSec.[/quote]

Well FWIW I *know* that analog PPP does header compression. It was important to us back at TEN.

Since PPPoE == PPP over Ethernet I'm assuming that the packet protocol is identical.

If you want to check, I've provided all the necessary links above! I couldn't find mention of it. If you are sure it's there, feel free to read the RFC and enlighten us...

I'm not sure if you've read the rest of this topic, but I think we've pretty much clarified that PPP does compression by now. The point I was making above is that the reasons behind PPPoE were not some desire to improve technology (as you said yourself, there's no real advantage to the consumer), and hence even the name is really just to reassure ISP's who are trying to decide how to implement their BB strategy.

There is, of course, a long and glorious history in the IT industry of deliberately misleading naming schemes .

Quote

I could be wrong but I wonder what the value would be in changing it.

Why add to the specification something none of it's implementors will ever want? It's a tech for ISP's, not for customers (except that, since it's cheap and easy to implement & maintain, theoretically customers will potentially benefit from lower cost BB access).

Why add to the specification something none of it's implementors will ever want? It's a tech for ISP's, not for customers (except that, since it's cheap and easy to implement & maintain, theoretically customers will potentially benefit from lower cost BB access).

You are assuming they started from scratch.

I'm assuming they hacked existing PPP code.

In either case it is indeed an assumption. *shrug*

In any case we are getting stuck on something which, as yourightly alluded, is technically interesting but performacne wise insignificant. If you have broladband you probably arent sweating an extra 22 bytes per packet.

Where the PPP handling of TCP really counts is right at the point where you need it most, across your thinnest pipes which bydefinition are your analog users.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

A real-life example of the imporance of counting the number of bytes per packet (from a Gamasutra article that just went up).

Points to note: 1. Xbox inflates the header sizes for packets 2. Elsewhere in the article it's revealed that Xbox's TCP implementation is fatally flawed: it drops a lot of TCP connections. This allegedly breaks TCP's "guaranteed delivery", although I suspect that they were in fact using the API's / protocol slightly wrongly (note the first para ). They claim the flaw is due to Xbox's memory-management, but I wouldn't be surprised if MS fixes this; it doesn't sound so permanent a problem as they make out.

"This is where I display my lack of experience in network programming, as I'm sure it will be common knowledge to anyone familiar with such things. At the start of the project I understood in an intellectual sense that packet headers could be expensive, but it wasn't until I started working out some actual numbers that I realized the scope of the issue.

A standard UDP header is 28 bytes, and a TCP header is 40 bytes. On Xbox they are larger, due to the overhead of the packet encryption and NAT traversal, requiring 44 bytes for a UDP header or 56 for TCP.

This may not seem like much at first, but consider the case of 16 players sitting in a lobby talking to each other. MotoGP uses a 40 millisecond granularity on the voice compression, which outputs an 18-byte chunk of speech data. 18 bytes times 25 packets a second times 15 listeners gives a total data rate of 53 kbps, but if you add in the UDP header sizes, this goes up to 182 kbps. That is an overhead of 243%, and makes the difference between meeting or missing our target of working over a 64 kbps connection.

Obviously, you don't actually need to send voice data every 40 milliseconds. The tradeoff is that the longer you buffer it up, the more latency you introduce, but also the less space you waste on packet headers. In MotoGP we send voice packets 4 times a second, which gives a 73 kbps data rate, 38% packet overhead. That's still slightly over our 64 kbps target, but not by so much that I can't get away with ignoring the math and pretending it is OK!

With so much of the total bandwidth wasted on packet headers, it is crucially important to minimize this in every way you can. Never send two packets where one will do. If you have a bit of data that you'd ideally like to send this frame, consider whether you might be able to hold it back for a while and then merge it in with some other information that is due to be sent in the near future.

Don't mix TCP and UDP. If you send some data by one and some by the other, you won't be able to merge packets, so you'll end up paying twice the header overhead. Even if you have to increase the payload size in order to do this, you can reduce the overall bandwidth usage by combining everything into a single protocol. Since TCP is unsuitable for game data and voice packets, in practice this means you should use UDP (or the Xbox VDP equivalent) for everything.

Once you have all your packets running over the same protocol, make sure they all use the same port, too. That minimises resource usage, and due to an optimisation in the Xbox network stack implementation, if you put everything on port 1000 you can save four bytes per packet, too."

The biggest one is that XBox live is broadband-only so unless hes being incredibly wasteful I don't really get why hes sweating bandwidth at all. I don't know where he's getting his 64Kbps target.

Next, he doesnt mention whether his VOIP is done with nagle on or off. If nagle is left on then hes going to get amalgamated packets which will significantly drop his TCP overhead. Given that audio is usually buffered anyway and can handle quite a bit of latency this would same the logical way to go.

And his "don't mix TCP and UDP" is too simplistic and off the cuff. There are legitimate reasons to use a combined strategy. There are some very sophisticated latency reduction techniques that depend on it (eg TEN's B.U.L.L.E.T.).

The truest thing he probably says is that he's not a very experienced network programmer, it sounds like he's approaching it all at a very simplistic and naive level

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

I've been re-reading this thread, now that i have a good set of reference books and such (TCP/IP Illustrated, Vol1-3)..

There seem to be some errors way back.

For instance.. blahblahblah wrote that TCP packets can be much bigger than UDP packets and therefore TCP was just as fast or faster. But looking at the specs I see that this is not really the case. large TCP packets would be split. In fact if ethernet is used somewhere along the way the actual transmission packet size is often limited to 1500 bytes a typical Maximum Transmission Unit - the larger UDP or TCP packet must be reconstructed from these, but it does mean multiple ethernet headers etc. TCP headers (not counting fancy compression with PPP) are larger, they have to be to allow it to implement the ordered delivery and such that you get with TCP.

In terms of raw speed something that wasn't addressed well are the throughput issues that the TCP stack has due to the need to process acks and re-transmit. This puts a theoretical upper bound on the transmission rate as a function of round trip time (ping) regardless of the speed of the channel.

Mind you I agree with all of the stuff about not trying to implement your own TCP-like reliability with UDP. For sure that's a waste. But it does make sense to look at alternative protocols and strategies where appropriate. For instance it is obvious that if you can take advantage of multicast there could be tremendous savings.

There are also considerations like not bothering with guaranteed delivery and acknowledgments if you are just going to send the updated state in a few milliseconds anyway. (E.g. there is an implicit re-transmit because the client must be kept up to date... you know that if the client missed a packet the next one is already on the way. Of course this only works if the packets don't depend of the client having an accurate picture of the previous state. But that's all the sort of stuff you need to consider when you are building a protocol for a game. Even if this information should be guaranteed, if you know the same state info will be sent many times then you should consider the probability that all of the packets will be lost and is it worth going for 100% reliability if the connection is that bad anyway. Maybe the problem having the client temporarily believe that things are not in the state they really are will sort itself out in a couple seconds anyway and that really won't effect game play as much as it would if they were lagged because the protocol was forced to be reliable. (E.g. while waiting to make sure the last packet gets through or simply be acknowledged a new more up to date packet is being blocked from delivery.)

Considering the ratios of header size to payload etc. It might be more economical to send the full game state instead of deltas.. Since sending a packet of just a few bytes and then the overhead of TCP/UDP (+ ethernet) headers doesn't give you optimal use of the transmission channel.

It pretty clear that using UDP properly is more difficult that using TCP. I think we need some articles about it in the wiki. There are so many things to consider.

I have pretty much no experience with networking protocols - that's why I bought the books... This is just my initial take on things after reading a few introductory chapters and looking at the fairly significant overhead in terms of achievable max packet sizes and headers at each layer of transmission.

large TCP packets would be split. In fact if ethernet is used somewhere along the way the actual transmission packet size is often limited to 1500 bytes a typical Maximum Transmission Unit - the larger UDP or TCP packet must be reconstructed from these, but it does mean multiple ethernet headers etc.

true

Quote

TCP headers (not counting fancy compression with PPP) are larger, they have to be to allow it to implement the ordered delivery and such that you get with TCP.

Not True for serial PPP. Serial PPP does header compression by storing the header info on the fat end of the pipe (the network end.) So if you are supporting 56K modem users TCP headers are actually SMALLER then UDP headers across the critical bottleneck, the last few miles to the user.

Quote

In terms of raw speed something that wasn't addressed well are the throughput issues that the TCP stack has due to the need to process acks and re-transmit. This puts a theoretical upper bound on the transmission rate as a function of round trip time (ping) regardless of the speed of the channel.

Not True. The sender doesn't wait for ACKs. Its a NAK based protocol (at least in all modern implementations.) The sender just keeps pumping packets but keeps a "sliding window" of history of past packets. When it gets a NAK on an old packet, it retransmits from that history.

Quote

Mind you I agree with all of the stuff about not trying to implement your own TCP-like reliability with UDP. For sure that's a waste. But it does make sense to look at alternative protocols and strategies where appropriate. For instance it is obvious that if you can take advantage of multicast there could be tremendous savings.

Reliable multi-cast protocols can be very useful within the datacenter. On the internet at large Multicast is still a dream.

Quote

There are also considerations like not bothering with guaranteed delivery and acknowledgments if you are just going to send the updated state in a few milliseconds anyway.

Yep if your message doesn't matter, send it parcel post

Quote

It pretty clear that using UDP properly is more difficult that using TCP. I think we need some articles about it in the wiki. There are so many things to consider.

Yep. And trying to recreate TCP's attributes is silly. A lot of VERY VERY smart networking folks have worked on TCP, and the network knows it and can do things to help.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

There are also considerations like not bothering with guaranteed delivery and acknowledgments if you are just going to send the updated state in a few milliseconds anyway.

Yep if your message doesn't matter, send it parcel post

IMHO this is still the most important point. Guaranteed delivery allows NOT to send 'unimportant', discardable data. Or only to send IMPORTANT data.

And NOT sending anything is the by all means FASTEST protocol you can think of!

Another point - that I really dislike - is that brute-force UDP transmission wastes a lot of SHARED resources. It is in no way a computationally minimal approach. (for the same reason I dislike simple desktop games that run with 200fps in a tight loop although nothing moves)

Reducing network load to the bare minimum in my eyes is a fine art. I love that. Cannot do it with unguaranteed delivery.

No. TCP or UDP still is application specific. And UDP is the better choice to setup own, task specific protocols (like Quake or NFS e.g.).

Maybe TCP is not the best protocol for games - but in my eyes it comes pretty close. Designing an own, guaranteed protocol can be extremely difficult (as Jeff mentioned) and the common approaches often taken in the games area turn out to be too naive and end up being very unefficient and yet not as reliable as TCP is. Doing it is just difficult.

And it IS a matter of taste. Quake is successful with UDP - bc. Carmack does not feel bad with flooding the network with redundant data. I do.

So far TCP did not cause any problems for me. My flightsim sends 1-3 messages per second. This is possible bc. I can be sure they will arrive. And I love to have a true algorithmic approach to avoid redundant data transmission. Latencies are there - as they are with UDP. Maybe higher maybe not. I have to deal with that anyway.

Writing a TCP-like layer over UDP is actually not as hard as it sounds, because the people who write the TCP specs also document the algorythms for everyone to see.

I have recently written my own MMOG network layer (to support 1000+ clients on a couple of ports) using the TCP SACK system outlined in 'RFC 3517' (gogle for it, as it also contains links to the previous relevant RFC's), and was quite pleased with the results and the time it saved using these docs.

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org