I'm asking this question here since I think people on this forum
probably have the best knowledge of firewalls and routers.

Say I'm designing a multi-player game and I want to push data to the client spontaneously. Since it's probably more efficient and puts less load on the server
than clients constantly polling the server to check for changes.

From my understanding, most computers today have firewalls and a lot of home computers are also behind a router, so if you just send packets it will probably be blocked especially since the router won't know which PC to route the packet to.

What are good ways to push data without being blocked?
In what situations do firewalls allow packets through on PCs with default configurations? and how will the router know which PC to route the packet to?

I hope this question isn't too big, I think the questions I asked are all related otherwise I'll delete the last question about the router since it might be less related.

EDIT: I say spontaneously sending packets but I mean only after the user has logged in to the game and sent his IP.

EDIT 2: (Answers to comment)

The game is over the internet.

How many traffic - initially it will have maybe a hundred concurrent users but should be scalable for several hundreds, but I'd still like to hear solutions for both cases because I might decide to have more servers if it changes the answer.

Can you add some further details? How many clients, over Internet, how much traffic, streaming, packet loss, etc. Is this time critical (latency) for near-real-time, etc.
–
david6Nov 21 '11 at 20:57

I edited the question to address your questions, about Streaming and Data loss - I'm not too sure what the question is here, and I'm not too fimiliar with the ways and consequences of implementing streaming
–
fiftyeightNov 21 '11 at 21:07

4 Answers
4

Universal considerations

It is common for most firewalls and NAT boxes to allow a server to reply back to the machine that sent a packet from the inside. Anything else is an edge case or insane behavior. Therefore, if a game client can initiate a connection to you, we can assume that you can talk back on that channel. The goal then is to keep the connection alive. With both TCP and UDP, NAT boxes will eventually time-out the application if nothing is ever sent.

UDP

Simple and straight-forward, most NAT routers will allow UDP packets to go back to the host that sent them. In real-time games, this is the normal protocol so that frame re-ordering isn't performed on dropped packets. The game and its server are responsible for synchronizing their states from within the protocol. Expect the connection to be forgotten after a minute of silence. DD-WRT boxes default to 2 minutes.

TCP

The overhead of keeping connections ordered is shifted from the game to the system. Packets arrive to the application in order, though this can create some unwanted delays when packets drop or arrive to the computer out-of-order. At least by RFC, NAT boxes should keep an established TCP connection open for at least two hours before forgetting them. Realistically, count on no more than 5 minutes. DD-WRT routers, for example, forget in 10.

Great explanation, thank you, I always heard the term "TCP connection", does a "UDP connection" act the same way? other than the fact you mentioned that it times out after one minute (and uses the UDP protocol of course)?
–
fiftyeightNov 22 '11 at 0:19

@fiftyeight Sorry, lapse... UDP is connectionless in that it doesn't have a state between packets, so the server / client OS is never keeping track of things. That works great for DNS, NTP, etc. where there is a simple question / answer exchange with no special timing. The NAT system does need to keep track of the fact that a "conversation" is happening for packets to go to the right place, though. Thus, the connection tracking.
–
Jeff Ferland♦Nov 22 '11 at 0:42

OK, so if there is no keeping track of the state, can you explain what you mean "the connection will be forgotten after 1 minute"? I assume routers will only accept packets from IPs to which packets were previously sent, do they "forget" after I receiving one packet? can the server send only one packet for each packet the client sent or can the server send as many as I want?
–
fiftyeightNov 22 '11 at 0:46

In my example, the router will forget where to send packets to two minutes after the last UDP packet transits it in either direction.
–
Jeff Ferland♦Nov 22 '11 at 1:31

OK I think I understand, thank you, UDP sounds very appropriate for a game though, it's probably rare for the client and server to not communicate for a whole minute. I think I read somewhere that routers tend to block UDP packets more than TCP but it's probably a non-issue if DNS counts on it.
–
fiftyeightNov 22 '11 at 1:36

You do not send packets out of the blue; you have to know who the client is and where it is. So the normal model is the following: each client opens a TCP connection to the server. The connection just sits there. When the server has something to say to the client, it just writes it on the relevant connection.

An idle connection entails no packet. You actually want to send some packets from time to time, because some router/firewalls tend to kill off connections which have been idle for too long (with "too long" being sometimes as short as "five minutes").

In some networks, external TCP connections are denied, and all network activity must go through a proxy under the HTTP protocol (I have seen that in some business networks -- exactly the situation where asking for special firewall rules in order to play a game might be somewhat frowned upon). The usual workaround is to go HTTPS (i.e. SSL), because then the proxy will work as a generic bidirectional TCP forwarder (with the CONNECT proxy command). If that's blocked as well, you will have to resort to trickeries such as sending a dummy HTTP request, and having the server send out an "unlimited" response (a response without an explicit 'Content-Length' header) by small chunks. See this blog post for a discussion.

Thanx I edited my question to make it clear I don't mean completely out of the blue, only after the client logged in. Also: Do routers and firewalls read HTTP headers? (in order to implement the "unlimited response" trick), why will they not block this?
–
fiftyeightNov 21 '11 at 20:59

@fiftyeight: some proxys filter content, and thus read HTTP headers and even scan the actual data. If the proxy is bent on blocking your specific application then it will succeed. However, such a proxy would probably block 99% of the Internet as well, or kill browsing performance (e.g. to prevent the "unlimited answer" trick, the proxy must temporarily store the whole response, which could mean several gigabytes -- most proxys will not do that). That's still an hide&seek game you are playing.
–
Tom LeekNov 21 '11 at 21:09

Overall you say the best way for most home PCs is to initiate a TCP connection and keep it open by sending packets every few minutes?
–
fiftyeightNov 21 '11 at 21:20

1

@fiftyeight: yes, that's what I would recommend. TCP is a rather proven technology. Home routers are all about allowing TCP connections as long as the client is on the inside (that's what NAT is about).
–
Tom LeekNov 21 '11 at 21:45

The general issue for home routers, or firewalls, is that they allow outward connections and disallow inward.

Each brand / model has generally the same capabilities to allow a bypass or inward port mapping. But, the exact method (GUI, settings) can be quite different (and therefore complex).

It would certainly be safer to have the clients connect (to server). But to still primarily send (from server), you could have the client first establish an outward connect, and then listen for packets over that path.

"It would certainly be safer to have them connect (to server). But, you could first establish a secure outward connect, and then listen for packets over it." - I'm not sure I understand the difference between these two. If you mean secure vs. insecure, it doesn't need to be secure, it's just game data, no credit card numbers or such things :)
–
fiftyeightNov 21 '11 at 21:21

a very interesting answer. thank you! It's not a browser game so there isn't a need for comet which from what I understand is a solution for server push to a browser, but the idea of using port 80 on the server is interesting, though I wonder if it would help if I use UDP (any idea?), and the node.js project also looks very interesting, especially since I love javascript.
–
fiftyeightNov 22 '11 at 17:37

If you're using UDP, I wouldn't expect much difference between ports :(. OTOH you don't have to use a browser as the client nor HTTP just because you're using port 80 (although HTTP would allow for connecting via an application proxy). Node.js still makes a lot of sense for this kind of architecture, regardless of UDP/TCP/port/protocol
–
symcbeanNov 25 '11 at 9:14