reliable - things that the player has to know about
unreliable - things that the player should know about, but no big deal if it is missed from time to time
local - things that do not impact other players

For reliable messages (you can send multiple messages in a batch as well), you will most likely need some sequence numbers so that all updates are applied in chronological order and that both client and server are always kept in sync. If the player is trying to loot something, then the player would send a reliable message, if the server does not ack the reliable message (packet loss) then player has to resend the request (or timeout). Neither the player, not the server change their state until ack is received (for reliable messages).

Unreliable messages are things that can be missed once in a while, like players movement, the direction where the player is looking, the appearance etc. If the player loses these packets, at the next update, this information may be resent later, no big deal. If no ack is received, then the server will eventuall resend the unreliable update, if ack is received for unreliable message at some point then the server will stop sending the unreliable updates. Also if the player changes position, there is no need to resend the old position, just resend the latest state until player acks it.

Local updates are things that each player may see differently but have no impact on the game, like animation of trees, weather effects, things for which a global state does not need to be kept.

Some other things worth considering :

Do not flood the network:

You do not want to send unreliable updates every time a player moves (ie packet per move), that will just flood the network, you want to send them at a certain interval (like having fps and sending the update every 10-15 frames). Also if you send every unreliable event, you will slow down the performance of your server, so you may want to filter out how many unreliable events you send. If there are 500 players in the room, moving around, you may not want to increase the refresh interval and or prioritize which state updates are more important.

Path prediction:

The client should have a prediction built in for the movement so at frame 0 you can say the player is moving towards this coordinate and at frame 10 you can say that the player has changed position and direction, the client has to interpolate what happens between frame 0 and frame 10.

Timestamp:

Ideally you want to use sequence numbers for tracking game state, every time you send any messages there should be a sequence number (for resending purposes), reliable messages should have another sequence number on top of your packet sequence numbers. You can also include timestamp (simulation time) every few packets.

Protocol/structures:

You want to pack your data as much as possible (so that you can send more information in less time). Use tricks like bitvectors to store flags. Instead of strings send string ids (unless its chat). You will most likely have large switch statements for encoding/decoding, you may want to build a tool to autogenerate these or use something like google protobuf.

Error handling:

If the player sends a reliable request, then the player can either get a valid reply or an error reply (or timeout and disconnect), for example if the player wants to purchase an item, he can either get a reply saying item is purchased or an error that the item is not available or has already been bought by somebody else, until any of the possible responses is received, the player should do nothing (no state update).

Does the player even need replies from the server, when nothing went wrong? There are probably updates from the server which get triggered by the player actions, but a "ok" reply for all actions seem's a little excessive.
–
K..Aug 22 '12 at 15:00

Where could I find those Quake and Unreal network docs?
–
joltmodeAug 22 '12 at 15:14

2

@da_bOuncer : Acks do not have to be sent separately, they can piggy back with the messages that are being sent to and from the server. So you may send a message like [my sequence] [ack last sequence] [unreliable] [reliable] etc. So the minimum message you can send is [my sequence] [ack last received sequence]. The acks are important for both reliable and unreliable as the client needs to know if the server got the latest state change or not, if there is no ack, then you have to resend otherwise you might get out of sync.
–
Vitaly OmelchenkoAug 22 '12 at 15:27

I see, but since I have to use TCP (the game should run in a browser) I get the "ack"s automatically, I guess. Also the reliable/unreliable stuff is probably none of my concerns, since all TCP packages get through.
–
K..Aug 22 '12 at 19:04

Every action the player does should be sent to the server with as much data as needed for the functioning of the game, but not more, to ease on the network traffic. Every data needs to be verified by the server to be a valid action. Invalid actions are logged for review, since they indicate bugs or cheating attempts.

The one overarching rule is to never trust the client with anything of importance: The client is in the hands of the enemy. This includes the time stamp, where the server should always dictate it (and thus the order of actions). What the packets should have is a serial number, so that the server can detect network problems easier and so that actions of a client happen in the same order as the client requested it. Two packets with the same serial number but different content also indicate a hacking or cheating attempt and should be marked as invalid and logged as above.

The state updates on the server have to happen atomically, though in practice this requirement is hard to fulfil without severe performance penalties, so most games opt for "locally atomic" instead (updates which happen "far away" and can't influence the state of the actor don't need to by synchronised with actions of the actor). This is a consistency/performance trade-off. Problems related to it have been a source of exploits like duplication bugs even in major games in the past.

The server then gathers the updates for each client into a per-client queue (usually the size of your networking data window, so something on the order of 1.5kb) and sends them after either a specified timeout or when the queue is full. The updates are bundled together in one packet - your system needs to be aware of it and handle multi-update packets. The server never sends client packets unfiltered to a client, not even to the same client. In particular, any text has to be filtered to include only the "safe" character sequences and any updates which the other client shouldn't be aware of aren't sent at all. Any actions which failed, for whatever reason including an invalid action, get a "failed action with serial number XXXX" notification sent along with the updates to the client which originated it.

so you mean, the client should correct the problems by itself?
–
K..Aug 22 '12 at 12:17

1

@da_b0uncer: For most cases, the client should make the player aware that an action they tried to perform failed (usually done with audiovisual clues). This is for actions like "Adam, picking up the Forbidden Fruit failed because Eva just did it first." In other cases, it strongly depends on what the nature of the action and error is - sometimes, even crashing with a meaningful error description and stack trace (to be sent to the developers) is better than trying to recover from an invalid internal game state.
–
Martin SojkaAug 22 '12 at 12:21

I see, the problem I see is the ping, if I got a >300ms ping, it could happen that Adam picked up the fruit and walked away and later the "error" would reach him.
–
K..Aug 22 '12 at 12:32

1

@da_b0uncer: No. In an MMO, the client always waits for the server to validate and accept the action (and send the correct update for it) before allowing the next action which depends on it. The only exceptions are movement actions, where the client can pre-emptively start to move the actors - but even then the server updates override that, which leads to "rubberbanding" and similar effects on laggy clients.
–
Martin SojkaAug 22 '12 at 12:39

1

@da_b0uncer: It depends on the pacing of the game. If the game usually has a mean time between actions measured in tens of seconds, minutes or even more, this isn't a problem at all. Same goes when very few actions can actually fail in such a way as to prevent further actions (for example, "talk" actions are of such a non-blocking kind) - here the client can queue up actions without waiting for the server response, though it should still show the character performing them after the server OK'd the action in most cases.
–
Martin SojkaAug 22 '12 at 13:00