This Is A Custom Widget

This Sliding Bar can be switched on or off in theme options, and can take any widget you throw at it or even fill it with your custom HTML Code. Its perfect for grabbing the attention of your viewers. Choose between 1, 2, 3 or 4 columns, set the background color, widget divider color, activate transparency, a top border or fully disable it on desktop and mobile.

This Is A Custom Widget

This Sliding Bar can be switched on or off in theme options, and can take any widget you throw at it or even fill it with your custom HTML Code. Its perfect for grabbing the attention of your viewers. Choose between 1, 2, 3 or 4 columns, set the background color, widget divider color, activate transparency, a top border or fully disable it on desktop and mobile.

Clock synchronisation

It's important that both the client and server's clocks are synchronised because if there is any time based interpolation, you want both server and client to agree on what time it is and therefore at what position your interpolated object is.

Asteroids on an interpolation orbit

I use this technique in 2D Space MMO to ensure the orbiting asteroids are in the same position across all clients and on the server. The asteroids are actually interpolating on an orbit around a central location - there are no update messages getting sent to correct their positions, the only thing which keeps them synchronised is having the same time value across all clients and server.

Before we can try to synchronise clocks, we need to be sure we're using the same concept of time on both server and client. We want to calculate the number of seconds since 1970/01/01, which is the same in both javascript and actionscript:

vartime = newDate().getTime()/1000.0;

var time = new Date( ).getTime( )/1000.0;

The way to synchronise clocks is to first work out how long it takes a message to do a round-trip from client->server->client, like a ping, essentially. We do this by sending a message which contains the client's local time. The server will then reply with the exact data it received and also an additional parameter which is the time on the server.

When the message arrives back on the client, the client can subtract the old client time in the message from the time 'now' on the client, thereby arriving at a total round-trip time. Taking half of this round-trip time we arrive at an estimate of the all important latency value. Latency is how long it takes for our messages to arrive on the server.

Using the second part of the reply from the server (the server's time), we can then compute an offset which will account for any time-zone differences.

In reality we don't just do this synchronisation once, because it is subject to error based on the current quality of the connection between the client and the server - imagine the client is on a 3G network travelling on a train, the round-trip time might vary considerably from attempt to attempt based on the proximity and line of sight to the nearest cell tower. In order to combat this problem, we continuously synchronise and take an average value.

In this code I'm using a simple moving average. You might want to do something more advanced like store more data points instead of just the last value and then filter out any outliers by comparing against the median of the dataset.

The latest sample was identified as an outlier and can be discarded

Flash and the socket policy file

Before we go any further it's worth mentioning how Flash works when connecting to a socket server. The Flash client will send a request for a policy file when it first connects to a socket server; this policy file tells Flash which domains and ports the socket server accepts Flash connections on. You can read more about socket policy files here.

Flash will send a request which looks like this:

<policy-file-request/>

It will do this first on port 843 and then on the port you chose to connect with the server.

The server must respond with a valid policy file which must be terminated with a \0 null character. Here is the function I use to form the policy response in node.js:

I've found that although the majority of Flash clients do request a policy file, there are some which wont so you have to be able to deal with both cases to be completely robust.

Synchronising state in a twitch reaction game

At this point it's important to talk about exactly how we can go about designing the system to be responsive enough to handle a twitch reaction game. There are primarily two different ways of synchronising state in a multi-player game (by state I mean position, orientation, velocity and other attributes of in-game objects):

Periodically send updates about all objects which have changed state

Send updates only when an event occurs

The first method sends data at a fixed interval about all objects which have changed state. This introduces a lag which is equal to the time between state updates, but ensures a steady flow of state information which is not subject to flooding.

The second only sends state when an event occurs, such as a key being pressed. This has minimal lag but can be subject to flooding; for example if the player starts hammering keys.

I chose to use an event based method because the response times are important in a twitch reaction game, and also because it allows me to handle the problem with bullets.

The problem with bullets

Imagine if every single bullet fired in game caused a message to be broadcast? This would soon overwhelm the server's available bandwidth and start leading to nasty lagging issues. To solve this problem I simply send key-presses from client to server and give the weapon auto-repeat so that the player is either firing or not firing based on which keys are currently down on the client.

If I detect key hammering in the game (which is quite a natural behaviour for this type of game), I put up a message telling the player they can hold down fire instead.

When the player is firing bullets, bullets are emitted at a constant rate on both client and server and there is a synchronised timer which ensures that both client and server fire bullets at the same moment in time.

Client side prediction and server authority

Does the client send its state to the server, or does the client send inputs only and simply await the new state from the server?

The former approach is lag free and responsive but presents a serious problem with trust - the client could have been altered by cheaters to gain an advantage in game. Imagine if the client had authority over kills, for example; an altered client could simply kill everyone else in game by telling the server it had hit everyone a thousand times, or it could upgrade the player with super-speed etc.

The latter avoids these problems but introduces an unacceptable amount of lag which is equal to the message round-trip time.

As I mentioned in the last article, although you don't need to worry about cheaters until you actually have a popular game, it's worth taking preventative steps ahead of time. The traditional method is to simply have the server be in authority over everything in game and to cope with the lag by doing client side prediction.

Here are some reading materials which cover client-side prediction and complete server authority:

I'm going to talk about a different method in this article. It has a number of advantages:

Incredibly simple to understand and quick to get running

Player never sees their character getting warped around or stuttering due to lag

Enables complex interactions with rigid bodies

Enables the game to be played acceptably over a 3G network

This method has server-side authority over everything in game, except the player's position and angle, which are client sided. This obviously raises a trust concern with cheaters teleporting around the level, so to mitigate this the server (and other clients) simply clamp the maximum speed of each player. Then if a player teleports, it's equivalent to them moving there at normal speed for all other clients and the server, thereby negating the benefit of the cheat.

In order to implement this technique I send the client's current position and angle along with key-presses.

When this data arrives at the server (and the other clients) a delta is computed between the current position and this new broadcast position. A fraction of this delta is then applied each frame to the position and angle, and then the delta is depleted by that amount.

Source moves 50% of the remainging way towards target each time

So the correction is applied most at the beginning of its application and least at the end. In fact it never reaches the end, technically.

It should be noted at this point that this is indeed only used for correction purposes; both the client and the server all run the same game code which responds to key-presses. All objects are integrated in the same way across all systems so client and server will be roughly in sync without this correction. The problem is, 'roughly' is not good enough and they will start to drift off without it.

The problem with drift

Although the same game-code runs on client and server, which responds to key-presses in the same way, integrates objects forward in time the same way using the same code, you will still get problems with drift if you do nothing to correct it.

Drift occurs due to a number of factors, the most obvious being accidental differences in the code between client->server, or differing mathematical result of calculations due to hardware or language specific implementation details, or a non identical time-step between client->server.

The other one is due to the unpredictable nature of lag. For example, the client holds down the forward key for 30 ticks and then releases it. That's two different messages getting sent from client to server, one key down and one key up. Due to lag, the server will receive these two messages at some time in the future, the problem isn't that; the problem is that if the lag changes from key-down to key-up, the number of ticks the server will have the forward key down for could be different than the client!

Difference in key-down duration client->server due to lag variation

In the above example a key is held down for a consistent number of ticks each time, but the lag varies on each attempt resulting in a inconsistent key-down duration on the server. This wouldn't be a problem if there was a constant amount of lag, but in reality it varies enough to cause problems.

By sending the client's current position and angle with each key-press this drift can be corrected as described before it gets significant.

Simplicity

The beauty of this technique is that it's incredibly simple. The player's position/angle is always slightly ahead of the server because of the nature of this system, so that when the correction deltas are computed they never represent a step backwards in time, like they would with a fully server authoritative technique so always appear smooth and un-jarring.

Of course, this technique does come with its own disadvantages which it is only fair to discuss:

If the player is very fast moving, the difference in position on client->server due to lag can be large enough to cause problems with hit-detection

A player with a very laggy connection will appear to behave oddly for all other clients observing

The maximum speed of the player must be clamped to mitigate cheating

That being said, from a business point-of-view the thing most likely to stop your game being a success is it not being done in time or at all. As long as the compromises are acceptable, any time you can save during development which could be spent on actually making the game better is extremely valuable.

You can read more about my love of simplicity in game development in this article on how to make games.

Integration and the time-step

I've already mentioned that the client and server must run the same code. It's worth talking about the time-step, because although we have a very robust correction system to deal with drift, in reality we want those corrections to be as small as possible.

With a variable time-step, the amount of distance actually moved over a long series of frames will be different each time even with a constant velocity. This is especially noticeable with the jump-height of a character in a platform game; I've seen it be pronounced enough that a character won't be able to make a jump in one part of the level which he has no trouble making in another for the same height.

Acting on key-presses

Ok, so lets talk about exactly how I transmit key-presses and what the server does with them.

Firstly, the key-codes from the client's keyboard are transformed into in-game actions and packed into a bit-field. You can read more about bit-fields in this article I wrote about understanding binary.

The reason these inputs are processed inside the Integrate function is due to the integration method I'm using, which compensates for frame-rate variation by calling Integrate multiple times per frame depending on the adjusting time-step.

Other in-game events

As I mentioned before, everything except player position and orientation is server-sided in this article. As such, events like:

Time synchronisation ping

Player takes damage

Player died

Player respawned

Player joined

Player left

...are all handled by the server broadcasting to the clients. Each message has its own message type and the client has a message processing section which deals with the data and actions represented in these messages.

The server has a similar message processing section and deals with messages from the client, which include:

Time synchronisation ping

New player ready for world state

Player pressed a key

Player chat message

Player entered his real name

Player connected

Player disconnected

Game state on player join

In this simple example, when a new player joins the game, the entire game state gets serialised and sent to the joining player and all other players get a message that a new player has joined the game.

/**
* Process all messages received from all clients!
*/
OnMessage:function(client, message){var thisPlayer =this.m_players[client.m_uid];switch(message.n){
...
case MessageNames.kReady:{if(this.m_numPlayers< GameLogicConstants.kMaxPlayers){//// client is ready! serialise the world!//// get data for worldvar world =[];for(var key inthis.m_players){var player =this.m_players[key];
world.push( player.GetData());}var defaultName ="Guest"+utils.PadNumber(client.m_uid,4);var newPlayer =new Player(client.m_socket,this.FindRandomSpawnLocation(), client.m_uid, defaultName);this.AddPlayer(newPlayer);// new player data for serialisationvar pd ={m_pos:newPlayer.m_pos, m_uid:client.m_uid, m_name:defaultName};// send this data to the client
Message.SerialiseAndSend(client.m_socket, MessageNames.kReady,{m_world:world, m_you:pd});// broadcast new player to all clients, except the one who is readythis.m_server.BroadcastExcept( MessageNames.kCreatePlayer, pd, client.m_uid);}}
...
}}

This technique works fine for this simple game, but if you have a much larger world, or many more players it will start to get bulky and expensive on bandwidth. You might want to look at techniques like Interest Management to handle this problem in larger games. I use Interest Management in 2D Space MMO.

Each object in game has a method called GetData() which returns the complete state required to create the object from scratch on the client. Here is what this looks like for the players:

You can see there are the vital details for the player, including his UID which uniquely identifies this object on the server, what keys are down now and which were down last frame, the bullet timer which synchronises the firing of bullets between client and server, health, number of kills, number of deaths and his name. The position, angle and velocities are actually stored inside the m_mod member which is inherited from the base class which describes all moving objects.

When the joining client receives and has deserialised this array of world data, the game state is synchronised and any further update in state will be handled by the normal event based messaging system. For the other clients present a more simple set of data is serialised which describes the new player:

// new player data for serialisationvar pd ={m_pos:newPlayer.m_pos, m_uid:client.m_uid, m_name:defaultName};

Because the joining state of all players is identical, bar the UID, the name and the position, these are the only things which are necessary to serialise to the existing clients. For example, all new players start un-rotated, all have full health, zero kills and have died zero times so there is no need to serialise that data - new players have defaults for those values.

Chat

No multi-player game would be complete without some way for players to chat with each other. Luckily implementing a chat system is very simple; chat messages are read from a text-box on the client, sent to the server and then broadcast to all clients, where upon the message is displayed at the appropriate location in the chat window.

The only thing to worry about is filtering the text for profanity. This can be done on either client or server, I've chosen to implement this on the client. Text must be filtered before being displayed on reception of the message rather than before transmitting it from the source. The reason is simple: imagine a hacked client exists where the hacker has removed the chat filter; he would be able to type unfiltered messages which then get broadcast to all clients!

It replaces sub-string occurrences of profanity with the asterisk character and returns the filtered string.

It's worth pointing out that it makes sense to store any data unfiltered (in a database, for example) because you are then free to change the filter and will still have the original data to work with.

As well as chat, you can display game events in the chat window, as in this example whereby kill events are displayed with the chat.

Chat text with game events

Bots

There are two bots in the live demo which accompanies this article, they are server-side bots so are genuinely equivalent to other real players in all things accept their level of skill... in most cases!

They are handled by creating a fake client for each bot which runs server side. Each bot creates a socketed connection to the server in exactly the same way a genuine client would, but the code the bot runs is comparatively tiny compared to the full client.

As you can see, the full extent of their AI involves sending a completely random key-state change at a random point at a probability of 1/51! Given how simple this is, I'm constantly surprised by how realistic they can seem. It goes to show that random events can go a long way toward emulating real behaviour in some cases.

Bots are created when the first player joins the game and are destroyed when there are zero real players left. In a PvP only game, it's important to have bots because a lot of the time there wont be that many real players playing, which will lead to a bad play experience and bad reviews.

Player personalisation

It's very important to be able to let players personalise themselves in a multi-player game. The simplest way to do this is to allow them to enter their own nickname. In this example it's handled the same way as any other message; it gets broadcast to all clients who then update their copy of the changed name. The server also stores this new name so that subsequent players will have it serialised to them when they join. The name is displayed on each player after being run through the profanity filter!

Avatar customisation in 2D Space MMO

Simple stats tracking can also help make the game more fun, like a way for players to compare themselves against others. In the demo accompanying this article the number in brackets after the player's name is the number of times the player has died subtracted from the kill count.

Version control

Once you've made your multi-player game and distributed it onto the portals, it's very important that you are able to update it and fix any bugs you find without having to manually locate and alter each copy of the game on every portal. You can do this by creating a boot-loader which loads the main part of the game from a fixed location which you control. Of course there are technicalities associated with this, not least of which is browser-caching. You can read more about how to solve this in this article I wrote a while back.

Buy the source code

As ever you can buy the source-code accompanying this article! Your purchases help me to be able to continue writing articles like these.

It will give you the complete prototype as shown in playable form above, with both server and client code. You will need either Flash Develop or Amethyst to build the client-side code and of course you will need node.js installed to run the server-side. You will also need Flex SDK version 4.5.1 or above. If you would like to edit the assets included with this demo, you will need Adobe Flash CS4+. Note that you cannot build the client side code with only the Flash IDE.

It comes in two versions, a personal edition which you are free to use for your own, non commercial purposes and a commercial version which allows you to use the code in any number of different commercial products or games:

If it seems expensive, bare in mind that it took a couple of solid weeks of programming to produce, which would have been around $3500 if I were contracted... Not to mention the many weeks and days it took to arrive at a solution powerful enough to handle a twitch reaction game over a 3G connection!

Share This Story, Choose Your Platform!

A games industry veteran of ten years, seven of which spent at Sony Computer Entertainment Europe, he has had key technical roles on triple-A titles like the Bafta Award Winning Little Big Planet (PSP), 24: The Game (PS2), special effects work on Heavenly Sword (PS3), some in-show graphics on the BBC’s version of Robot Wars, the TV show, as well as a few more obscure projects.
Now joint CEO of Wildbunny, he is able to give himself hiccups simply by coughing.
1NobNQ88UoYePFi5QbibuRJP3TtLhh65Jp

11 Comments

A data storage tip for your readers: if you’re going to store data the user has entered, think about the ‘type’ and the ‘filters’ separately. For example, a name might be alphanumeric (type) and no profanity (filters). Before saving on the server, correct the type but don’t apply the filter. Apply the filter when you’re displaying it on the client. That way if you later change the filter, or if you allow users to turn on/off the filter, or you want to display in a different context, you still have the original data and can transform it again.

You should consider writing a book exclusively about the networking part of multiplayer games (design, development, maintenance etc). Of the dozen game development books I’ve seen, the networking part is always lacking, usually textbook copy-pasted and new developers have complained about it to death. The market is there. Capitalize it.

I’m not sure I fully get the need to sync the clocks. Is it purely to work out the lag?

If the client knows the lag is 250ms and receives a key event from the server, should it skip the player ahead to where it would be if it received the key event 250ms earlier? Or does the drift adjustment account for that?

Good question – the answer is that although the latency value is never used with key-presses it can be very useful in other cases, especially where there is a long-term interpolation taking place. There isn’t an example of this in the demo, but as I explain in the article I do use it for calculating the positions of the asteroids in 2D Space MMO and I also use it when the UFOs are created; because they move at constant speed in a constant direction their positions would be off if I didn’t extrapolate their position when the message is received from the server.

Hi there! Great article. I’m trying out your solution to server authority and wondering if you have any suggestions on coping with the difference in velocities (in the integration step) this technique introduces.

You mention that sending along the current position mitigates the problem with drift, which would be true if the movement integration had fixed velocities. However, this is rarely the case as any physic /game engine would calculate new velocities for each integration step, for example react to keyboard inputs by gradually increasing the velocity of the player (accelerate). By sending along the current position, you only make sure that the position is corrected, but the difference in time due to a non constant latency would mean the different clients would let the keyboard events “exist” and affect velocities differently.

I guess this isn’t a huge problem if the velocity is clamped at a relatively small value, but say we have a long acceleration curve before reaching the max velocity, the drift could lead to all sort of problems.

Am I missing something? Looked through the source code and can’t really find anything that accommodates for this. Or is the effect usually negligible in a real world scenario? Any ideas / suggestions would be greatly appreciated.

It really depends on what the range of different velocities are – if you have a fairly low maximum velocity, as is the case in the demo, this isn’t a problem. In general its only a problem when the player is doing a lot of velocity changes as it will eventually correct itself anyway.

In the demo the effect is negligible, but you may have very different requirements 🙂

I have some slightly different requirements yes (more interaction between players which make things a bit more complicated). I ended up sending along a frame delta (the difference in iterations of the integration loop between the previous and next event) with the events. Then I just integrate the velocities at the receiving end based on the difference between the actual / real frame delta and what’s already been run. Seems to work.

I should add that I’m testing with a simulated latency which is probably set to be worse than what most people would experience, so I might be borrowing trouble here 🙂

About the author

A games industry veteran of ten years, seven of which spent at Sony Computer Entertainment Europe, he has had key technical roles on triple-A titles like the Bafta Award Winning Little Big Planet (PSP), 24: The Game (PS2), special effects work on Heavenly Sword (PS3), some in-show graphics on the BBC’s version of Robot Wars, the TV show, as well as a few more obscure projects.
Now joint CEO of Wildbunny, he is able to give himself hiccups simply by coughing.
1NobNQ88UoYePFi5QbibuRJP3TtLhh65Jp