July 11, 2012

For the past few months I’ve been working on a game I’d like to call “Alpha One”, but is still called “Duck Hunt” for now.

Exciting top-down action! Programmer graphics! Unfinished gameplay!

I’ve been working on it in the train, with a mindset of: just get it done. I’m not really bothering with architecture all that much, I just want a working prototype. And that mindset is necessary, because every day I only get an hour and a half, split in two, to work on it.

For the past month I’ve been working on storing the game state to disk. This due to the advice of @ivanassen, who worked on Tropico 4 (and a whole slew of other games). His advice was to start working on the subsystem that stores the game state to disk as soon as possible, because it touches everything.

What I’ve found is that he’s absolutely right.

So, what do I need to store to disk for Alpha One?

Background – My background is divided into layers, each layer containing items. These are static and won’t change throughout the game.

Camera’s – Right now I have only two camera’s: one for the game and one for the editor. But I would like to store their position and orientation in the game save as I might add more camera’s later.

Lights – I don’t have any right now, but I definitely will in the future.

Objects – Everything that’s moving and interacting. Right now I have only three classes: Player, Enemy and Bullet. And even that has proven to be a headache.

XML is terrible for a lot of things, this is one of them

My first idea for storing the game state was to simply write it to XML. This was before I really researched serialization in games. This is what that looked like:

It looks bad, it feels bad and it’s very cumbersome to add new variables to this definition. What doesn’t help is that everything uses strings, so I first have to convert my floats to a string before I can store them.

I was also starting to worry about security and performance. TinyXml2 is blazing fast, but my levels would grow in size very quickly. On top of that, storing your game state as plaintext is a bad idea. It’s practically begging to be messed with. However, I didn’t really look too much into these problems, my main concern was just getting it to store my game state to a file.

What I noticed, however, was that every time I made a relatively minor change to my XML, like putting the Object’s id in an attribute instead of a child node, I would have to change massive amounts of code. It was bothering me, but not enough to actually do something about it. But then I wanted to change my Camera’s position from a Vec3 (one value) to a JuicyVar>Vec3< (three values). And that was such a nightmare that I finally set down to research serialization.

So that’s how you serialize your data…

What I found was magnificent. Google has an open source project called Protocol Buffers (Protobuf for short) that they use internally for all their projects.

The basics come down to this: instead of describing what your data is, why not describe what your data looks like?

Alright, an example. This would be a position stored in XML:

<Position>0.0 100.0 -10.0</Position>

Now, this is what it looks like using a Protobuf definition:

position {
x: 0.0
y: 100.0
z: -10.0
}

This looks much cleaner in my opinion. It only specifies the name of the field once and it labels the values.

This .proto file is fed to protoc.exe, which converts the file to a header (.pb.h) and implementation (.pb.cc). Now you can include those generated files in your project and use them to parse the data.

Let’s shake our definition up a bit. I don’t want a static position, but a juicy one, which wiggles and wobbles to the target position over time. We’ll need a Vec3 as data, a Vec3 as target and a blend factor. First we’ll add a new message:

The main problem with XML is that it’s extremely brittle. If your data doesn’t match up with your definition, you’re pretty much screwed. You have to add a lot of checks to make sure that doesn’t happen. Checks I haven’t even added here.

With protobuffers, a lot of these common annoyances are smoothed away. If you use mutable_target() instead of target(), you are guaranteed to get a pointer to a PbGame::Vec3, even if the message doesn’t have one right now.

Another advantage is that protobuffers can be saved to and loaded from a binary file. That means that you have a text version of your data where you can make changes in and a binary version that you ship with, for speed and safety. This also means that they’re extremely useful for packing data to send over an internet connection. You don’t have to keep a record of what each byte stood for because that’s already in your .proto file!

Conclusion

I really, really like protobuffers. They took a while to get used to, but once they click, I suddenly had a shiny new hammer and everything starts to look like a nail. Now I just need to figure out what the downsides are.

June 29, 2012

A while ago, I did a retake for a course on procedural programming. One of the assignments was to generate a textured sphere. You would be marked on getting the texturing right, but I got distracted and decided to try making an icosahedron instead. However, I also made a version that used a more traditional subdivision method: generate circles on a cylinder, and have the radius of the circles depend on the cosine of the distance on the cylinder’s height. Here are my spheres:

Looks pretty round right? However, let’s take a look at their wireframes:

Click on the image for a larger version.

From the thumbnail, the second version looks unchanged. But when you click on it, you’ll notice that the lines are so dense that it looks textured!

What’s the difference?

The sphere on the left (sphere 1) uses the traditional method of sphere generation. It has 31 subdivisions on the y-axis and 31 subdivisions on the z-axis of the half-sphere. This half-sphere is then mirrored to the bottom. It has a total of 3,844 faces.

The icosahedron is an almost perfect sphere, but it comes at a high price. It uses a lot more faces to achieve the same effect.

But okay, that's not really fair. Let's scale down the quality considerably:

Sphere 1 uses 5 subdivisions on the y-axis and 5 subdivisions on the z-axis, for a total of 100 faces.

Sphere 2 uses 0 recursive subdivisions, for a total of 100 faces.

They use the same amount of faces, but in my opinion, the sphere on the left looks better. It looks less lumpy and a lot more round. Let's take a look at the amount of faces per quality level.

Icosahedron:

Level 0 - 100 faces

Level 1 - 420 faces

Level 2 - 1,700 faces

Level 3 - 6,820 faces

Level 4 - 27,300 faces

Level 5 - 109,220 faces

Subdivided sphere:

YZ: 5 - 100 faces

YZ: 10 - 400 faces

YZ: 15 - 900 faces

YZ: 20 - 1,600 faces

YZ: 25 - 2,500 faces

YZ: 30 - 3,600 faces

It's easy to see that the subdivided sphere gives you a lot more bang for your buck. The 30-subdivisions version is comparably in quality to the 5-level recursive icosahedron, but it uses only 3.2% of the faces!

Texturing problems

The truth is: you don't *need* the precision an icosahedron will give you. Because they both hide a much harder problem: texturing a 2D plane on a 3D sphere. Here's what the top looks like:

On the top-left, you can see the texture being used. Coincidentally, it's also being generated procedurally. (Hey, it was a course on procedural generation, right?) It looks terrible, but this is as good as it's going to get. I got top marks for my texture mapping, because most people don't even get it this right.

Why is it such a problem to map a texture to a sphere? Well, xkcd explains it better than I can:

Image taken from http://xkcd.net/977/

The way I solved it is by generating polar coordinates from normalized coordinates on the sphere to get the texture u and v. I won't go into too much detail, because I don't want to ruin the course material. But I do have a fix for the dateline issue, which took a very long time to figure out. When the texture goes around the sphere, you get to a face that has to wrap around from 1.0 to 0.0. If you don't fix that, you will get an ugly band.

It's not a lot of code, but it isn't explained properly anywhere else. The same code is used for both the icosahedron and the subdivided sphere, in case you were wondering.

Conclusion

Consider using cosine and sine to generate a sphere. It generates a lot less faces for the same amount of detail. For most games it will be "good enough". Unless you're generating planets that really, really need to be round all over its surface, you can get away with a subdivided sphere quite easily.

February 6, 2012

This is part 2 of an on-going series. Start here if you don’t know what’s going on.

Hello readers!

Last week I spent most of my time working on a school assignment, but I still managed to find some time to cram in some new features into this fancy indie game I’m making.

But most of all, I spent time playing one of my favorite indie games of all time: Mount & Blade: Warband. So I want to start this blogpost with a critique of that game.

What’s so bad about MB:W?

When looking for comments about it online, you’ll find a lot of people talking about the first person combat, and not a helluva lot of talk about the third-person world overview. Even though you’ll be spending close to 70% of your time in it!

The way it works is that you get an iconic representation of your party: a horse with a dude on it with a number representing its movement speed. All the other parties get the same treatment, so you can start fights by colliding with them.

However, that is just one way to get rich. You can also try your hand at trading, where you buy goods for cheap in a village or town and sell them in another town where the demand for that good is high. Of course you’ll have to factor in that some goods will lose value as they get eaten or start to rot, so to get rich you have to focus on the unspoilables like salt, velvet, tools and furs. But how do you know if the price you’re getting for an item is amazingly cheap or horrifyingly expensive?

You don’t. And it’s very problematic. In fact, I started a spreadsheet just to see what the average price of items was:

What is this, EVE online?

After a while, you start seeing patterns. For instance, if you see grain at <10 denars, you should always buy it. That's because can sell it to most towns and villages for 20-30 denars. But then get another problem: the game tells what for, but doesn't tell how much paid in first place! So spreadsheet also has a column selling price see that grain is quick profit, it's never much. However, if iron <70, ~300 A huge profit! And would be nice actually told about these things. If let discover workings
of local economy, help make profit without bloodshed.
But enough this hugely popular successful built by two people an apartment. Let's talk new features PostApocalypse!

Saving game state

This proved to be quite the headache. How do you store game state in a file? Well, remember how I talked about MVC last week? That model is very, very handy when you want store only the state of the game and not any additional fluff. You simply store the variables in the Data modules, because you don’t have to care about the rest!

But what should the save file structure look like? My first thought was to use an .ini like structure. And you should always start small before investing a lot of time an energy into a new system, so I started with saving and restoring the camera’s position and rotation:

This looks fine, but it has a problem. These guys can explain it much better: Engine Tech: Concurrent World Editing. They talk about the problems they had with their save file system and how they resolved it. The gist of it is that when you store floats as strings, you tend to lose precision real fast. Did you know that the floating point number “0.2” cannot be accurately represented in the floats we use in our floating point hardware? It’s true and it’s not a conspiracy! It’s just how the math works out.

The result of this is that it’s a bad idea to store floats as strings. So I looked at the article I linked earlier to see how they fixed it. Well, here’s what they do:

They store the float as both a string representation and they store it as a hexadecimal integer! That’s brilliant, because you can always read the hex correctly and convert it to a float.

But as I tried to implement this myself, I realized: why don’t I just make a binary file in the first place?

The tool I'm using here is called 010 Editor. It's awesome.

That probably looks like a big ol’ pile of gobbledygook to you. As well it should! But after reverse-engineering a great many stupid file formats, I know what makes a great file format.

The savefile

The most important thing of any binary format is the header. This helps identify the type of file you’re dealing with, but also the type of data. I use a PNG-like block structure, where each block has its own header.

Keeping the header separate like that allows you to load the entire header in one go, without those pesky conditionals. The name variable is used to identify the block. Right now I have only two blocks: “PASV” (PostApocalypse SaVe file) and “CAMR” (Camera). The first block allows you to quickly identify that yes, this is a PostApocalypse save file and it also tells you how many blocks are going to follow.

The camera block simply has its data (position and rotation) dumped into the block as bytes.

And that’s it. :) Surprisingly easy to work with, reliable and extensible! The version variable might seem like overkill, but it will vital in allowing backwards-compatibility. Hopefully, by the time this game reaches completion, I will still be able to convert and load this savefile.

The Graphics User Interface

UI is one of those things that nobody ever enjoys doing, but is vital to what separates a mediocre game from a great game. Mount & Blade: Warband, that game I use to steal… I mean get inspired by? It has a terrible user interface. It tells you either too much:

Why not just tell me: 108 prisoners. Sure would save screen estate.

Or too little:

How many days can I survive on these food supplies? How much does each horse increase my movement speed? What the hell is encumbrance even for?

So I set out to improve upon that by designing a great UI from the start and integrating it with the rest of the game.

Firs things first: that silly font rendering I had had to go. Obviously it was Windows-specific and it would be a pain to port to other platforms. But I’ve used FreeType in the past and I could never get it to render quite right. Also, I don’t want to spend all of my time designing a system I don’t really care for, so let’s take an existing package.

I don’t want to use CEGUI, because last time I used it I produced code looking like this:

That RockScroll image on the right side really tells you all you need to know about the GUI code.

But I didn’t want to let my prejudices get in the way of progress, so I gave it a try anyway. And then I found out that you can’t really control anything. Don’t want to use XML to load scheme files? Too bad, the scheme loader uses private, not protected members. Want to use your own font render? Yeah, no, this one just works for us you know?

So I decided to look at alternatives. And I ended up with libRocket, the cute little engine that could.

That’s not to say I haven’t had problems with it. For instance, what if you want to know whether input should be handled by the game or by the UI? I still haven’t figured out a reliable way, but here’s what I came up with:

January 27, 2012

Beginnings

Last summer, I needed to stay busy. You don’t want to come out of your summer holiday with a set of blunt tools, so you need to continue using them. It was actually my father who gave me inspiration for this game. My dad is an accountant who specializes in automating business processes. Yes, it’s as dull as it sounds. But he was complaining that if the world were to end, his skills wouldn’t be of use to anyone. I disagreed, because even after the apocalypse food supplies will need to be kept track of, trading will still have to be organized and eventually governments are to be rebuilt.

And that… sounded like a pretty cool game to me.

What helped was that at the time, I was playing boatloads of Mount & Blade: Warband. If you’ve never heard of it, it’s a game where you start with a lonely fighter on a horse and end up with a mighty army. But what I especially enjoyed was the trading, because you can make a lot of money with very little effort. Some settlements will have a surplus of one material, but lack another. For instance, grain will be cheap in city A, but expensive in city B. If you can keep track of those differences (for instance, on a piece of paper) you can send caravans from one city to the other and reap mountains of gold.

It took me forever to find this screenshot of Mount & Blade: Warband. Apparently nobody else enjoyed the boring bits as much as I did.

So my very first design goal became: if you don’t need paper to keep track of “hidden” game state, the design has failed.

False start

Another thing I wanted to try was to build a game completely data-oriented. Previously I have blogged about data-oriented design and how it could help your game run faster, but I wanted to see what would happen if I did everything that way.

It was a disaster.

One thing I decided early on was that I wanted hexagonal grid cells as a landscape. So I had a “Landscape” class and it had “DataChunks” and “DataCells”. The cells themselves would have to be built every frame, using the center position as a reference. Using some math you can then derive the normals for each triangle of the cell. That’s all fine and dandy, but the function to do that (_GenerateNormals) assumes cells are in one chunk. When you add more chunk, it becomes increasingly complex.

Basically: the code architecture was horrible to work with and impossible to add features to once written. I had coded myself into a corner.

I abandoned the project and started focusing on school projects again.

Reboot

That was half a year ago and now I know better than to try and build a game using data-oriented design. I now (almost) have enough credits to go on an internship. For the first time in a long while, I have some free time. So this week, I decided to take a look at PostApocalypse again.

Yup, still as bad as I remembered. I could reuse maybe 5% of the old code, but threw the rest away.

Progress

Right now, the game looks like this:

So what currently works?

Generating a landscape using Perlin noise

Generating a default mesh with a default texture

Loading DDS textures (the grass)

Rotating the camera using the keyboard

Rendering the landscape and the units using OpenGL

Placing new units on the landscape

Selecting and deselecting units

Pathfinding between points on the landscape

Moving units across the map

Saving and loading game state.

I’d say that’s pretty good for one week of development. :D

And now for the cool part: explaining the game’s architecture!

New architecture

The solution is divided into three projects:

Game – The actual game code

Render – The rendering library

Tools – Static library both projects link to

Game links to the Render project, but obviously doesn’t do any of the rendering. This will make it easier in the long run to port the project to other systems. Because both the Render project and the Game project need to log to the same file, the logging class is put into the Tools project. Ever wanted to use a singleton across the DLL-Executable barrier? Here’s how you do it:

So how does that voodoo magic work? Well, the Render project defines PROJECT_DLL, while the Game project does not. So when the Game class compiles the header, it creates a new instance of the Log class and returns it. But when the Render class compiles it, it calls a function inside the executable exposed to the DLL’s that returns that very instance. Now, both the executable and the DLL have the same pointer to the Log class. :D

RenderManager

Because I’m trying to avoid OpenGL deprecated calls, I’m using a RenderManager class. This keeps track of global state like matrices, the current material and the current shader. It makes it easy to abstract all kinds of rendering operations without losing performance.

I built the first version of the rendering architecture for my Procedural Generation class. I wanted to experiment with using “new” OpenGL (> 2.1) and had to figure out how I could still have global state.

Pretty sweet. But what if want find a person by name in our database? We’ll add a method to the Controller:

DataPerson* GetByName(const char* a_Name);

MVC in PostApocalypse

This pattern is the best I have ever seen for game development. Because you keep everything separated, it becomes a breeze to add new logic or update the rendering. Currently, there are three controllers:

ControllerLandscape – Keeps track of the landscape, the chunks and the cells.

ControllerUnit – Keeps track of the units on the landscape.

ControllerPath – Combines data from the landscape and the units to find a path for a unit between two points.

When you place a unit, you need to know what cell to add it to. So first you do a raycast into the scene to get the cell:

I can’t stress enough how easy I got this working. I spent two hours doing pathfinding and an hour to put it into its own controller, but when I was done it was completely abstract.

Saving the game

What I’m currently working on is trying to save the game state to a file and loading it back in. You might think this is a bit early in the project, because I don’t have that much state left to save. But I’m doing it this early because it’s easier to keep adding to a save game implementation than it is to retrofit it when the game is done. All you have to do is set up the basics and make sure it continues to work throughout the project.

So I have two classes: GameSave and GameLoad. When you press F5, the following happens:

Before loading, you specify the target pixel format. That way you immediately have your data the way you need it.

For a full list of supported formats, please refer to the “Supported formats” page in the supplied documentation.

TinyImageLoader is licensed under the MIT license.

What’s new?

1.7.0 introduces flexible pitch. Pitch refers to the width plus padding of an image. In TIL’s case, I talk about both horizontal and vertical pitch. Why is that handy?

Well, suppose you need to load images on a platform that doesn’t support non-power of two textures. How would you ensure that the image is always a power of two when you load it? You can’t with 1.6.3. In fact, you’d have to load the image and then pad the data manually. Not fun.

This function takes the width and height of an image and sets the horizontal and vertical pitch.

Then you can attach the function to the pipeline, after initializing TinyImageLoader.

til::TIL_Init();
til::TIL_SetPitchFunc(CustomPitchFunc);

And now your images will always be padded until they are the nearest power of two.

You will most likely not have to use this functionality in your projects. In fact, you're more likely to walk over to your artists and smack them on the head for not sticking to powers of two. But it's nice to have and it has other uses too.

DDS support

1.7.0 comes with improved DDS support. DDS has many forms of compression, but only uncompressed, DXT1 and DXT5 are supported, as these are by far the most common types of DDS textures.