In the past couple of weeks I started the (huge) task of designing a game engine from the ground up. I have read a couple of books that were good and I learned something from each one, but still the task proves very difficult. I am trying to use UML to design it, but it is so big that it looks impossible to get almost everything down at design stage.

Do you have any advice as how I should approach designing such a big piece of software? I did software designs before for various projects at work, but none was so complex as this one. Also, it is quite hard to identify use cases, giving this is not an end user application, and I never designed a middleware before, so any input will be appreciated.

Draw up the specifications for the game that this engine is going to be used to build first, and then you'll have your use cases.

Developing an engine without having a companion game project (or several) will do that - you won't have concrete requirements, specifications, goals... it'll suffer feature creep, or have groups of features that don't gel together. It's like trying to design a vehicle when you don't know if it has to get you to the corner store or get you home from outer space - design a vehicle without those specifications and god knows what you'll end up with.

Original post by HodgmanDraw up the specifications for the game that this engine is going to be used to build first, and then you'll have your use cases.

Sounds like a good advice, unfortunately I do not have a game project to write this for, I was thinking of designing and writing a game engine just for learning. However, I realize what you are saying would be very helpful in determining the use cases. In this situation, should I just make up a game project (which eventually can become the demo game) so I can nail down the requirements?

I wonder how companies behind other engines, like for example Unity or C4, did this task, as they didn't make the engine for a specific game...

Original post by meeshooI wonder how companies behind other engines, like for example Unity or C4, did this task, as they didn't make the engine for a specific game...

That's why most of those companies only find customers within the indie scene and not professional game studios. U wont really find a AAA game using C4. Thats why "we should write games, not engines ". An engine is usually the result of the requirements of your game. U wont be able to produce something useful, unless u've worked already with a professional game engine on a real game.

Writing a game and extracting the engine is the only way to make something useful - if you lack experience.

It doesn't need to be a fully fledged game with great assets and interesting mechanics, but it must fully exercise the engine. If you design a tech demo that doesn't reflect real use then don't be surprised if the engine turns out to be poorly designed in the areas the tech demo didn't cover.

The fallacy of such an approach is frequently evident by many "engines" with pretty tech demos that never seem to grow into games.

You know yourself that you need experience to guide design. The designs you made at work would not have been possible had you done smaller scale design beforehand. Design is an iterative process (both during one project and between different projects), you cannot immediately jump to the deep end when you lack experience in a given domain.

Original post by cruZThat's why most of those companies only find customers within the indie scene and not professional game studios. U wont really find a AAA game using C4. Thats why "we should write games, not engines ". An engine is usually the result of the requirements of your game. U wont be able to produce something useful, unless u've worked already with a professional game engine on a real game.

Hmm, interesting idea. I have some game designs written on paper, however I don't have any that would require more than a 2d engine or a simple 3d engine. I think then I should first try to think of a game genre that is more visually demanding and try to fit one of those designs there.

Writing a game and extracting the engine is the only way to make something useful - if you lack experience.

It doesn't need to be a fully fledged game with great assets and interesting mechanics, but it must fully exercise the engine. If you design a tech demo that doesn't reflect real use then don't be surprised if the engine turns out to be poorly designed in the areas the tech demo didn't cover.

The fallacy of such an approach is frequently evident by many "engines" with pretty tech demos that never seem to grow into games.

You know yourself that you need experience to guide design. The designs you made at work would not have been possible had you done smaller scale design beforehand. Design is an iterative process (both during one project and between different projects), you cannot immediately jump to the deep end when you lack experience in a given domain.

Writing the game will give you this experience.

Thanks for the article and the advices, they were very helpful. I have a game design that has been on my mind for a few months now(already written a part of it), I think i can adapt it to a 3D environment, I'll start by doing that.

I'm back with another question. Whenever I develop a modular application (like a game is), I usually go for a plug-in system. This is quite easy to implement if I decide that game should run on only one platform (e.g. Windows) through dlls. However, building a multi-platform plug-in system can be difficult from what I read in during some research. I found a good article about it, but it is quite complicated and I don't know if the overhead is worthy.

Should I try it or I should just go with a big exe for the game and a big dll or static library that contains all reusable modules?

I just put each of the modules in it's own static library. Unless you want the end-user of the game to be able to add their own modules (mod the engine, maybe?), then there's not much need for the core engine modules to be dynamic plug-ins.

Original post by HodgmanI just put each of the modules in it's own static library. Unless you want the end-user of the game to be able to add their own modules (mod the engine, maybe?), then there's not much need for the core engine modules to be dynamic plug-ins.

Well, the only thing i want is to for example to have a Rendering interface and more renderers that implement it (one for direct x, one for opengl, one for opengl es, etc). Of course, for start I'll have only one (direct x), but I want to keep it open for porting on other platforms. Same thing for the input system. I think this result can be achieved with static libraries, thanks for the tip.

Original post by meeshooWell, the only thing i want is to for example to have a Rendering interface and more renderers that implement it (one for direct x, one for opengl, one for opengl es, etc). Of course, for start I'll have only one (direct x), but I want to keep it open for porting on other platforms. Same thing for the input system. I think this result can be achieved with static libraries, thanks for the tip.

First, I'd suggest dropping DirectX and go to OpenGL first if you really are worried about "other platforms", where the most common ones (linux, windows, iPhone/pad) all use OpenGL or the OpenGL-ES variant. (flame-war like subject... others will say stick with directX. You could port to the X-Box and it works under linux in wine, etc.)

Second, worry about hiding the graphics API from the game before hiding the graphics API from the engine. You should be making "models", "materials", "Effects" and other high level interfaces for the game to use. Then you can worry about being able to port those parts later.

Lastly, I'd say go look at SDL or Ogre or the like to take care of some of those abstractions for you.

Well, I thought about opengl too, i mean I know way better the opengl api then the dx one, but dx has some very nice helpers like D3DX for doing the math, so i won't have to do a new math library for starters.

About ogre and the like, my point here is to learn to make a 3d engine by myself, so using an already made one won't help. I already have a license of C4 engine to make 3D games, but really what I want is to make a game with it's engine by myself. SDL might prove useful however for the input part.

Ok, I managed to integrate some modules that I have previously written for another application, like a direct input module and some direct 3d simple renderer, into the engine as static libraries, and I also managed to do this with cross-platform in mind, using a set of abstract interfaces.

Now I have another question:

Some game engines "are the game itself", they just load the actual game as a module and run it. I can think of Unreal, Crytek and others. It's like you develop the game in an editor that actually allows you to play the game inside already and it's WYSIWYG.

On the other hand there are these engines which are basically a big collection of very robust libraries that work together and the game itself is an independent application, one for each platform, that implements platform specific tasks like window creation and such, and this game uses the engine as a library of utilities and tools, nothing more.

Which do you think it's the best approach? Do the window creation and everything as part of the engine and run the game like it is some kind of plug-in, or have the engine considered a huge library and the game should just use whatever it needs from it?

I used to try to modularise common bits into libs in what I thought was a fairly sensible way - e.g. graphics.lib for looking after Direct3D.

What I found happened was that each project, I would need to make some seemingly minor adjustments to the library for that project's specific needs. This would then break all the previous projects that also depended on the lib.

What I've done the last couple of years is just literally copy-paste the .h and .cpp files from the last project into the new one and add them to the project. Now free to tinker at will.

I've been playing around with Direct3D for about six years now and still don't have any real idea how to create a general purpose library wrapper that would suit all my needs without being horribly bloated.

So my advice, for what it is worth, is to use third party libraries but just add your own code in text form to new projects. Yes, it takes me fifteen minutes or so to get a new project set up and running, but I've been working on my current game for well over a year now so that isn't really significant.

Original post by ApochPiQI'd say it's up to you, really. There are obvious pros and cons to each approach, but neither is inherently "better" for a general-purpose engine.

Ty for your opinion. I think I'll go for the "library collection", as it can always be transformed into the WYSIWYG approach.

Quote:

Original post by Aardvajk

Well, I did this too in the past, but ultimately the goal is to obtain an engine, not really the game itself, so I'll try to stay as modular as possible. It can become bloated indeed, but I guess only if you try to support too many things at once. I guess all big engines are a bit bloated, but however you get to choose what to link and use for your own game, nobody forces the user to use all the libraries and stuff.

1)As already advised, building an engine without knowing what it's for is likely to be a never-ending project, unless you're experienced at planning large projects from the ground up. The urge to plan big and go for a generic engine with elegant architecture, rather than something simple that you tie to a game and work on as you go, is normally quite strong. But good software of this size is rarely designed up front and then brought to life, unless you know what this type of software should 'feel' like from working on similar things in the past.

2)How much experience do you have using game engines? I don't mean playing with them, but knowing in reasonable depth how they fit together - scenegraph, rendering engines, etc? Part of your research and design should definitely include looking at other engines like C4, Ogre, etc. You don't need to go into every line of source source, but understand how classes fit together and so on.

1) Agreed with you and this is why I have chosen a game project for it.

2) Not much of an experience, but I studied a bit their code and architecture and they are nice, although I have something else in mind for this one.

Btw, I have set up a blog about my progress (actually a long time ago, but haven't written anything since 3 years ago) so please check up my progress there, the last 3 or 4 posts are about my progress.

I wonder how companies behind other engines, like for example Unity or C4, did this task, as they didn't make the engine for a specific game...

Sure they did. You just may not have seen them, because they were games that existed only to test engine design requirements, or they were games that were built by the developers before they started working on the engines in question.

The reason one should build games first is not because it is impossible to build an engine from scratch without them, but rather to build up the experience of the requirements for games so that you can better tailor the engine to them.

Experienced developers who have worked on many projects already have this knowledge.