1. Introduction

This is a short (success) story of porting my 3D engine, named TDME, from JAVA 1.6 to C++11.

To introduce myself a bit I can tell that I became in 2010 jobwise and by accident part of a 20-25 man power team that created browser games.

I did this for 3+ years, while playing a key role doing backend stuff, and came to the conclusion that creating games was the most interesting, challenging and also most fun thing to do.

For some reasons I decided in 2012 that I wanted to dig deeper. My interest ranged from multiplayer network code to 3D realtime rendering and such as alternative efficient data storage than relational databases.

Then I just started to develop TDME as a hobby project, which means doing a lot of research, doing a lot of trial and error and just do programming with a strong hands on mentality. This way I got a small JAVA based 3D visualization engine with some basic physics system built in which included e.g. collision detection.

On the backend side a friend of mine helped out and ported parts of the JAVA based TDME engine like collision detection to C++. Later I added network code. TDMECPP was born.

Yes, my luck was that I had some mates with me all the time that were kind enough to support me. Some of them still do and I am very thankful for that. They provided art assets, did game design and helped out gaining ideas for tools and such.

Then we actually decided to create a 3D realtime multiplayer game which type is known as MOBA.

Soon we had a game prototype that even had a multiplayer game mode included.

And now we get to the point. It turned out that doing such kind of game with JAVA in frontend (and C++ in backend) was not the best idea.

2. The issues with TDME/JAVA

2.1 Garbage collection

I had to fight garbage collection which blocked the frontend unpredictable for
even a second sometimes which is bad for such kind of game.

There was this one idea to fix this by allocating all the memory required on game session start up and never release it until the game session was finished, which improved the situation but lead to e.g. ugly code. At the end I decided that this approach was not good enough.

2.2. JIT

One minor thing was also that you could watch getting logic being handled by JIT step by step. Means logic would take some more time if being executed first and later be faster.
Not that big issue but still an issue. The idea to work around that was to have a warm up run or something similar. But I never tried.

2.3. Multi threading

There were some other hacks required because this way getting feasible multi threading ready logic was harder to implement as this kind of code was not renentrant by default even if it could be in C++ easily.

The game was multithreaded. It had one thread for visualization engine, one for game logic plus physics and one for path finding.

2.4. Network latency

Another issue was that frontend network code had too high network latency which I had no influence on as I was just using JAVA functionality. Using JNI and roll something own was not an option for me for various reasons.

2.5. SIMD

Another very valid point is that JAVA does not allow to do SIMD explicitly.

2.6. JAVA and C++ in the same project

Last point is that having C++ in backend and JAVA in frontend means that you can not share code between them thus having game logic twice. And I just did not wanted to do JNI or such.

3. The C++ port

I decided to try a complete C++ port of TDME. Meanwhile TDME had approximatively a 2 MB big codebase. So I searched the web for a JAVA to C++ translator and found J2C. I also checked the option to compile JAVA to native executables but found nothing feasible.

3.1. What does J2C do by default?

J2C is a Eclipse plugin for JAVA 1.6. Basically It is able to parse the sources of a JAVA project into a AST / syntax tree and prints this tree as C++11 into a new project. The generated project also includes a working makefile as well as JRE classes with empty definitions that are required by the project being ported.

You need to know first that during the whole process the top priority was to get a part running first, then fix or improve things, and test if it still works. This way the port might have taken a bit longer but it was the safest way.

3.2. What did I do to adapt J2C to suite the needs

parse JAVADOC and add it into method declaration in header files

do not use full namespace in definitions or declarations, always import classes in header of files

.hpp -> .h

removed null pointer checks (and thus throwing NullPointerExeption)

some other minor things

3.3. Port pre process

Namespace root simplification net.drewke.tdme -> tdme

use as less as possible JAVA classes

use own ArrayList, HashMap, Console, … classes

Rename some classes to avoid name collision

3.4. Port post process

not 100% of the generated code did compile, there were problems with a few files, but they were easy to fix

remove JOGAMP

use OpenGL and OpenAL instead

replace it with GLUT for Window management, GL context initialization and HID events

implement required generated JRE classes/methods, because they get not implemented by J2C!

import code from TDMECPP like

jsonbox

tinyXML

…

get rid of JAVA API usage for accessing JSON and XML files and use the above

use call/return by reference and constness for various classes like std::string, tdme::math::*

convert J2C constructor (structure) to ordinary C++ constructors

do memory management

put variables on stack instead of heap where appropriate

J2C creates everything on heap

implement destructors to release memory eventually

Include and use 3rd party libraries for certain JAVA functionality, e.g.

libpng for loading textures

vorbis/ogg for audio package

pthreads for multi threading

…

do inlining where appropriate

convert JAVA protected package modifier, which has been transformed to C++ by just using C++ public modifier, to be implemented via friend class relatationship

manually transfer TDME code comments to TDME2 because Eclipse AST / syntax tree does not contain them thus J2C can just not do it

…

4. Where are we now?

TDME2 is the result of TDME(-JAVA) port to C++11 and TDMECPP. Check its GitHub site.
It already has lots of stuff build in. But please note that TDME2 had no release yet.
A bit of stuff is still left to finish the port. SIMD is missing still too.
But I already continued development on TDME2, as requirements were added, like adding FBX model file support with FBXSDK and much more.

One very good side effect is that now I can now port the engine to other platforms where JAVA is not available or parts of its eco system like JOGAMP.
E.g. I plan to port the engine to Haiku and maybe NetBSD. Lets see.

Yes. I love open source software and open source operating systems.
TDME2 already runs on MacOSX, Windows and Linux.

Now I use the engine for a new game that I am developing with our team. It already works great!!!

I am btw. currently hired to port a medium sized Flash game to OpenFL/Haxe.
Somebody seems to have faith in my porting abilities. 🙂

Nobody likes lags. So for some (real time) applications like real time rendered 3D games it is nessessary to avoid memory allocations and garbage collection when beeing e.g. ingame as GC might block your game just when you do not need or want it.

Additionally there is no way to control garbage collection in standard JVMs.

When profiling your application with tools like Visual VM (http://visualvm.java.net) you will notice iterators very fast as one main cause for frequent garbage collection.

While you can work around creating iterators of ArrayList using simple old for loops like

As there is no way to avoid creating iterators in theses cases I just programmed an own unordered hash map which is set up to reuse a single values and key iterator.

There is one drawback as you can not do nesting with the same hash map iterator in it self which is rarely required.

Basically a hash map does some kind of partition based on a hash function (which is provided e.g. in String.hashCode(), ..., Object.hashCode())

Here it is called buckets. Each bucket can contain several key value pairs.
The bucket is identified by the hash function of the key object. The key value pair is finally identified by Object.equals() in a bucket as a bucket can contain multiple key value pairs.

The number of buckets always equals the number of elements stored in a hash map.

In best case you would have N buckets for N elements and each bucket would contain a single key value pair.
In this case the complexity for put, get, remove would be O(1). With collisions (means multiple key value pairs in one bucket) complexity will have a little O(n) shadow.

In last time i saw more often security holes in web applications from web developers that actually do create web applications and run them – SQL injections.

I do not understand that, because avoiding them is actually no voodoo.

1. Understanding SQL injections

Its easy. You instruct a mysql server to do things with SQL commands. At the application level these are normal strings consisting of a human readable syntax.
So this is actually where exactly the injection can happen.

Usually you will not write harmful SQL code on your own, so where do they really come from?

I could give more examples. But the idea should be clear now.
No. Magic quotes is not a fix. Its rather a mess and deprecated. Just qoogle that.

2. How to prevent SQL injections

2.1 Escaping

A key to success could be escaping. In easy words escaping means that “user input” data will be prepared to be safely used in a SQL query.
If tied to old functional mysql API you just need to do it with mysql_real_escape_string(). If using PDO just have a look at: PDO::quote()

A better way to prevent SQL injections is to use PDO with prepared statements.
Prepared statements have advantages and disadvantes too.

2.2.1 Advantages:

Prepared statements are SQL commands without the actual parameters. They are more like templates for queries of the same type. They become compiled in the SQL server and can be reused which gives a performance gain as compiling is only required one time.

I tested it once and if i remember right, my application speed increased by 20%
Unfortunatly web application requests have a very short life time so that reusing them does not often makes that much sense.

Well, as the parameters are not included in the SQL command you just can not tweak the SQL command but only its parameters.

2.2.2 Disadvantages:

You need one more request to the database server to set up the prepared statement.
You cannot use parameter binding in SQL commands like:

You should always be alerted if putting input data not constructed directly from or not validated of your application itself from like $_GET, $_POST, $_REQUEST, … into e.g. a SQL query and thus escape it.
Its even better to have a abstract security concept which might be e.g. using prepared statements.
…

4. XSS/HTML injections

Injections mean injecting something! So injections are all similar.
A simple example of a XSS or HTML injection would be:

$userName = $_GET[‘username’]; echo ‘Hallo ‘ . $userName;

So with this kind of script i can easily put HTML or Javascript on your page just by putting some HTML or Javascript into a GET parameter “username” when calling the script.