Monster Madness – creating games on the web with Emscripten

When our engineering teams at Trendy Entertainment & Nom Nom Games decided on the strategy of developing one of our new Unreal Engine 3 games — Monster Madness Online — as a cross-platform title, we knew that a frictionless multiplayer web browser version would be central to this experience. The big question, however, was determining what essential technologies to utilize in order to bring our game onto the web. As a C++ oriented developer, we determined quickly that rewriting the game engine from the ground-up was out of the question. We’d need a solution that would allow us to port our existing code in an efficient manner into a format usable in the browser…

TL;DR? Watch the video!

Playing the Field

In our tests, Flash ran slowly and had inconsistent behaviors between Pepper (Chrome) and Adobe’s plugin version. Combined with the increasingly onerous plugin requirement, we opted to look elsewhere for a more seamless, forward-thinking approach.

NaCl had the issue of requiring a walled-garden distribution site that would separate us from direct contact with our users, and also being processor-specific. pNaCL eliminated the walled-garden requirement and added dynamic code compilation support, but still had the issues of being processor-specific code necessitating in our view device-specific testing, and a potentially long startup time as the code would be linked on first run. Finally, only working in Chrome would be a dealbreaker for our desire to have our game run in all major browsers.

A custom plugin/extension with C++ would require lots of testing & maintenance efforts on our part to run across different browsers, processor architectures, and operating systems, and such an installation requirement would likely scare away many potential players.

As it turned out, for our team’s purposes the Emscripten compiler & asm.js proved to be the best solution to these challenges, and when combined with a set of other new-ish Web API’s, enabled the browser to become a fully featured platform for instant high-end 3D gaming. This just took a little trial & error to figure out exactly how we’d piece it together…and that’s what we’ll be reviewing here!

First Steps into a Brave New World

We Trendy game engineers are primarily old-school C++ programmers, so it was something of a revelation that Emscripten could compile our existing application (built on Epic Game’s Unreal Engine 3) into asm.js optimized Javascript with little to no changes.

The primary Unreal Engine 3-specific code tweaks that were necessary to get the project to compile & run with Emscripten, were essentially… 1, 2, 3:

No really, that was about it! Within a day of fiddling with it, we had our game’s Javascript ‘executable’ compiled & running in the browser. Crashing, due to not having a graphics API implememented — but running with log output! Thankfully, we already had Unreal Engine3’s OpenGL ES2 version of the rendering subsystem ready to utilize, so porting the renderer to WebGL only took another day.

WebGL appeared to essentially have a superset of features compared to OpenGL ES2, so the shaders and methods used matched up by simply changing some API calls. In fact, we were able to do improvements by making use of WebGL’s floating point render targets for certain postprocessing effects, such as edge outlining and dynamic shadows.

Postprocessing makes everything prettier!

But how’s it run?

Now we had something rendering in the browser, and with a quick change to capture input, we could start playing the game and analyzing its performance. What we found was very encouraging: straight ‘out of the box’, in Firefox the asm.js version of the game was getting nearly 33% of the performance of the native executable. And this was comparing a single-threaded web application to the multi-threaded native executable (so really, not a fair comparison! ;). This was about 2x the performance we saw with our quick Flash port (which we still utilize as a fallback for older browsers that don’t yet support asm.js, though we eventually hope to deprecate entirely).

Its performance in Chrome was less astonishing, more towards 20% of native performance, but still within our target margins: namely, can it run on a 2011-model Macbook Air at 45-60 FPS (with Vsync disabled)? The answer, thankfully, was yes. We hope Google will continue to improve the performance of asm.js on their browser over time. But as it currently stands, we believe unless you’re making the browser version of ‘Crysis’ with this tech (which may not be far off), it seems you have enough performance even in Chrome to do most kinds of web games.

60 FPS on an old Macbook Air

Putting the Pieces into Place

So within a week from starting, we had turned our Unreal Engine 3 PC game into a well-running, graphically-rich web game. But where to take it from here? Well, it’d still need: Audio, Networking, Streaming, and Storage. Let’s discuss the various techniques used for each of these systems.

Audio

This was a no-brainer, as there is only really one robust standardized web audio system apart from Flash: WebAudio. Again, this API matched up pretty well to its mobile cousin, OpenSL, for which we already had an integration. So once we switched out the various calls, we had .

There was an apparent issue in Mac Chrome where sounds flagged “looping” would sometimes never become destroyed, so we implemented a Chrome-specific hack to manually loop the sound, and filed a bug report with Google. Ah well, one thing we’ve seen with browser API’s is there’s not a 100% guarantee that every browser will implement the functionality to perfect specification, but it gets the job done!

Networking

This proved a little trickier. First, we investigated WebRTC as used in the Bananabread demo, but WebRTC of course is for browser-to-browser communication which is actually not what we were looking to do. Our online game service uses a server-and-client architecture with centralized infrastructure, and so WebSockets is the API to utilize in that case. The tricky part is that we have to handle all the WebSockets incoming and outgoing data in JavaScript buffers, and then pass that along to the “C++” Emscripten-compiled game.

With some callbacks, this worked out, but we also had to take our UDP game server code and place the WebSockets TCP-style layer onto it — some iteration was necessary to get the packets to be formatted in exactly the way that WebSockets expects, but once we did that, our browser game was communicating with our backend-hosted Linux dedicated game servers with no problems!

Streaming & Storage

One advantage to being on the web is easy access to the browser’s asynchronous downloading functionalities to stream-in content. We certainly made use of this with our game, with the initial download clocking in at under 10 MB. Everything else streams in on-demand as you play using standard browser http download requests: Textures, Sound Effects, Music, even Skeletal Meshes and Level Packages. But the bigger question is how to reliably store this content. We don’t want to just rely on the browser cache, since it’s not good for guaranteed immediate gameplay loading as we can’t pre-query whether something exists on disk in the regular browser cache or not.

For this, we used the IndexedDB API, which lets us asynchronously save and retrieve data objects from a secure abstracted storage location. It works in both Chrome and Firefox, though it’s still finicky as the database can occasionally become corrupted (perhaps if terminated during async writes) and has to be regenerated. In the worst case, this simply results in a re-download of content the user already had received.

We’re currently looking into this issue, but that aside, IndexedDB certainly works well and has the advantage of providing our application standard file IO functionality, useful to store content that we download. (UPDATE: Firefox Nightly build as of 12/10 seems to automatically reset the IndexedDB storage if this happens and it may not recur.)

Play it Now, and Embrace the Future!

While we still have more profiling and tweaking to, as we’re just now starting to use Firefox’s VTune support to symbolically profile the asm.js performance within the browser. Even still, we’re pretty pleased with where the things currently stand. But don’t take our word for it, please try it yourselves right here, no installation or sign-up required:

We at Trendy envision a day when anybody can play any game no matter where they are or what device they happen to have, without friction or gateways or middlemen. With the right combination of these cutting-edge web technologies, that day can be today. We hope other enterprising game developers will join us in reaching players directly through the web, which thanks to Emscripten & asm.js, may well become the most powerful and far-reaching “game console” of all!

11 comments

Very interesting indeed!!! Still, a part the learning side of the thing, I wonder, what about using Unity3D which has a cross platform fast native plug-in AND the support to Google Chrome native client?

The main benefits of Emscripten vs the Unity Plugin+NaCl are two-fold in my view:

– On browsers where you’d have to go with the Unity plugin, having the plugin install requirement may lose you a lot of potential players. From some folks who work at Trendy Entertainment who previously created Unity web games, they tell me their user attrition rate at the plugin gate was as high as 75%. Even though Unity’s a fairly “well known” plugin, to the average user it is still a potentially scary proposition installing something they may have never heard of before.

– By using Emscripten, you don’t have to be locked into just using Unity. As nice as it is, there are other powerful game engines out there too which provide full source code to compile in Emscripten, or maybe you’d like to create your own to make a nifty application :)

Also, Emscripten use doesn’t have to just be for entire web applications, it could also be theoretically used to make web-hosted functional libraries that do heavy performance-sensitive internal work with inut, and then feed output data back to any web application or Unity plugin game via JavaScript. We soon intend to use this approach to standardize our networking library, for example.

I will say, in my own personal view Unity would benefit from having an Emscripten web version as an deployment option. Hopefully someone over at Unity HQ may be reading this!

Actually, there is no reason we don’t run the WebSockets on a standard port like port 80! That was our oversight, and we’re going to fix it on Monday! I’ll post here / update the blog when it’s moved to port 80 and then you should be able to try the game behind any corporate firewall, mwahaha!

@Tom:
Compiling the code for each architecture and lots of native code implementations or using unitiy3d, which is great for easy beginning, is not nearly that impressive as bringing the whole game to platform independent javascript with this performance!
Keep in mind that this solution is using open standards and no proprietary base like unitiy3d, no license fees at all and accessible for everyone.

A note on pNaCL. Above, it says “only working in Chrome would be a dealbreaker for our desire to have our game run in all major browsers.”

Luckily, that’s no longer true. Pepper.js, which is fairly new, will transpile any pNaCL application into JS through Emscripten to run on all major browsers: http://trypepperjs.appspot.com/

Also curious, what’s blocking you from putting Monster Madness on the mobile web? I think it’s fair to say that both Firefox and Chrome (which support WebGL & Web Audio) want to enable you to do that. :)

Thank you for the heads-up on pepper.js! That’s a very interesting approach and we’re going to try it out here. I’d still have potential test-oriented concerns about the need to try out the pNaCl compiled application on the various CPU architectures to ensure there aren’t any linktime/runtime issues, but perhaps I’m overestimating the possibility of that to cause issues. Definitely the potential of pNaCl applications to run in other browsers via Emscripten makes it much more attractive… Emscripten saving the day again? ;)

About the Authors

Jeremy Stieglitz has been creating games & game technology since 2001, when he started co-developing "Reality Engine", a hobbyist 3D game engine that he sold to Epic Games in 2005. Since then, he has developed numerous Unreal Engine indie & studio titles, including Monster Madness: Battle for Suburbia and CellFactor. In 2009 he co-founded Trendy Entertainment, where he created Dungeon Defenders. For Dungeon Defenders 2, has has been focusing on Trendy's cross-platform Playverse online infrastructure, and recently founded Trendy-subsidiary "NomNom Games" to develop indie-scale titles for this service. He is a big believer in technology-driven cross-platform, device-agnostic game development, and the power of the Web to connect content creators directly to consumers with no barriers.
Jeremy is the Co-Founder & Chief Technical Officer of Trendy Entertainment (www.trendyent.com), and the Director of Nom Nom Games (www.nomnomgames.com)

Technical Evangelist & Editor of Mozilla Hacks. Gives talks & blogs about HTML5, JavaScript & the Open Web. Robert is a strong believer in HTML5 and the Open Web and has been working since 1999 with Front End development for the web - in Sweden and in New York City.
He regularly also blogs at http://robertnyman.com and loves to travel and meet people.