It’s probably never good to start a blog post being shocked at the date of your previous post, but given that somehow 4 years have passed some mild shock is probably to be expected. How can this be? Where has the time gone? Etc…
So, in those 4 years lots has been going on, but nothing I was especially ready to, or in a position to share.

About five years ago I started working on a ‘simple’ 3d puzzle game for mobile devices, my thinking at the time was, “OK, so the app stores are now flooded with me-too 2D apps, how can I stand out? I know, I’ll do something more challenging in 3D”. Famous last words… Everyone knows that 3D game development is an order of magnitude more complicated and fiddly than 2D games, so I was gambling on my experience as a developer to compensate for the extra effort and to help me get a game out quickly. What I didn’t take into account was getting ideas of grandeur, with influence coming from other games I would soon be contributing to…

So, during the summer of 2012 I took some time to consider which SDK to use to develop the puzzle game in, I had my own full PC based 3D game engine which I toyed with porting to mobile, I had a minimal (let’s call it a) game framework based upon the Imagination Technologies SDK (so enough there to manage and draw 3D assets, but not much else), I had my previous Objective-C based iOS apps, then I investigated UDK, and finally settled on Unity3D.

Unity had the advantage of being accessible and quick. UDK offered a technically stronger platform, but wasn’t very useful for mobile devices, and was very inflexible being pretty much hardcoded around FPS games, which my idea could stretch to, but platform flexibility was just going to be too limiting. The power of Unity, which was on version 3 at the time, was that I could try out and prototype gameplay within 7 days of using the engine for the first time – that’s 7 days only working in my spare time too. Such was a joy, I had a really quick response to my ideas, I could try things out and the engine made things really easy, not getting in my way at all. The only real downsides at the time were performance, Unity 3 was really really slow, and C#, I could feel my intelligence wasting away as tasks I was used to dealing with manually were all being dealt with under-the-hood. Don’t get me wrong, the ease of C# was just great, I could do anything without having to think too hard about it, the problems came later, when I returned to ‘real’ programming and suddenly my C/C++/Assembly brain had left the building.

The idea originally was to implement a basic game, based around navigating an open outdoors space, trying to avoid an AI based baddie, while getting to a safe place. I don’t want to go into any more detail than that at this point, as I’m still working on the game, and I hope I might eventually finish it. The focus of this post is the feature creep, that even an experience developer – who knows better – can still experience, mainly as a result of working alone and therefore an excessive amount of time passing by, and expectations shifting.

The aim was to produce very few assets, that could be configured to create simple puzzle spaces, that when navigated in a first person, using drag to change view and tap to move on a mobile device, would create an interesting challenge around ‘where are you looking’ and ‘can you sprint safely to that location’. At the time, even such a basic premise, being in 3D, and some of the free bonus atmosphere of a horror ‘lite’ theme, might have stood out a little on devices that lacked 3D games content.

I quickly discovered that the landscape rendering in Unity 3 was terribly slow. Far too slow for the ‘mainstream’ mobile devices I wanted to target at the time. But by this point I’d got into the idea of using Unity, and I enjoyed the responsiveness of the engine to allowing me to implement my ideas. While testing on PC the landscapes rendered just fine, so here we trigger my first shift from the planned course! A shift to PC made some sense, there were indie stores opening up online, everyone else had gold-rushed to mobile now, so maybe there would be some space on PC again?

So, with a PC focus, I could not explore getting the most out of Unity’s landscape rendering, which now appeared a little too basic for a PC focussed title, so I invested in World Machine to generate more realistic terrains, and an off the shelf shader to save me some time bringing that content in to the game. I invested in some vegetation models, an autumn themed kit, which again gave the visual quality a boost for a PC focus, and offered game-play shifts away from some of the original puzzle aspects, but providing foliage that could block line of site to the AI. Even playing such a basic prototype, with simple AI, but broken lines of sight actually worked much better than I’d expected. Even playing my own prototype game, I had a strong sense of fear for where the AI character might be, not bad for such a quick implementation, the potential was clear.

Around about the same time this was going on, I started to get involved with The Chinese Room’s projects, initially Amnesia: A Machine for Pigs, and later Everybody’s Gone to the Rapture. Being involved in Amnesia was great, the creative team producing the content were top notch, experienced developers, great creativity, knowledge, and good communicators. There was an excellent team feeling, even though everyone was working remotely in various places and our ‘office space’ was a constantly open chat box in Skype. This worked surprisingly well, mainly because the team engaged with it, constantly chatting about relevant topics, posting when making updates or taking a lock on source assets, making suggestions, spotting when the scale was completely out on a section of the game, etc. Good times. This period really reminded me of when I was full-time in industry, and the achievement of completing A Machine for Pigs and have it released, further pushed my desire to get back to working on projects more representative of the ‘full’ projects of my past, and move on from smaller/quicker mobile focussed products.

Amnesia was followed by Rapture, and again it was a feeling of returning to the familiar, being involved in discussions for the early prototype of the game, pitching such to Sony and helping to secure the engine technology. This felt like an important step for Dan and The Chinese Room, it was the step to becoming a ‘real developer’, with real money, real timing, for a real platform holder. With these real opportunities came the logical progression towards The Chinese Room becoming a real studio, so with that they withdrew from overlaps with the University of Portsmouth (my full-time job) to focus on the demands of a real full production schedule. I found this disappointing, given the investment that several of us had made to bringing the company to this stage, but understood from my own experiences that this was part of a studio ‘growing up’. I decided I would invest this disappointment into my own project, and push that to a higher standard, introducing more story, more themes, more quality, to emulate something of at least Amnesia levels of quality, but within the constraints of Unity which I believe had been updated to version 4 around this time, yes with much letter landscape and rendering performance in general.

In the space of around 12-18 months, my original easy to implement 3d puzzle themed AI baddie game, had shifted form an iPhone game, to a 3D story/AI driven horror adventure, based in a game engine that could barely manage First Person games, but wasn’t really designed for it.

My initial expanded story draft for the game had around 11-12 separate maps, each with a specific theme, that told a part of the journey the player was to go through. Early prototypes of levels ranged from 2km x 2km to 8km x 8km in size, I implemented around 5 of these roughly in Unity, shifting from map to map as my mood took me, to maintain my interest. Over time, and being involved in the map design for Rapture, it slowly became clear that such a massive area is an epic pain, both from the point of view of implementing and filling such, and also from the point of view of getting a player around the spaces quickly enough for it to be interesting. Add on to that the sheer number of assets needed to feed so many maps, wow, this was going to take a long time to implement. The original idea to implement so many maps seemed reasonable at the time, I was going to carry vegetation, assets and textures across maps, sharing themes in batches of levels, so there would be a good chance to re-use content, but each map would certainly need a good amount of unique content to help with themes and puzzles. Eventually I decided to scale things back, and removed levels that were clearly going to be duplicates, this brought the total down to 8 maps.

Time passes, Epic Games released the first public version of Unreal Engine 4, with full source code access. This is significant for me, as alongside everything else, I have been maintaining my (3rd) full 3d game engine technology, complete with deferred renderer etc (the works). The release of the UE4 source code, let me guilt-free step away from my game engine. My engine is still there, hosted on a private GitHub repo, just in case, but the freedom of not having to maintain technology, as well as trying to implement content for a game (hence my motivation for using Unity in the first place) was a significant release. The day UE4 was available for download was the day I dumped all my (not insignificant) investment in Unity and moved my project over to ‘a proper engine’.

So, the goalposts had moved again. With UE4 comes expectations of higher production quality. With Unity, even version 4, there was an acceptance from developers and the public, that products would have a certain ‘indie’ look to them, so visually functional but likely lacking in advanced technologies and polish. So, there are and were great Unity games, that played really well, but… visually weren’t at the edge of what was possible, even for the time.

Most of my art assets didn’t look great in UE4. Vegetation, trees especially, looked far too low poly, having looked acceptable in Unity. Speedtree came to save the day a couple of months later, but that was more time and money, revisiting the vegetation for the game, and working around bugs and underperformance in UE4 during the earlier releases. CryEngine, the tech we had gone for with Rapture was famous for its amazing looking jungles trees/vegetation. I couldn’t shift engine again, and I knew from my time with the source code for CryEngine, it wasn’t the engine for me, UE4 looked much more like my own coding style, so I could get around it quickly and easily. I’d just have to wait and hope that Unreal would come to me, and eventually implement Open World rendering performance to match CryEngine, thankfully, eventually, it did – but there was worry for well over a year.

With the increased complexity for feeding UE4, the increase in model quality, the increase in well everything quality, some further scaling back was needed. I had brought over 7 levels from Unity, a starting location to set the story (a map for this purpose), a typical level (which I’d made the most progress on in Unity), a maze themed level (with which I wanted to re-create the need to draw your own maps for games like back in the ZX Spectrum days), a classic horror themed location (because I had ideas for some effects), a slightly urban area (because I liked the idea of the buildings and the change in challenge), a huge map (intended to revisit the games themes and gameplay elements one last time) and a finale map (with surprise ending).

The finale map worked out really well, it was implemented in UE4 pretty quickly, based around the ideas I’d had for the Unity version, the main problem I had was with Speedtree Pine trees taking considerably longer to render than my deciduous trees did, but I managed to hack around a balance of tree types that worked. There is still a little placeholder art in there, but this was the first content I’d implemented in UE4 that set a high standard and showed the potential for the other levels. I then focussed on the starting map, and completely failed to make it sit well with the game and look, the lighting was just wrong for UE4, it felt tacked on to the story, so after wasting a month or so trying to make it work, I dropped the first level and repurposed an area of the second map to play the same role. This worked much better, and it was a relief to move on and to drop yet another level from production.

Then my focus moved on to filling the levels with detail, I decided not to focus on the code side of things for a while, because I’d tested the basic premise of the game logic worked in Untiy, and it was better than expected even with basic art assets, so I felt that focussing in ‘artist mode’ for a while would enable efficiently when producing content, and reduce the number of plates to keep spinning. For a while I jumped from map to map, implementing content as my mood and any spare time allowed. While progress was being made, it didn’t feel like it, because no one map seemed to be getting any closer to finished. I decided to focus on one map, and to speed things up, I started setting students at the university tasks to prepare assets for the game, so I could focus my time on areas that needed implementing. This would have been a great idea, if the students could have been relied on to finish what they started, finish such on time, or even to a quality or technical specification that was usable in a shipping game. I realised this was my fault, not theirs, I’d expected too much from them at a time when they were still working out what would be expected of them in the ‘real world’. So future tasks would be smaller, more specific parts of the asset creation process that could be managed by final year students, so some occasional success was possible, especially focussed around character design (thanks @LeeAnnPicknett).

With UE4 came a solid implementation of Physically Based Rendering, so all the texture and material assets needed to be also be revisited, initially this was a case of kludging the Unity prepared content into Unreal, before finally experimenting with the PBR production tools available, I invested in both nDO/dDO and Substance Indie packages (Substance was a lucky grab via a Steam sale, a bargain at the time). With Substance eventually winning the battle for my limited time and becoming, at least until they increased the cost of ownership, my suite of choice and my recommendation to others (students) to use. The introduction of substances had a bigger impact on the production pipeline of 3D assets than I’d anticipated. At first substances, in their texture form, were used to replace the older pipeline assets from Unity, so just replacing one batch of textures for another. But with the increasing visual fidelity of the PBR assets came a shift in which elements of a model might be included in geometry and which might be represented within the material itself. The material (substance) started to be more important than the overall geometry. So, we had shifted from a model taking the time and you would apply an appropriate texture at the end to, the material had to look realistic, and you then needed to model around that. ‘Anyone could model’, but it was really critical to have good PBR materials to apply to the model, to sell the reality and believability of the 3D assets in game.

I started to realise, if I’d stayed in Unity 4, the game likely would have been finished and shipped a couple of years ago. The visual fidelity would have been lower, but the community was trained to expect that from Unity at the time. The game idea and play would likely have been very similar to how it is now in UE4. It probably wouldn’t make a major difference to sales either. The original intention had been to ship a quick turnaround 3D game that might stand out from 2D games on mobile devices. The hope had been the game might attract enough interest, due to being more ambitious than most for the time, maybe get noticed by Apple, main paged, and possibly generate enough income so I could pay for an ongoing contract artist, so I could focus on my day job, and do essential development in my own time, dropping provided assets into future project ideas, saving myself a bucket load of time and getting products out quickly.

Should I continue? I understand investment, and I’ve invested a lot in this game, even knowing that the window for success likely passed a good time ago, my investment into the project to date makes it necessary to continue. I’m a firm believer that the difference between a creator and a wannabe-creator is actually finishing stuff, for better or for worse. While I could walk away from my own game engine, thanks for the open model of development for UE4, I can’t do the same for my own game.

I have ‘rested’ the game a number of times over the past 2-3 years, stepping away for a couple of months at a time, mainly during especially busy periods at work where I’m too tired at the end of the day to think of doing any extra work at home. Each time I return to the game with fresh eyes, I’m pleased and impressed with what I’ve done. Somewhat surprised that given for most of my games industry career I was a lead programmer/producer/technical director/M.D. and never the artist (apart from the very early ZX Spectrum games), I have done well with the visual implementation of the game so far. UE4 expects to be fed properly, with good quality assets, so I’m pleased I’ve been flexible enough to adapt to the challenge.

I am trying to see this as a sort of novelist approach to games development, like a writer sat alone with his typewriter creating a story, I’ve been doing the same with a game engine.

If you’re interested in Retro games, below is a link to the site hosting Bob Pape’s story of making the Speccy (Sinclair ZX Spectrum) version of R-Type. For the record, I got the Time Scanner gig Bob refers to, and probably was involved in preparing the videos of R-Type he also mentions, but I forget the specifics after all the beers, I mean all the years…

As everyone is probably aware by now, Microsoft made a Developer Preview of Windows 8 available last week during their Build 2011 event. Microsoft has clearly learnt a lot from Apple and Google about hosting such events, even stretching to a Google style hardware give away for lucky attendees (grrr!). The keynote and workshop presentations are thick with enthusiastic evangelising of the ‘new’ (some previously available) technologies included in the platform, and I admit to being swept along with their enthusiasm.

I think the Metro platform is a good idea for Windows, even more so as a new API is available that sidesteps some of the monolithic Win32 code. Microsoft might have a chance of actually catching up (and even passing) Apple/Google in the ‘slate’ (note not-pad) race, although hardware/software price is likely to be a limiting factor for the initial year after the OS is released. Windows is one of those things you ‘just upgrade’, only occasionally is a release so bad you give it a break, people even upgraded to Vista before discovering it was just a little bit slow (much slower than the prior beta releases).

As a developer I’ve enthusiastically watched the announcements regarding the Windows Store, the Xbox Live integration (which I’m sure was already available on Windows but maybe it was all a dream?), and the accessible Metro platform, but as I’ve watched and researched the example code doubts have started to slip in.

I hope I’m not breaking any license agreement by discussing this and it seems to me other people are being pretty vocal on Microsoft’s feedback pages about topics raised so my current concerns are…

The Store – This is seems to be a closed space, where approval is required. Ok, I guess this is a good idea, because who can find anything useful on an Android device? It’s pretty hard to be seen. But equally, I see the same problem on my Windows 7 phone. The Xbox Live labelled content gets centre stage, promoted heavily by Microsoft and drowning other content. Indie developers even have to resort to including an ‘Xbox Live’ like banner in their app icon, in the hope of getting taken more seriously.

Related to this is the issue of getting apps into the Metro interface. As a ‘Windows’ OS I’d assumed you could install your software, like you’ve been able to do for decades now, and it’d appear on your ‘start menu’, now the Metro interface. It seems this might not be the case, at least if you’ve created a Metro specific app. My understanding is Metro apps will only be available via the store, with developers able to ‘sideload’ their apps.

App Development and Deployment – This brings the worry of a Windows Phone style app limit, where currently only 10 developer apps can be deployed – it’s a real pain. Often I’ll have several experiments on the go, in addition to my active titles in development, and I have to delete unfinished tests to try something else. This’d be too much to take if it was applied for Metro on Windows also. Then there’s App approval, currently Windows Phone Apps (plus Xbox Community Apps) are submitted through a portal at create.msdn.com. It’s not the most stable window on Microsoft and my experience has been prone to technical errors (including issuing 3 rather than 10 app deployments to Windows Phone), delays in responses for support or rapid pruning of forums.

C++What? – Taking a look at some of the Metro sample code, I had to do a double-take to check that I’d actually downloaded the C++ versions of the apps. What I seemed to be looking at was C#!? Given I have limited spare time to keep up to date, I’d assumed that the official specification of C++ had been seriously updated and it was probably time for me to get an up to date book (something less that 10+ years old). But having checked out more of the content on the Build 2011 site, it seems clear from the many upset community posts that Microsoft has been language building again, seemingly creating their own C++# or C## (doesn’t that already exist?). It’s not clear at this point how much of a pain this is going to be to interface ‘plain’ C++ code with, but I’m sure if something’s simple to do, this won’t be… (Yes, I’m a half-empty type of guy).

Xbox Live – I’m sure the Xbox live API has been available on Windows for a while, even with the ability to play against X360 players? Wasn’t there a FPS, which no one bought except me, that let you do this? Still we seem to be being told that this is available for the first time (maybe the first time in Metro?). I thought this ‘first time’ might indicate that it’ll be an open API for the first time, so excitement was high, until I got to the end of the presentation on the topic, when it because clear this was the same story as usual, only certain games will be allowed access to the technology. To get into the club you need to engage Microsoft’s interest, but I’ve already discovered over many years, not being based in the U.S.A. makes your chances very slim. Where apps outside the U.S.A. have been accepted, especially regarding Windows Phone, these seem to have been prior proven apps from other platforms, or apps developed within Microsoft’s group of companies. So there ends this excitement, unless I move to the U.S.A., San Francisco to be specific.

I’ve installed the Windows 8 Preview on three machines, my aging gaming rig, my not so old gaming laptop and my really old and actually dead before last Saturday x32 bit only XPS M1710 laptop (I finally attempted the GPU in the oven trick – and it worked). It ran well on all the platforms, although I had to do some trickery and use a hacked Windows 7 display driver to get GPU acceleration on the M1710. The only problem I encountered, which I would have realised if I’d read everything posted on the preview distribution page, was that you can’t build Metro apps on the x32 version of Windows 8, even if you install the Pro version of Visual Studio, which is a bit of a pain and I hope just a temporary issue due to the current distribution disk images. Still I hope to use this machine as a remote debug target, assuming it’ll be allowed to launch the Metro apps sent from an x64 machine.

So to conclude, I am still very interested and excited about the possibilities, but I’m worried this could still become a U.S.A. only club, as feels the case on Microsoft’s other stores platforms. The development experience could be ruined if they impose silly app deployment limits (an attempt to stop piracy, which probably won’t I expect) and issues of compatibility caused by ‘new’ versions of C++. The store could be swamped with Official Microsoft Approved Apps, drowning others, making developing content pointless.

Let’s hope Microsoft listen to the community, as they currently present themselves as doing, and make a few tweaks before release.

I’ve been intending to make this post for a number of weeks now, initially before taking part in the University of Portsmouth’s GameJam 2011. However, I spent the week before the game jam preparing, then the jam itself takes place over a week and finally I’ve needed the best part of the two weeks since to recover from it all!!! My post today relates to my first encounters and notes regarding developing for Android OS going back to April this year.

I’d initially downloaded the Android SDK shortly following it’s initial release, but as it was exclusively Java at the time, I knew performance for games was likely to be an issue (as was a lack of portability of the code to other games platforms), so I decided not to bother investigating further at the time.

Having attended GDC 2011 and given the recent release of the Honeycomb platform, I decided to acquire a Motorola Xoom and take the plunge with Android again.

The first step with working with a new platform is to install the SDK. The developer area on the Android site does a reasonable step-by-step job of guiding the user through the process of installing Android SDK, Ant, the NDK and unfortunately the Eclipse IDE. I’ve never had good experiences with Eclipse in the past, and it seems the same was destined to be true again. As I’m installing on Windows I initially allowed the Android SDK to install into the default ‘Program Files’ folder, but in a later install changed this (and other SDK components) to C:\Android to simplify path names.

Eclipse worked fine with regards to compiling Java code, the problems arose when trying to add the Native (code) Development Kit or NDK. It’s at this point the confusion caused by Google having released so many platform versions and NDK updates, in such a short time, raises its ugly head. Various Google searches, looking for clues as to whether NDK content should compile within Eclipse created an impression that such code didn’t actually compile within the IDE and that you had to drop out to a command line to compile such each time you needed to. Other sources however, seemed to indicate that NDK code could in fact be compiled from within Eclipse. A very confusing situation, not made any clearer by Google’s own build instructions.

I spent about 4 weeks (in my spare time) trying to ensure environment variables were correct, uninstalling and re-installing various packages, Eclipse plugins and even Motorola’s own version of the Eclipse environment which intends to take the pain out of the process by installing everything for you! But still the NDK content wouldn’t compile from within Eclipse. I even tried installing the OSX versions of everything and had no luck there either.

Eventually I attended a Motorola MotoDev event in London, and after hassling a support technician several times, I was able to determine that NDK content should be able to compile from within Eclipse, but as luck would have it, their demonstration machines weren’t configured to support NDK development on the day… So no luck there finding the cause of my problem.

Any novice developer would have probably given up weeks ago, this is a shocking state for an ‘OS SDK’ to be in.

I eventually determined that the issue I was having was due to the most recent updates to the NDK SDK, apparently Google had changed the directory structure for r5 from r4, so a third party plugin enabling compilation of C/C++ code within Eclipse wasn’t working. What didn’t help with the confusion however was that the plugin’s developers were reporting having fixed it to work with r5, so why wasn’t it working for me? At the point of giving up I realised that I was using the newly released (at the time) r5b release! Oh my! Even a subversion change had also broken the C/C++ plugin! All of this chaos was compounded by the odd behavior of Eclipse which would crash, show dialog errors and generally had a difficult time doing its job as an IDE.

Google don’t seem to provide links to previous versions of the NDK on their site, but I was able to download other versions by guessing and manually editing the URL for the current SDK version to access the others.

At this point I gave up with Eclipse, it seemed unstable, and I’d already spent over 4 weeks (admittedly not full time) trying to get a tool that should increase productivity to work and I just had to get on with something productive or call it a day for Android. I’d have to go back… Back to the Command line!!! (cue BTTF soundtrack).

I should have done this in the first place. While feeling very retro 1990s, a similar ‘hacked’ environment to home brew Nintendo Gameboy development kits (where only minimum tools and no useful debugger were available), I was finally able to actually get on with some actual development!!! I’m now in a situation where I can compile and run code for Android, then take the app specific code/assets and rebuild them to run on iOS, which is pretty cool and lets my aging MacBook Pro take a rest during the grunt work.

Following are the notes I made for myself, regarding the fundamentals of command line compiling for Android, some of you might find them useful:

Android Command Line Building

The cygwin command line won’t see the ‘android’ command even though it’s in the path, needed to explicitly include the .bat extension.

First impressions are good, the Metro interface has evolved since Win Phone 7, with some clever tricks such as pulling in from the border of the screen to bring background tasks to the fore. However I can see this causing chaos for some game controls that stray too near the border of the screen. The large tile area per app allows for widget-like behaviour and information to be presented which I liked.

‘Classic’ (traditional windows) mode and the Metro interface switch near instantly, allowing the user to run their existing Win32 apps, but I don’t get the impression that a single app can be made to run on both (Metro & classic), at least from what’s been shown so far. However apps are built to run on ARM and x86 platforms using the (new?) SDK.

So is the Metro interface a layer running on top of windows, or is it more closely integrated into the OS as a whole? Is DirectX available for Metro apps? Will we be forced to use a silly language (C#) to implement Metro Apps, using managed languages as done on WP7? I’d really hope not, I’d like to port my C++ Game Engine tech over to the Metro interface.

There’s a Build event scheduled for September, so maybe more will become clear by then.

My intention is to keep a Games Development Blog, with a focus on my Indie games development projects.

In general I don’t follow blogs myself, but I have found them really useful when doing an Internet search for tips and tricks for the various problems I’ve had with my projects. My intention is to give a little back, updating on my discoveries so others might find them from a similar search.

Also I hope to make the public aware of my varoius projects. It’d be nice if someone was aware of a games release before it hits an app store and maybe I can get a little feedback before release too.