other stuff

Earlier this year, I was invited to port the remastered re-release of classic point and click adventure Day of the Tentacle to GNU/Linux.

With a successful release now behind me, I wanted to take some time to explore the experiences and learnings I've gained both from working on DotT Remastered and from developing my own software on Linux.

A piece of fan art I created in 2014 when it was originally announced that Day of the Tentacle Remastered would support Linux. Larger sizes can be found here.

In this article, we'll take a look at what it's like to get started on a port, and what kind of ups, downs and betweens are likely to be found along the way. In addition to Day of the Tentacle related anecdotes, the article also includes quotes from several other Linux porters who share some of the perspectives they've gained throughout their own porting careers.

Day Of The Tentacle And Me

In 1996, I purchased my first computer. For the previous eight years, I'd been content with sharing the family Amiga 500, but traveling between my parents' houses and starting high school provided enough justification for me to be able to convince the Gatekeepers of Money to smile upon me and allow me to purchase a Toshiba KTX 880. Along with my shiny new laptop, I'd also managed to weasel for myself copies of the first two LucasArts Archives Volumes.

Although I'd played and loved both Monkey Island games (at the time, there were only two), Star Wars was my priority - the Special Edition re-releases of the classic trilogy were around the corner and fourteen year old me, already being intensely interested in any piece of behind-the-scenes background information he could find, ended up deciding to grab Volume I specifically for Star Wars Screen Entertainment. Star Wars Screen Entertainment was an assembly of screensaver-ish apps and "a special message from George Lucas on the next Star Wars trilogy".

The LucasArts Archives Volume I.

As fun as watching Jawas steal your icons and portions of your desktop background is, or how enthralling an endless cartoon rendition of Obi-Wan's lightsaber battle with Darth Vader can be, I quickly absorbed as much information as SWSE had to offer and lost interest in it (the storyboards "module", which also included a copy of the film's script was great though!) and turned my attention towards the collection's other titles.

If The Secret of Monkey Island was the dawn of the Golden Age of Adventure Games, ushering in a range of new sensibilities based on Ron Gilbert's Why Adventure Games Suck, then for me, Day of the Tentacle was where LucasArts hit their stride. Without belittling the games that came before it, or suggesting that Day of the Tentacle is without its own fair share of rough edges, DotT feels more like the product of matured technologies and seasoned creatives who know their craft. In my mind, it raised the bar and set a new standard for the games that would follow.

My Day of the Tentacle disc along with some limited run postcards I printed to send out as thank-you messages.

Day of the Tentacle manages to keep a significantly complex and non-linear premise feeling well paced and understandable - that in itself is an incredible challenge. The writing is top-notch and the animation is fantastic. The dialogue and voice acting is pretty good considering that it was the first LucasArts adventure game to be designed as a "talkie". So far as I am aware, Day of the Tentacle is also the first LucasArts adventure to have a time based in-game hint system designed to give "meanwhile" prompts to help steer players towards and help contextualise unfinished goals.

While Day of the Tentacle may not be my favourite game, my favourite LucasArts adventure game or even my favourite Tim Schafer or Dave Grossman game, it is one I love dearly. Beyond that, it stands out to me as being an important title in the history of the adventure game genre, and I was overjoyed when I saw the announcement that a remastered re-release would be coming to all three major desktop platforms.

The opportunity to share a game I care about with people on the platform I use always makes me gleefully happy. Unfortunately, circumstances conspired to see the Mac and Linux ports arrive after the release of the Windows version. When I became aware of this, I offered to assist with making a Linux release doable in any way I could. At the time, I'd envisioned that that would be some sort of consultancy or support role, and after negotiating to assist with Linux Quality Assurance (QA), it came as a great surprise when I was offered the opportunity to port the game myself starting in May 2016.

My personal connection to this game has expanded from being that of a player and an appreciator of its accomplishments to include that of a developer and in some respects a historian. I've had the opportunity not only to peek behind the curtain and experience a game I like from a new angle, but I've also been able to look back in time at a fascinating cross-section of LucasArts history. I've seen SCUMM code from the 80s written for Maniac Mansion (in one file, there's a lovely code comment saying there's a front row seat in hell for me if I edit it - I like to pretend that that's Ron Gilbert's "grumpy" persona yelling at me from across the decades), I've seen early 90s work from Day of the Tentacle's development, I've seen the "Remonkeyed" engine code written for the Monkey Island special editions, and I've seen Double Fine's work on adapting that for DotT's remastered edition.

An image I created to tease Day of the Tentacle Remastered's release date. Larger sizes can be found here.

All of this is an opportunity that I'd never dreamed of having. In so many ways, it has been a positive experience, but it has not all been rainbows and basking in the radiance of a cool game. There's hard work and long hours involved, no small amount of learning, and a modest amount of hurdles (which was to be expected - it's rare to have complex code run without problems the first time it's built, and I get the impression from speaking with other porters, that it's even rarer when it's a port of existing code that doesn't target your platform). All things considered though, I feel like I had a comparatively easy job. Day of the Tentacle Remastered's codebase already had a large amount of platform abstraction before I began work, and thanks to the Mac and Playstation ports, it also had a functioning OpenGL renderer and POSIX friendly implementations for some functionality.

Two months' worth of work later, Day of the Tentacle Remastered for Linux released. I found myself feeling equal parts anxiousness and excitement (which is normal for a big release) as I both looked forward to and dreaded discovering the reception that the game would have.

Asking For Advice

When it looked like this project would be falling into my lap, I reached out to a number of friends and acquaintances with porting experience to ask if they had any advice to share that might be of value for a first-time porter.

Cheeseness: Looks like I just landed my first porting contract (don't know whether to be happy or worried >_< )

Cheeseness: flibitijibibo: Any tips for surviving when you're in over your head that don't involve drinking?

Having more than a vague idea of the sorts of things that would be involved, I knew that there'd be pitfalls and gotchas which would eat up time and attention with little meaningful gain beyond a learning opportunity. Equipping myself with knowledge seemed like the only sensible way to avoid them.

The comments I received were a mix of rueful anecdotes (which discretion necessitates I keep to myself), self-reflection and encouragement. Some were helpful in preparing me for my adventure, and the rest was still valuable for strengthening relationships and camaraderie. The response I got back from Ryan Gordon took the form of a published changelog from his first freelance port - a fantastic treasure and piece of porting history that I was very happy to see unearthed.

Throughout this article, I have quoted insights from some of these porters[1]. Their perspectives are diverse, and conflict with my own opinions as often as agree with them. Different people experience things differently and have different relationships with their work. Some approaches may be more healthy than others - I'll leave it to readers to work out the best fit for themselves.

Don't worry too much about having to know everything going in — not every port requires a knowledge of everything, and every port will involve learning a whole bunch of new things, be they languages or APIs or version control systems. Also, don't be afraid of asking for help — every porter I've met has been really friendly, and will no doubt be
more than happy to answer questions, or give advice. - David Gow

With Day of the Tentacle, I was lucky to have access to Oliver Franzke, who as well as being a friend of mine was (amongst other things) also Lead Programmer on the Day of the Tentacle Remastered, Rendering Engineer on The Secret of Monkey Island: Special Edition and VFX Engineer on Monkey Island 2: Special Edition.

Oliver was kind enough to give me feedback on my initial impressions of the codebase and porting time estimates. Having someone with his level of familiarity be able to confirm my guesses at where the majority of platform specific code was concentrated and to point me towards a few bits and pieces I hadn't noticed was immensely valuable in allowing me to approach the project with confidence.

Planning And Prioritising

While the project itself didn't have any publicly announced dates, I still had time related pressures (both internal and external) that influenced my approach.

Initially I had planned to restrict myself to eight hour working days and take weekends off. I'd also set up a "pomodoro timer" with the intention of making sure that I never stayed sitting down for periods longer than 3 hours, but as circumstances shuffled into place, I found myself abandoning that in favour of taking ad-hoc breaks. Looking back, I'm not entirely certain that I gained any meaningful amount of productivity from doing this and I'd advise anybody who'll listen against it.

From the outset, I was keenly aware that the codebase I'd be working in was of a scale beyond anything I had contributed to before. I'd been given the opportunity to review the codebase before signing a contract, but that really wasn't enough of a window to get the familiarity that I'd need to deliver a finished port - I'd need to be able to learn as I went without slowing myself down too much.

[For] prioritising? Well, I just try to do first things first: dependencies must work before the dependents. Can't have a renderer running if you can't build your binary and e.g. link in all the middlewares.

I set myself up a rough plan that would allow me to focus on achievable things while also exposing myself to more of the codebase as I went, which looked something like this:

Set up my build environment ahead of time

Work out-of-tree so that exploration and experimentation doesn't get committed

Get the codebase to compile (stub out anything that looks like it needs rewriting)

Sift back through looking for platform specific #if directives that might've been missed

Look through all build scripts for any platform specific stuff that might've been missed

Prepare an initial port commit and start working in-tree

Rewrite any stubbed code

Focus on fixing crashes/behavioural differences

By treating compilation as my initial goal, I'd be able to make forward progress with zero knowledge of the codebase - compiler errors would tell me where to look, and I'd learn bits and pieces about the project's structure as I went through doing superficial stuff like correcting case issues and making Linux specific copies of other platforms' #if directives.

The process of porting a game tends to break down nicely into stages:

Get the code.

Hack at the code until it compiles. (Comment/STUB() out whatever code
you need to.)

Slowly implement things until it runs without crashing.

Now make the game actually playable.

Performance and polish.

Release. (This often involves dealing with publishers, people wanting to "tie in" the release of the port with events, etc.)

As soon as the project compiles, that's a benchmark against which further progress can be gauged. Until that point, it's hard to have a solid understanding of what needs attention and what impact changes are having. For example, a fix for a problem that causes a build process to bail at 80% might expose another problem that manifests itself 30% of the way through the build process. When a project compiles (even if it crashes before it does anything), it is (generally) easier to see the impact that changes have.

Similarly, using grep to hunt for platform specific code in source files and build scripts (providing the project at hand is on multiple platforms) is something that can be done without any existing knowledge, while at the same time giving more insight into the kinds of platform specific concerns that other developers on the project had previously encountered.

At this point, it's hard to get any further insight "for free", and there undoubtedly is still a lot of remaining work that primarily involves digging through code and working out what things are meant to do, but every little bit helps!

In general, this kind of prioritisation is likely to map against the approach that a more familiar and/or experienced porter may take for reasons that have nothing to do with maximising opportunities to discover and learn about a codebase, but being mindful of what I could learn as I went has definitely helped me.

I have an internal release schedule based on when contracts start and that's about it. Everything else, I pick what to fix based on what I feel like looking at. - Ethan Lee

As part of my initial review of the code, I set about making sure that I could build the game on Mac OS. Having the ability to build and run the game on a system it already supported (especially one that I'd be basing my port on) made it easier to see how things were meant to work and identify any impact that my changes may have had.

Even though the game supports some older versions of Mac OS, it makes use of some version defines that weren't available on my system. My first code modification ended up being to comment out a Mac OS version check (which I never committed).

A screenshot from the first time the game rendered anything on Linux. Rendering bugs are a result of the game being unhappy with llvmpipe.

With the approach listed above, I had the game compiling on day 9, a window being created on day 11, and the game being playable (for a very loose definition of playable) on day 21. The rest of the project's duration was primarily taken up with identifying and correcting crashes and other undesirable behaviour.

Porting Is Not Software Development

Something that was both expected and a surprise to realise was that my experiences of porting didn't really fall into the realm of what I would normally refer to as programming. In fact, reflecting on my experiences, I feel that it's probably fair to consider that it's a porter's goal to write as little code as possible.

All of the creative/design decisions around code structure, systems behaviour and dependencies are typically locked in place, and deviating from them means a port that's less likely to be maintained once it's completed.

Remember — the code your working on isn't yours, you're merely its custodian. People will need to work on it after you're gone. Do your best to make sure that the game will still be able to be played in the future, long after you have gone. - David Gow

The other really important thing to know when planning is if you're going to be working on the main branch of the game, or in a separate branch which will likely never be merged. You need to decide how much freedom you have to rewrite or refactor existing code, versus carefully changing only what is required. When in doubt, don't change things you don't have to, and document your changes well. - David Gow

The value of programming experience and familiarity with the language(s) used can definitely not be understated, but the amount of time spent writing original code (as opposed to tweaking/refactoring/fixing code that already exists) feels so small compared to time spent setting up build environments, reading code, reading documentation, debugging, etc. that it paradoxically seems negligible.

In some ways, all this is liberating, but in other ways, it can feel restrictive. Replacing dependencies or fundamentally changing the behaviour of a set of functions isn't something that can be done lightly if integrating a port with the main codebase rather than leaving it as a fork is a priority (and it should be!). A porter often has to work with and respect decisions and conventions that may be at odds with their own preferences and sensibilities (which isn't inherently bad or unpleasant).

When porting, all the code is already there, you just have to make it work. When doing your own development you have to develop functionality in addition to making it actually work. - Aaron Melcher

I do want to be clear that I don't mean to belittle the work of any porters or that I would prefer to refer to people working on ports as anything other than programmers or developers. Porting is just a different kind of work to what I'd consider "game development" or "software development" to usually entail, and if someone were to enter into a port expecting to be able to make the kinds of creative decisions that (to me) are core to the experience of software development, that person would be in for an unpleasant surprise.

Porting was a huge disappointment for me as an experience. I was very much looking forward to it when I started, but found that there is little satisfaction from it. For the most part, you're just making sure that someone else's ideas run well in a different environment. There are some brighter spots when you develop tools to help you in porting (see Ryan Gordon's Mojo* project family, or Ethan Lee's FNA, or my glext-emu for emulating Direct State Access on OpenGL drivers that don't support it), but in general, I have learnt that porting is a difficult, unrewarding job that dulls your creativity.

Developing your own software, on the other hand, is also difficult, but at the same time creatively stimulating, and thus, inherently satisfying. For me, that is, and it's the reason I got back to regular game development.

All things considered though, the Day of the Tentacle codebase already primarily made use of cross-platform libraries and had a good amount of platform specific abstraction before I became involved, thanks to the foresight of the team at LucasArts who put together the Monkey Island Special Edition engine in 2008. The engine that DotT inherited already had a functional OpenGL renderer and POSIX implementations for a fair amount of functionality. Without these, my workload would have been greatly increased, and I would have had to rewrite more code.

Don't Be Alone

This may be a point of contention, as I do know some porters who don't agree with this outlook, but it seems important to me to make sure that I have lines of contact to developers who have previously worked on the project.

If there's a piece of code that needs to be understood before forward progress can be made, and that's either going to take two days to work through alone or twenty minutes' worth of conversation time, then it's pretty clear which approach is in the best interests of the project.

When you're the only porter, remember that you're not the only developer. You can often ask the original developers a few questions, or work with the developers of the libraries you're using, or indeed with other porters or developers on other games. The driver developers (especially the open-source driver developers) are usually more than happy to help out if you've got a problem on their hardware. Finally, testers/QA for the port are usually more than happy to help out, and particularly with Linux users, you can often have surprisingly detailed technical discussions. Ultimately, you may be the only person with the complete picture, but there are lots of people who can help with little bits. - David Gow

You're only at the stage of needing help or knowing you don't need help once you've started writing and have exhausted your ability to search further on your own (you are only allowed to give up once you are physically unable to read the screen anymore, literal exhaustion). And yes, this does include the latter category; the upper bounds are the part you should understand the most in the event that something hits you over the head with it. - Ethan Lee

While I would encourage considering the experience and insight of other developers as a bonus resource, I also think it's very important to respect their time and availability. They are (most likely) under no obligation to offer advice and may not have relevant insight to offer, but there's no harm in asking if someone is available for a quick chat.

This sort of work can also be significantly isolating, particularly when working on proprietary projects, and there's a lot to be said for the psychological benefits of having someone to talk to (talking to yourself in code comments can only get you so far).

People are cool and all, but I enjoy not having to depend on anyone. - Aaron Melcher

Different people respond differently to varying working environments, and the kinds of isolating and time related pressures that this type of porting work normally entails aren't a good match for me personally. Even though it lead to disappointment for some eager Linux users, the lack of a target release date allowed me to sidestep a lot of stress that could have slowed me down. That said, there was definitely an element of time pressure involved - instead of a countdown to a release date, I had the number of days since the game's Windows release steadily counting upward, and every few days, I'd hear from a Linux user who was frustrated by the wait[2].

In contrast to this, all my contacts at Double Fine were positive and relaxed. Their calm everything-is-normal attitudes helped me focus on my work without panicking over deadlines or projected dates. The value of Oliver Franzke's support on this project[3] can not be overstated, and insights from Trevor Diem of Shiny Shoe also helped me save time by narrowing down some unknowns that I could have wandered aimlessly in for much longer on my own.

Knowing Your Tools

Compilers (never forget that there are multiple ones and that alternative compilers may give additional insight into or reveal problematic code) are the key tools in porting. They take abstracted code and compile it into binaries containing platform specific instructions or bytecodes. I imagine that anybody reading this is already aware of that, but it's important to keep in mind that without a compiler, porting would come down to re-writing everything in assembly.

Because the compiler is playing such a huge role in a porting task, it is vitally important to be as familiar as possible with its behaviour. It is critical to not only understand any platform specific options it might have, but also what diagnostic and warning flags are available which might help track down problematic code (regardless of the compiler, chances are that there are dozens of useful options kicking around).

Read and write until you are physically unable to do anything at all, then rest on it until you can continue the next day. That way, you'll be too tired to ask for help prematurely until you're at the position where there's nothing left to read or write and you're still conscious. You say this is crazy, I say it's your best bet at actually reading the documentation and source code for the projects you're about to tightly couple with the quality of your life for the next several years. - Ethan Lee

Awareness of more idiosyncratic behaviour, such as how a compiler handles code that is "undefined" according to relevant language specifications or any outstanding known bugs, is helpful too. Beyond this, if the project is tied to a specific compiler version (say, the versions of GCC and Clang that come with the Steam Runtime's build environment, for example), then it's also worth being up to speed with bugs that have been closed and fixes that have been implemented after that release.

In general, those same levels of thorough familiarity can be valuable to have with debuggers, language specifications, libraries and even the target platform itself. Getting to know their intended behaviour, as well as their error handling, logging and other diagnostic functionality can also help make identifying problematic code easier.

Avoid adding too many external libraries or middleware if you don't trust them. Sometimes you have to, and sometimes they help, but for everything extra you add, you'll have something extra to fix. Make sure you know and check what the licensing rules for libraries and middleware are — you don't want to get in trouble because you used ffmpeg (GPL — you need to release your source) to decode h264 (patent-encumbered). If you don't have the source to a library or piece of middleware, you're going to have a bad time — sometimes it's unavoidable, but having source is better than only having a disassembler. - David Gow

Shortly after getting the Day of the Tentacle running (an early milestone), I had encountered an issue where audio cues were firing incorrectly[4]. Advice from other developers who'd worked on the game was that it was most likely an interrupt loop that controlled the audio system being too far outside of its target 300ms interval, but nothing I tried on that front seemed to help. In an effort to track down the issue, I'd upgraded GCC to gain access to newer compiler flags that I hoped would help me narrow down my problem. In one late night flurry of bugfixes, I'd announced the problem solved by some unrelated commit and moved on.

When the time came to build the project inside the Steam Runtime, which just so happens to ship with GCC 4.8.1, the problem reared its unpleasant head again. After several days of slowly stepping through the game and scrutinising its behaviour, I discovered that somewhere between 4.8.5 and 4.9.2 (I couldn't spot anything useful in the GCC changelogs, and it didn't seem in the project's best interests to spend time digging out the relevant commit), the behaviour of the following C++ syntax, which is undefined in the language spec[5], had changed.

ptr += some_fuction(&ptr);

The ambiguity arises when this statement can be interpreted as either ptr = ptr + some_function(&ptr) or ptr = some_function(&ptr) + ptr. In the case we care about, the function modifies the value of ptr, and was used to calculate the length of data (say, an int representing how long an audio cue should last) in packed in memory. Instead of getting the correct memory address to read our value from, we end up reading partway through some other value and things spin wildly out of control from there. GCC's -Wsequence-point compiler flag is meant to help identify these kinds of problems, but it seems that the involvement of a function that manipulates the value of ptr confuses things a little.

A screenshot from the first time the opening cutscene rendered on Linux.

Newer versions of GCC generate code that provides similar behaviour to Clang, MSVC and the other compilers used for building Day of the Tentacle for other platforms, which on the surface sounds nice, but undefined behaviour is undefined behaviour and until a spec arrives to clarify these things, I think it's best to not expect consistent behaviour anywhere. Up until I'd narrowed down the problem to that specific line, the statement responsible for my problems had seemed innocuous. Better familiarity with the relevant C++ specs would have helped me identify it much sooner.

My friends and acquaintances who have made a career out of porting games reassure me that stumbling across on these kinds of hurdles is normal, and the knowledge and experience to navigate them without getting stuck typically requires having had to overcome them "the hard way" once or twice.

You'll write your own tools, libraries, wrappers, etc. Having your own quick implementations of basic things like case-insensitive file loading, or data structures like hashtables, or a header with all those annoying windows.h types can come in handy again and again. - David Gow

One friend and porter who would prefer to remain anonymous has passed on to me an aphorism that I found amusing: the more time spent on a bug, the smaller the fix. It's meant to be taken as a Murphy's Law type reassurance that tripping up on things that seem small in hindsight is normal, but it reflects some interesting characteristics of problem solving.

Small issues such as the one mentioned above are more easily overlooked, and the longer they wear on, the more frustrating they can become. With each hour, day, week or month that an elusive problem consumes, the number of more-obvious causes it could be hiding under dwindle. We learn about common problems like off-by-one errors and learn to look out for them, but until we've gotten that perspective, it's awfully difficult to spot them for what they are. Some things are just easier to see with experience, I think.

01112002 - Found a division by zero in the ray casting routines that was causing all sorts of trouble if Sam's gun was pointing straight down (Engine/World/WorldRayCasting.cpp). - Ryan C. Gordon, Serious Sam 1 Linux changelog

Knowing Your Project

Deep familiarity with the project and its behaviour on other platforms feels super essential, and being able to do comparative debugging against builds on another platform that the project is known to be functioning correctly on has been immensely helpful for me.

Being able to identify when something is close-but-not-quite-right can mean discovering problems earlier. If a difficult to spot problem is identified before the project hits QA, it can be worked on without incurring the communication overhead of whatever relevant QA bug reporting processes are in place.

Understanding the role and behaviour of any dependencies that the project relies on is important as well, particularly when it comes to any platform specific issues or shortcomings those dependencies may have. This knowledge can help inform decisions on how worthwhile persevering with a problematic dependency may be.

If a project is using an unfamiliar version control system (VCS), it's worth spending time getting familiar with it and keeping documentation on-hand. While version control systems typically have a way to roll back problematic commits, that's disruptive, creates extra work, and is worth avoiding. Using a separate VCS to keep track of porting work and then using diff/patch to apply those changes to the project's VCS or making use of bridging projects (such as git-p4) are ways of incorporating familiar version control systems into a porting workflow, but it's still important to be aware of how the project's VCS behaves regardless.

Don't worry too much about doing the perfect port — you never will get it quite perfect. Sometimes you've got the luxury of patching it after it's released, sometimes you don't get to patch it at all. A successful port is all about knowing which shortcuts to take, and these can change from project to project. If there are only a couple of shaders, port them yourself, but if there are lots, just use mojoshader. - David Gow

Day of the Tentacle has been a fascinating project to work on, and has given me opportunities to discover and learn about tools and techniques that I wasn't previously familiar with.

My first hurdle turned out to be an embarrassing miscalculation on my part. Forgetting that the repo also included asset sources and release ready versions of the packed data file, I'd ended up making my development Virtual Machine (VM) too small to actually build the project. Through an unfortunate coincidence, I'd also not left enough room on my host system to resize the VM (a process which effectively uses double the VM image's size worth of space) and ended up losing a day to shuffling files around.

Also, as a form of autobiography, you can see when I started to sputter out. I had fixed the big problems, like it not running at all, and was at the point where every problem would take longer and longer to attack for less and less return on that investment. And then, I stopped taking notes. You can see it end after four months of silence, not with a bang but a whimper, on April 24th, 2002, when I wrote the date, no notes, and left a few sad wishlist bugs at the bottom, probably never revisited. By this point, all the notes would have been me banging my head against the wall, trying a million unsuccessful things, cursing my limited tools and time and skills and maybe my own mortality, and just eventually saying fuck it. - Ryan C. Gordon, .plan notes on Serious Sam 1 Linux changelog

After spending many years becoming comfortable with Git, working with Perforce proved to be a bumpy ride. I've heard that it is good at handling large files, but since I wasn't actually committing changes to any large files, I can't comment on that. I managed to avoid getting the repository into an undesirable state (something I did a lot when I first started using Git and had done from time to time with Subversion before that), but I did discover that Perforce requires a specific command ("reconcile") to be run for it to notice changes in files that haven't been checked out.

I knew that file name case issues would be something I'd have to deal with, but my initial assumptions were that that would primarily be down to dealing with file loading at runtime (assets, save files, etc.). As it turned out, the project was able to load all external resources comfortably thanks to a couple of tweaks by Matt Russell of Shiny Shoe before I started. I soon discovered however, that there were still case sensitivity issues abound within the codebase itself.

Primarily hiding within original SCUMM source files were discrepancies between the case of file names and the case used in #include statements. Most of the #include statements were consistent, but unfortunately it turned out to be impractical to change the case of the files themselves (apparently something to do with the repo's configuration). The work of checking and updating #include statements straightforward and compiler errors helped make sure I didn't miss anything, but it was also tedious and time consuming, and would have been good to consider in my initial review of the codebase.

This isn't normally acceptable practice, but for porting, there isn't usually a direct set of changes you made precisely--there will be later, when the thing is mostly working and you're applying spackle to all the cracks--and you aren't usually working with anyone else anyhow and if you had to look 100 revisions in either direction it's just an unbuildable wasteland anyhow. - Ryan C. Gordon, .plan notes on Serious Sam 1 Linux changelog

I hadn't previously worked with FMOD, but I did encounter a couple of noteworthy hurdles. The first is that FMOD internally sets the stack size for its threads (to a value that seems to be slightly below what would have been convenient for DotT). I don't have any advice on workarounds for this, but it's definitely worth being aware of.

The second hurdle was related to naming audio streams so that they are given useful labels in audio mixers. Typically, we'd use sound driver specific environment variables to control this on Linux, but FMOD sets those internally, resulting in all streams being called "FMOD Audio". FMOD's initialisation functions take a generic parameter for driver specific data. For some sound drivers this is a struct containing a bunch of different values, for others it is a char array containing the file path to write wav data to. The documentation that I've been able to find doesn't cover any Linux specific drivers, but I did eventually stumble across a Q&A post by Jørgen Tjernø (former Valve developer, former Uber Entertainment developer, current Undead Labs developer and cool person) in which he queries how to set audio stream names on Linux, and eventually discovers on his own that passing a char array in as the extra driver data parameter will do this. With FMOD automatically selecting the appropriate driver to use, and behaviour being undocumented for the majority of supported drivers, this solution made me feel uneasy, but it seems to work.

The third issue didn't rear its head until after the game was released. On systems running ALSA (and possibly other audio systems, but not PulseAudio), it turns out that FMOD isn't good at respecting the default output device, and also seems to have issues with parsing certain configurations[6]. If the opportunity to deploy a patch arises, I'm hoping to expose command line arguments for retrieving the indices of enumerated audio devices and for setting the preferred index in an effort to provide at least a small amount of flexibility to users with multiple output devices present - a suggestion offered by fellow porter Ethan, who has done something similar with some of his ports.

[Familiarity with your tools, your project and yourself] sound like pretty obvious things to me, though I find that it's even more valuable to not be afraid to play with the unfamiliar. Comfort zone is a deadly place, and it's what keeps many game developers away from Linux. - Leszek Godlewski

Another "gotcha" that I thankfully had the opportunity to discover and resolve before release was "large file support". With Day of the Tentacle's data file weighing in at 2.7GB, its size moves beyond the number of bytes that can be represented by a signed 32 bit integer. Defining _FILE_OFFSET_BITS to be 64 at compile time, allows the game to make use of large file friendly IO in glibc. This is really only an issue for 32 bit applications (at least until the seemingly gargantuan 64 bit limit is exceeded), but one that's worth having awareness of.

When it came time to move into testing, we encountered a fairly common and awkward situation. To allow access to testers who had normal subscriptions to the game on Steam, we would also be exposing Linux users to partial Steam configuration, which would cause the game to show up in their library and be listed as downloadable. Steam would "download" an empty folder and the first real indication that the player would get of something being wrong would be a misleading "missing executable" error. In other projects I've been involved with, this has generated support requests and grumbles from people assuming that developers don't know what they're doing.

Following the example set by some other porters (Ryan and Ethan in particular have been known to do this), I spent half an hour putting together a quick application[7] that could be placed in the default non-beta branch and let Linux users know that the game was still coming soon.

A screenshot of the "coming soon" app. Sources can be found on GitHub.

Users who were keen to pick up the game ahead of the Linux release could now do so with an increased level of confidence that a port would eventuate. Linux users were also able to receive Steam card drops from running the coming soon app, and those that panic about platform stats were able to have peace of mind that there would be Linux playtime counted within their first week.

To spice things up a little, I used a piece of fan art depicting a scene from the game's opening cinematic, but with a penguin (the universal symbol for Linux) in the place of Purple Tentacle - a piece we also used for general promotion of the port. This was well received on multiple levels and signaled some of the negative comments I was getting from Linux users who were frustrated at the Linux release's lead time/missing out on sale discounts that users of other platforms could participate in. Excitement and anticipation for the port increased and ultimately, the "coming soon" app was a success. Even after release, I'm still receiving positive comments about it.

Knowing Yourself

When working on large scale projects, especially potentially isolating ones, maintaining perspective can be difficult. Small problems and pressures can feel amplified. The natural ebb and flow of optimism and pessimism can easily cloud any sense of progress. Keeping track of known remaining work, known areas of uncertainty and maintaining a roadmap for approaching those can be useful in staying on-track, but without acknowledging that any bug or work item may be obscuring additional work, that kind of planning can easily set false expectations for how close the end of a project may be.

I've always preferred total isolation, so it's a pretty sweet deal. Before getting engaged I used to live months at a time away from people entirely (oh, to have a 24-hour grocery store again), so even in my personal life I'm not bothered by it. And truthfully, NDAs are easy when other things are going on in the world. Focus on that instead. Save it for when you're done, when you'll actually understand what you've done well enough to make the subject interesting to anyone else in the first place. - Ethan Lee

Recognising that emotional and psychological state impact heavily on productivity, and making appropriate decisions around when to schedule breaks or recreational activities can help make sure that longer projects don't become overwhelming. In general, being aware of one's personal limits and reactions is important (see my game jam survival guide for some more thoughts on that) for being able to navigate and cope effectively with pressures and constraints.

Typically, I like to keep a rough window of time defined by best-case-scenario estimates and conservative not-so-good guesses for remaining work. When a relevant date (whether it's an external milestone or an internal personal goal) encroaches upon that window, I start adjusting my own expectations and make sure to communicate that upstream to whatever relevant contacts may need to know. Avoiding unpleasant surprises and being able to get the ball rolling on extensions or recalculated timeframes early can help make those (often unavoidable) situations easier to deal with and feel comfortable with.

Trello/slack when working with collaborators. Todo.txt file when solo. Always prioritize based off functional requirements. I spend zero time on code cleanup and/or refactoring. Always try to get first visible as fast as possible. - Aaron Melcher

To help keep track of progress and assist with triaging, I keep a log of thoughts, approaches and solutions for completed work as well as a list of known needed work. Typically this is more verbose and rambling than is appropriate for commit messages, but a bug tracker can be a good fit. Most of the time, I make do with a plain text file.

Having that kind of information available has been super helpful in keeping momentum and feel more comfortable stopping for sleep when in the middle of solving a problem.

Bernd Kreimeier was the master of the Loki changelog, and he insisted every day that I write my own, too, because you can't write it later. That information evaporates into the air. So I didn't just write what I changed in the source code, I wrote what I was planning, what my upcoming concerns were, what I tried that didn't pan out, other things you had to do on the project outside of revision control, like the time you spent rebuilding a Win95 box from scratch to see what a game was supposed to be doing. It would have been a good example of what you'd say at an Agile Scrum Standup thing, if that was a thing we did. - Ryan C. Gordon, .plan notes on Serious Sam 1 Linux changelog

With the kind of impact these sorts of projects can have on a developer being very personal, it's often difficult to talk about less desirable effects.

As mentioned earlier, I abandoned my planned regular breaks and let this project eat up my life in a way that in hindsight, I'd prefer to have avoided. It disrupted my sleep and exercise patterns and became my entire lifestyle for a month and a half in a way that will take time for me to bounce back from. That said, I did put some some hard limits in place that I think were really important:

Sleep at some point during every calendar day. Ignoring rest that your body needs can have lasting repercussions - I've been lucky in the past and I'd like to not have to test that luck again.

Take at least a couple of hours off each day to hang out/watch a show with my girlfriend. Becoming consumed by your work can negatively effect relationships - no work is worth sacrificing your loved ones for.

Make room for daily recreation. I've found that without any ability to unwind and think about something else, it can become increasingly difficult to maintain focus and feel good about an intensive piece of work. I slipped up and didn't adhere strictly to this one, but I did make time (20 - 40 minutes) to play through a few games' daily challenges[8] most days.

Working on Day of the Tentacle prevented me from participating in a number of other projects that I had planned to and kept me away from the activities my Patreon supporters pay me to do. Feeling these windows of opportunity slam closed as the port wore on provided some external and internal sources of stress that I was not expecting. For any other project with different constraints and pressures (most of which came from me), I would have preferred to pull back and split my attention.

Some ports you do will never be released. It's not your fault — it just happens sometimes. Sometimes someone else will be asked to finish a port you were working on. Again, not your fault — it just happens. - David Gow

Embracing the outcomes of this prioritisation and accepting that any opportunities sacrificed along the way were my own decision was an important step in not resenting the work I was doing, and there's nothing worse than being at odds with work you're immersed in. I have no valuable advice on how to identify where the limits of that kind of commitment should lie, but it definitely should not be infinite. It's hugely important to feel there's an option to say that a project is a lost cause or otherwise worth bowing out of - it's a sad reality that sometimes things aren't reasonably achievable (something to be thoughtful of when reviewing contract terms).

Don't Do Your Own Testing

Where possible, it is important to make sure that someone else is also doing playtesting. As important as having multiple environments to test on myself and having close familiarity with a project is, I've found that having someone (or multiple someones) who can use a fresh perspective to approach the game in a different way or do things in a different order can reveal things that I wouldn't have spotted on my own, no matter how thorough I was being.

In the same way that having someone else edit your writing (something I am very guilty of not doing enough of!) can be a safety net for identifying typographical errors and unintentionally awkward sentences that an author's familiarity can easily overlook, an extra pair of eyes and attached hemispheres can spot things that might otherwise slip under the radar.

On more popular platforms you just hire a QA team (internally or and external company) and they test your game, but there are very few that do it for Linux. And even when they do, they usually are not very competent with it, and the testers struggle with the most basic concepts, like case sensitivity in file paths or executability being a file attribute, not a result of extension. As a result, a lot of QA time is wasted on false positive reports of "game does not start". And mind you, I'm not saying they're stupid, they're just absolutely unfamiliar with Unices. After all, why should they be? The market does not seem to justify it.

In reality, it seems like the best you can do is to release a public beta with automatic problem reporting and collection.

For Day of the Tentacle, I was able to make use of both Double Fine's internal testers and a broader group of volunteer testers[9] from the community. At the point that I felt comfortable bringing testers in on the project, my own experiences from running the game suggested that it was far closer to ready than testing feedback identified - an impression that I knew I couldn't trust.

By bringing in people more and less familiar with the game than myself (not only from running the Windows or Mac versions, but also from playing the game on its original 1993 release platforms and via ScummVM), we were able to find edge cases and more common issues that I didn't come across and had no real reason to suspect beyond the assumption that I couldn't see everything.

Outside the magical world of FNA it's usually just going "okay, did all of us do a playthrough without something weird happening" and marking it as ready then. The surface area we have at that point is fine, and if we find new things once we expand it, that makes perfect sense and it's not something to be afraid of. - Ethan Lee

The key issues discovered during testing were primarily UX concerns like the mouse cursor not being grabbed in fullscreen (I run everything windowed and would never have noticed this), the gamepad controls screen being missing, and some common keyboard shortcuts (for example, Alt+Tab and Shift+Tab) providing unwanted game input due to Tab being mapped as an action.

The testing team was also super helpful in narrowing down a minimum set of dependencies required on fresh installs of the target distro, and in identifying specific library versions (even upstream commits!) that were responsible for some crashes. I am incredibly thankful to have been supported by a fantastic group of people who were willing to dig in and be thorough.

Before you release, test on every machine you get your hands on. Find
some people to do QA — if they've done Linux QA before, they'll probably
check everything that needs checking for you. - David Gow

On the note of fresh installs, it's worth highlighting an issue I've encountered on several games I've helped out with. Steam's updater and file validation system ignores case changes in file and folder names, and does so without warnings or errors. The file contents themselves will continue to update, but the file name will not change until the game is uninstalled and then reinstalled. Shortly before Firewatch's launch, the main executable was renamed from firewatch.x86_64 to Firewatch.x86_64 while leaving the data folder as firewatch_Data.

This issue wasn't visible to myself or any other testers who already had the game installed, and managed to remain unnoticed until the game launched. Since Unity expects a game's data folder to use the same case as the binary (beyond name, Unity's binaries are not game-specific), simply changing the Steam launch configuration wouldn't resolve the problem. Some users had renamed their data folder and/or binary and were getting by, but this wasn't really something that Linux users could be asked to do before running the game.

After several hours of trying to nut out a nice solution, we eventually gave up and the executable and data folder were renamed to fw.x86_64 and fw_Data, which effectively triggered an entire re-download of the game. Always do a fresh install before release!

Closing Thoughts

And now, just like that, my work with Day of the Tentacle is more or less done.

Post-release porting contracts may sometimes (often?) have clauses for some limited amount of post-port support, but compared to the kind of relationship that game developers typically have with their work, a porter's ends pretty abruptly.

The Day of the Tentacle Remastered splash screen.

Anybody who has worked on a big project will know that there's an adjustment period following the intensity of release. When working in teams, camaraderie and shared experience make this easier to work through. The ability to interact with communities and critics, to speak to and for one's work can be cathartic as well.

Many porters (particularly solo porters) don't have the same avenues for support. Porters generally are not authorised to speak on behalf of their employers, and without much creative input, the sense of personal investment can be diminished (to me, Day of the Tentacle is not, and never will be "my game"). The opportunities for working through "post release blues" are often very different for contractors.

For some projects that I've worked on, doing post-release support has been a nice way to wind down after a release, and doing volunteer outreach and tech support for Linux users that I've come across has been nice in that way. I can fully appreciate though, that this kind of direct community contact is not something that all porters are likely to find enjoyable or have permission to engage in.

I never knew I enjoyed the society of other people so much (especially the creative, intelligent lot that game developers usually are) until I started working from home on ports. You can get quite a lot of online exposure if you want, but it has a huge potential of backfiring, as unhappy customers will definitely use whatever means available to express their dissatisfaction directly at you, with varying levels of aggression.

To each their own, of course – I know a number of people who are exactly otherwise. It might be a good way to learn things about yourself anyway, though, if you're willing to auto-reflect.

Valgrind - A dynamic analysis toolset containing tools for profiling as well as identifying memory and threading bugs

GNU Debugger - A portable debugger compatible with many unices (can also be integrated with many IDEs)

Don't assume that data types will always be what you (or the code) expect. For example, the length of a wchar_t is usually only 2 bytes on Windows

At this point in time, I'm not looking to pursue porting as a career (I have my own games to make!), but I am always willing to assist other developers who are keen to support Linux. The skills and experiences I've gained while working on Day of the Tentacle Remastered will definitely reach far beyond this project.

Credits from the Linux version of The Day of the Tentacle Remastered.More info on what these cool people do outside of testing cool games can be found at the bottom of this article.

This article and the Day of the Tentacle Remastered Linux port could not have been possible without the interest and support of Double Fine and the lovely people who work there. Being asked to work on the port was unexpected and I am humbled to have been invited to work on a game I love.

From the work done by Matt, Oliver and the rest of the DotT Remastered team; through to Craig Derrick's and the Monkey Island Special Editions' teams' efforts; Tim, Dave and the original DotT team's work; and all the way back to Ron, Chip and Aric's work on the original SCUMM engine, the codebase I inherited for my port was well designed and (so much as these things can be) a delight to work with. I am very thankful to have been able to "work with" such talented people on well-made software.

[1] I'd like to extend thanks to the following porters for their words of wisdom, support and inspiration. Helping games exist on Linux is far bigger than any one person, and I'm very glad to have been able to include their more experienced perspectives as supplements and counterpoints to my own.

There are also some additional porters and developers who offered me advice and suggestions who have preferred to remain anonymous. You know who you are, and you're awesome <3

David has also expressed willingness to be contacted by any porters or prospective future porters who'd like to talk. He's a super nice guy and has some great advice to offer.

[2] It is very clear to me that future players who are keen for information (whether that be on a port or a release or an update) don't have malicious intent, but the impact that they can cumulatively have can be negative. Beyond that, porters are typically not involved with (and sometimes not aware of!) decisions surrounding timeframes, and rarely have authority to communicate project status on their contractor/employer's behalf. As always I recommend that eager fans stay patient and respectful - the alternative is for porters and developers to withdraw from communities and become unapproachable, and that doesn't benefit anybody.

[3] Although I wouldn't go so far as to say I needed handholding or having work done for me, knowing that there was someone with technical knowledge that I could query about the game's behaviour gave me huge peace of mind. Oliver's eagerness to hear progress updates/respond to them with encouragement increased my productivity and provided some valuable reassurance when I was feeling neck deep in some obtuse problem.

Oliver was also a guest on Season 2, Episode 2 of SteamLUG Cast in which we talked about Broken Age, Monkey Island, adventure games, Double Fine, crowdfunding and all sorts of other stuff.

[4] The first point at which this issue would noticably manifest itself was during the opening cutscene, where the sound of the bird falling would loop endlessly until the player gained control of Bernard in the mansion lobby. Every time you launch the game, you hope it's fixed, and at some point even hope itself starts to feel sour. Even now, a month after that bug was laid to rest, I can't hear that sound effect without feeling a sense of rising dread.

[9] I'd like to extend a very big thank-you to the testing team for their attention, efforts and enthusiasm. Wonderful people that I'm glad to have worked with. Please check out their respective projects!