Mastering C# and Unity3D

Memory Allocation Without the GC

Unity’s garbage collector is super slow and the bane of our programming life. It’s the reason we can’t use foreach, have to make pools of objects, and go to great lengths to avoid boxing. It’s also seemingly mandatory, but that’s not quite true. Today’s article shows you a way that you can skip the GC and still allocate memory!

To recap, C# has reference types and value types. Reference types are class instances and value types are primitives like int and structs. Reference types are created on the heap, reference counted, and garbage collected. Value types are created on the stack unless they’re fields of classes.

So it’s good practice in Unity to avoid using a lot of reference types because they ultimately create garbage that the garbage collector has to collect, which fragments memory and creates frame spikes. So we turn to structs instead of classes because they don’t create garbage when we use them as local variables. Think of Vector3, Quaternion, or Color.

We can use these structs as local variables and pass them as parameters with or without the ref keyword. But those are temporary operations. Sometimes we’d like to keep them longer, so we add a struct field to a class or we create an array or List of structs. Unfortunately, that class or array or List is itself going to become garbage someday. And so it seems we’re painted into a corner and must accept the inevitability of the GC.

Thankfully, we have a way out! There is a way to allocate memory without using the new operator and without using any classes, including arrays. The way to do it is to call System.Runtime.InteropServices.Marshal.AllocHGlobal. You pass it the number of bytes you’d like to allocate and it allocates them on the heap and returns you an IntPtr struct. When you’re done, call Marshal.FreeHGlobal and pass it the IntPtr you got back from AllocHGlobal.

In the meantime, you can do whatever you want with that IntPtr. You’ll probably want to turn on “unsafe” code by putting -unsafe in your Asset’s directory’s gmcs.rsp, smcs.rsp, and mcs.rsp. Then you can cast the IntPtr to a pointer and access the memory however you want. Here’s a little example where we store a struct on the heap:

usingSystem;usingSystem.Runtime.InteropServices;usingUnityEngine;struct MyStruct
{publicintInt;publicboolBool;publiclongLong;}unsafevoid Foo(){// Allocate enough room for one MyStruct instance// Cast the IntPtr to MyStruct* so we can treat the memory as a MyStructvar pStruct =(MyStruct*)Marshal.AllocHGlobal(sizeof(MyStruct));// Store a struct on the heap!*pStruct =new MyStruct {Int=123, Bool=true, Long=456};// Read the struct from the heap
Debug.Log(pStruct->Int+", "+ pStruct->Bool+", "+ pStruct->Long);// Free the heap memory when we're done with it
Marshal.FreeHGlobal((IntPtr)pStruct);}

Other than the Debug.Log, this code doesn’t create any garbage. We can store the MyStruct* long-term just like a reference to a class. Copying a pointer is cheap, too. A MyStruct* is just 4 or 8 bytes regardless of how big MyStruct gets.

Of course we can do whatever else we want with the heap memory we get back from AllocHGlobal. Want to replace arrays? Easy!

usingSystem;usingSystem.Runtime.InteropServices;usingUnityEngine;/// <summary>/// An array stored in the unmanaged heap/// http://JacksonDunstan.com/articles/3740/// </summary>unsafestruct UnmanagedArray
{/// <summary>/// Number of elements in the array/// </summary>publicint Length;/// <summary>/// The size of one element of the array in bytes/// </summary>publicint ElementSize;/// <summary>/// Pointer to the unmanaged heap memory the array is stored in/// </summary>publicvoid* Memory;/// <summary>/// Create the array. Its elements are initially undefined./// </summary>/// <param name="length">Number of elements in the array</param>/// <param name="elementSize">The size of one element of the array in bytes</param>public UnmanagedArray(int length, int elementSize){
Memory =(void*)Marshal.AllocHGlobal(length * elementSize);
Length = length;
ElementSize = elementSize;}/// <summary>/// Get a pointer to an element in the array/// </summary>/// <param name="index">Index of the element to get a pointer to</param>publicvoid*this[int index]{get{return((byte*)Memory)+ ElementSize * index;}}/// <summary>/// Free the unmanaged heap memory where the array is stored, set <see cref="Memory"/> to null,/// and <see cref="Length"/> to zero./// </summary>publicvoid Destroy(){
Marshal.FreeHGlobal((IntPtr)Memory);
Memory =null;
Length =0;}}unsafevoid Foo(){// Create an array of 5 MyStruct instancesvar array =new UnmanagedArray(5, sizeof(MyStruct));// Fill the arrayfor(var i =0; i < array.Length;++i){*((MyStruct*)array[i])=new MyStruct {Int= i, Bool= i%2==0, Long= i*10};}// Read from the arrayfor(var i =0; i < array.Length;++i){var pStruct =(MyStruct*)array[i];
Debug.Log(pStruct->Int+", "+ pStruct->Bool+", "+ pStruct->Long);}// Free the array's memory when we're done with it
array.Destroy();

One downside of this approach is that we need to make sure to call FreeHGlobal or the memory will never be released. The OS will free it all for you when the app exits. One issue crops up when running in the editor because the app is the editor, not your game. So clicking the Play button to stop the game means you might leave 100 MB of memory un-freed. Do that ten times and the editor will be using an extra gig of RAM! You could just reboot the editor, but there’s a cleaner way so you don’t even need to do that.

Instead of calling AllocHGlobal and FreeHGlobal directly, you can insert a middle-man who remembers all of the allocations you’ve done. Then you can tell this middle-man to free them all when your app exits. Again, this is only necessary in the editor so it’s good to use #if and [Conditional] to strip out as much of the middle-man as possible from your game builds.

Now just replace your calls to Marshal.AllocHGlobal and Marshal.FreeHGlobal with calls to UnmanagedMemory.Alloc and UnmanagedMemory.Free. If you want to use the UnmanagedArray struct above, you might want to do this replacement in there as well. The final step is to put this one-liner in any MonoBehaviour.OnApplicationQuit:

UnmanagedMemory.Cleanup();

UnmanagedMemory.Cleanup();

Hopefully this technique will be a good addition to your toolbox as a Unity programmer. It can certainly come in handy when you need to avoid the GC!

So Dispose gets called for them. That’s a handy convenience and the code looks a lot cleaner, but it’s functionally equivalent to the Destroy and Cleanup methods in the article. On the downside, as you point out, it adds garbage for the UnmanagedMemHolder class instance. So you’d never want to use it on types that come and go often throughout the app, like UnmanagedArray.

One change I could see would be for the UnmanagedMemory class to be made non-static and implement a destructor:

~UnmanagedMemory(){// ... cleanup code}

~UnmanagedMemory()
{
// ... cleanup code
}

The destructor would be called when the UnmanagedMemory instance is garbage-collected, which would hopefully only be when the application exits. Therefore, it seems like the usage would be to create one in a MonoBehaviour that’s active in the first scene, store it as a field, never destroy or deactivate that MonoBehaviour, then set the field to null in OnApplicationQuit. You’d then either need to make UnmanagedMemory a singleton or pass instances of it around to anyone who needs to use it. It’s hard to see how either way provides any benefits over the static class. Perhaps without the singleton it’s easier to unit test.

You’re definitely right about C++ RAII. It’s destructors are totally different from the destructors in C#. They run at well-defined times and the classes you put them in don’t create any garbage because C++ isn’t a garbage-collected language. We don’t have anything like that for C# though. We don’t even have access to destructors in structs. So we just remember to call our Dispose functions.

You’re right, Dispose isn’t called automatically. Thanks for clearing that up. Finalize is, though, so it might be possible to free unmanaged memory in an implementation of Finalize.

The thing I like about the idea of tying the lifecycle of unmanaged memory to the lifecycle of a managed object is that it confers some of the advantages of a managed memory approach, such as not having to remember to make an explicit call in order to free memory. Have to write a working implementation, though, which I failed to do ;)

C# destructors are essentially finalizers. You could use them to free the unmanaged memory. Unfortunately that would mean that you need to use a class to get garbage collected, which kind of defeats the point in the case of types like UnmanagedArray.

If you like the C++ RAII approach there are two ways you can get it. First, you could use C++/CLI to produce .NET DLLs. I’m not so sure how well that would work with Unity and IL2CPP, but it might. Also, “pure” (no native) DLL support is deprecated in the C++/CLI compiler so you’ll probably run out of support soon. Second, you could call into native code via P/Invoke (e.g. [DllImport]) and then write your own real, native C++ there. There’s potentially a lot of work to the “glue” layer between C# and C++, but it’s an option if you really want to escape .NET.

How would one go about tracking (I assume) managed allocations? Would this be anytime we use the ‘new’ keyword? Does it even make sense to? Can we track the lifetime of a resource? I apologize if any of these questions are naive or redundant.

Managed allocations (e.g. new MyClass()) are tracked by the garbage collector. It keeps track of how many references there are to an instance of an object. Sometime after there are no more references to an object the garbage collector will reclaim the memory those objects were using. Currently, Unity’s garbage collector is very slow, runs on the main thread, collects all the garbage at once, and causes memory fragmentation. This is the reason why I have written so many articles with strategies to avoid creating any garbage.

On the contrary, AllocHGlobal allocates unmanaged memory. The garbage collector doesn’t know about this memory. It doesn’t keep a reference count for it and it never reclaims it. It’s entirely your responsibility to call FreeHGlobal when you’re done with the memory you asked for with AllocHGlobal. That can be very error prone but also very efficient, so the article is there to inform you of that option.

My follow up question, albeit perhaps naive, is: can we get a count of the GC managed references at will? For example, say simply for printing the count to the console on demand.

A second question, semi-related to this article and topic: do we need to work in unmanaged territory to reap the benefits of SoA (struct of array) design? Or are we guaranteed contiguous (parallel?) blocks of memory when defining several arrays as fields in a c# class? Or is that only a guarantee in a c# struct? Or no guarantee at all?

For reference see: ‘managing data relationships’ by noel llopis. His article (as well as this talk at gdc 2017 by tim ford) has inspired me as of late and all this talk of memory management I feel is related. I am curious because traditionally we only read about DOD from a c++ perspective, not from a c# perspective.

I don’t know of a way to get the number of managed objects, but you can get the total managed memory size with System.GC.GetTotalMemory. The GC class has other useful functionality, such as Collect to force garbage collection (e.g. on a loading screen).

For SoA, you can easily make a class or struct containing arrays. These arrays are managed references to Array objects that contain the pointer to the actual memory (i.e. array of values) as well as other data such as the Length of the array. It’s the C++ equivalent of a struct containing multiple std::shared_ptr<std::vector<T>>, except the shared_ptr is garbage-collected instead of reference counted.

If you want to contain the actual array in your class or struct, you have two options. First, you can use a fixed array of a compile-time known number of primitives (e.g. int) in a struct. That’s pretty restrictive, but it may work in some use cases. Second, you can use unmanaged memory as described in this article to store pointers directly in your class or struct. You can then treat those pointers as arrays as you choose. If you need them to be contiguous, simply allocate (i.e. with AllocHGlobal) a block of memory large enough for all your arrays and then set the pointers to offset into the memory you get back. For example, if you need three arrays of contiguous floats you can do this:

Unmanaged memory is a lot more flexible, but has the downsides of needing to explicitly free it, double-frees, using freed memory, etc.

As for DOD, it’s really only discussed among C/C++/Assembly programmers because they tend to be the ones who care most about squeezing out the last bits of CPU performance. If that’s your concern, C# is a terrible option. It’s just not designed for that purpose. That said, there are certain “escape hatches” you can use to get around a lot of its overhead. This article discusses one of them: use AllocHGlobal/FreeHGlobal to avoid the GC entirely. Structs, “unsafe” code, and pointers are more such “escape hatches”. If you avoid most of the language and .NET library then you can pull off some semblance of DOD in C#, especially with IL2CPP.

with Unity games, often at loading time, while for example deserializing protobuf , the need for memory heap gets really, really high… so internally Unity bump the heap size, but unfortunately never shrink it again, so we get memory allocated for nothing, for ever, and less memory for system memory (textures, audio …)

So .. using your technique, I am wondering if its going to also grow the same heap used by the Managed Memory .. in that case, maybe we could call Marshal.CoTaskMemAlloc() instead of AllocHGlobal ?

“Deserializing protobuf” makes me think you’re talking about the managed heap since Unity doesn’t provide any Protocol Buffers functionality as far as I’m aware. In that case, AllocHGlobal should be fine since it allocates unmanaged memory. I assume it’s a pass-through to a native function like malloc, but it might be implemented by a native memory manager. Given that the doc for AllocCoTaskMem says it’s for COM purposes, I’m not sure how this is implemented on non-Windows platforms like iOS and Android. It may just be a synonym for AllocHGlobal.