.NET Generics and Code Bloat (or its lack thereof)

.NET Generics and Code Bloat (or its lack thereof)

Introduction

I recently got questions from a couple of customers on the implications of using generic types (generics) in .NET on code bloat (also known as code explosion).

This is a legitimate concern and let me explain why.

Among the most popular programming languages, C++ was one of the first to provide generic programming through templates. And the way templates work in C++ is that, for each concrete type for which the template is instantiated, the code is actually duplicated: the templated arguments in the source of the templated class are replaced for every concrete type instantiation, and then compiled.

The obvious consequence is that the memory footprint for the templated type increases linearly with the number of instantiations of the templated type.

Another consequence is that the source code of the templated classes must be available to the compilation unit which instantiates them with concrete types. Usually this means defining all the templated types in header files only.

To be fair on this, it should be mentioned that there are techniques to reduce the code bloat with C++ templates. For example, common, non-templated functionality could be factored out in methods of a non-templated base class. Depending on the nature of the templated class (that is, how much of its functionality actually requires templated arguments and how much does not), this technique may reduce code bloat to some extent.

So you may wonder: what’s the story with .NET Generics? Well, read on….

.NET Generics: types, IL and JITted code

Generic types were introduced in .NET Framework version 2.0. They differ from templates in C++ in that their code is compiled to Intermediate Language (IL) code as such, that is as generic code (IL has specific opcodes for operations on generic types). At runtime, different instantiations generate different types (that is, different System.Type objects). And the generic type itself also has its own type.

Note that at runtime a generic type’s name terminates with the “`” character and the generic’s arity, that is the number of its generic parameters.

What we said so far concerns the type system and the IL code, but what about the native code generated by the CLR JIT Compiler? As usual when we do not know, let’s use some low-level means to go and check ourselves. As you may have guessed by now, it will be the WinDbg debugger with the sos.dll debugger extension. But before doing that, a bit of background on some inner data structures of the CLR is in order.

One word of caution at this point: we will be inspecting some implementation details of generic types in .NET. While this can provide valuable information to developers who are interested in developing efficient code, these implementation details are not guaranteed to stay the same in all future versions of the framework. The analysis below is relative to version 2.0 of the CLR, that is version 2.x and 3.x of the .NET Framework.

MethodTable and MethodDesc

At runtime each .NET object points to its type information, which is divided into 2 parts: the MethodTable and the EEClass. The MethodTable, in turn, points to method descriptors, which describe methods for a type. Thankfully, we do not need to deal with the details of the memory layout of these data structures, because we can use some commands in the sos.dll debugger extension to dump them out. Let’s double-check this with an example:

The isJitted field indicates whether the JIT Compiler already compiled the method. If so, the CodeAddr field contains the address of the JIT-ted code in memory.

Armed with these new tools, we are ready to investigate how the CLR generates native code for generic types.

Inspecting MethodTable and MethodDesc for instantiated generic types

The idea is very simple: we can look at method tables and method descriptors of instantiated generics at runtime and figure out to what extent the CLR can reuse code (thus avoiding code bloat) and to what extent it has to duplicate the code for different instantiated generic types.

Let’s start with this example: we define a generic class GenericClass<T>, and we instantiate it with 2 different reference types, A and B:

We first searched for objects of type GenericClass<>. As expected, we found 2 (gca and gcb in the program above). They are of different types, GenericClass<A> and GenericClass<B>.

But when we dump out the method table we find out that the method descriptor for generic methods (constructor and TestEqual) are the same. This also implies that the JIT-ted code must be the same. Also note, in passing, the System.__Canon argument shown above. We’ll come to it later.

Can we infer a rule from this? Well, not yet. Let’s run the same program with an additional type, a value type this time

What makes A and C different from each other, while A and B are not? Well, here we have one value type and one reference type. If we try with different value types (say GenericClass<C> and GenericClass<D>, where D is also a struct), we’ll find out that C and D are also different from each other (debugger output omitted here).

So a plausible inference from these tests is that the CLR produces one code for reference types, which is reused for all reference type instantiations, and different code for each value type instantiation. The reason is that reference types are all 4-byte (in 32-bit processes) or 8-byte (in 64-bit processes) values. So the layout of data types, and conequently of the code that accesses them, does not change. On the other hand, the layout of each value type is potentially unique. This is, indeed, the way the CLR works when JIT-ting code for generics. The blog post main purpose was not only, or may be even not so much, to establish this rule, but to show a technique to find this out

We mentioned the System.__Canon type shown as the type of generic functions. System.__Canon is an internal type which is used to make the canonical instantiation of a generic type. For code reuse purposes, the canonical instantiation provides the code for all reference-type instantiations.

Generics with Multiple Parameter Types

So far we have considered generic types with one parameter type only (arity == 1), but it is fairly common to have many. The .NET Framework itself have some, for example Dictionary<TKey, TValue>.

The rule can be easily extended to the case of arity > 1: if all the parameter types are reference types, the code is shared. Otherwise, it is not.

NGEN and Generics

A legitimate question at this point may be: what happens with pre-JITTed modules?

The short answer is that all the instantiations that are known at NGEN time, as well as the canonical instantiation, are compiled into the native image. Other instantiations are JIT-compiled at runtime.

Let’s show this once again with an experiment: with reference to the code above, let’s place A, B and C in one assembly, along with the Main() method which instantiates the generic type (be it MyGenericsTest.dll). When the assembly is ngen-ed, instantiation of GenericClass<A>, GenericClass<B> and GenericClass<C> are detected. Therefore, in addition to the canonical instantiation (which covers GenericClass<A> and GenericClass<B>), the methods for GenericClass<C> are also compiled into the native image MyGenericsTest.ni.dll.

The commands above show that the code for the TestEqual method of both instantiatiations MyGenericClass<A> and MyGenericClass<C> is in the native image MyGenericTest.ni.dll.

Now let’s move the generic instantiation out of MyGenericsTest assembly, directly in Program.Main() in ConsoleApplication.exe. In this case, when MyGenericsTest.dll is NGENed, no instantiations are detected, so only the methods of the canonical instantiation are compiled into the native image. As a consequence, the code for the MyGenericsTest<C> instantiation is not in the native image and has to be JIT-compiled at runtime:

Summary

The main purpose of this post was to show how you can experiment yourself with some features of the CLR and find out, with the debugger, some of its inner workings.

By applying these techniques to generics, we have seen that:

The CLR goes a great deal in the direction of minimizing code bloat with generic programming. In particular, the CLR generates one implementation only for all reference types

When CLR code is pre-JITted through NGEN, all known instantiations are pre-JITTed to native code, in addition to the canonical instantiation that applies to all reference types. In other cases (value type instantiation unknown at NGEN time), the code is JIT-ted at runtime.