IDisposable: What Your Mother Never Told You About Resource Deallocation

One difficulty of the IDisposable interface overcome with the Disposable Design Principle.

Introduction - Recommended Reading

A lot of the information regarding .NET internals in this article was gleaned from "CLR via C#", 2nd Ed., by Jeffrey Richter of Wintellect, Microsoft Press. If you don't already own it, buy it. Now. Seriously. It's an essential resource for any C# programmer.

This article is divided into two parts: the first part is an evaluation of many of the problems inherent with IDisposable, and the second part develops some "best practices" when writing IDisposable code.

Deterministic Resource Deallocation - Its Necessity

During more than 20 years of coding, I have occasionally found it necessary to design my own small programming languages for certain tasks. These have varied from imperative scripting languages to a specialized regex for trees. There are many choices to make in language design: a few simple rules should never be broken, and there are many general guidelines. One of these guidelines is:

Never design a language with exceptions that do not also have deterministic resource deallocation.

Guess which guideline the .NET runtime (and, by extension, every .NET language) does not follow? This article will examine some of the consequences of this choice, and then propose some best practices to try to work with this situation.

The reason for the guideline is that deterministic resource deallocation is necessary to produce usable programs with maintainable code. Deterministic resource deallocation provides a certain point at which a coder may know that the resource has been deallocated. There are two approaches to writing reliable software: the traditional approach deallocates resources as soon as possible, while a modern approach lets resource deallocation occur at an indeterminate time. The advantage to the modern approach is that programmers do not have to deallocate resources explicitly. The disadvantage is that it becomes much more difficult to write reliable software; an entire new set of difficult-to-test error conditions is added to the allocation logic. Unfortunately, the .NET runtime was designed around the modern approach.

The .NET runtime's support for indeterministic resource deallocation is through the Finalize method, which has special meaning to the runtime. Microsoft has also recognized the need for deterministic resource deallocation, and has added the IDisposable interface (and other helper classes which we will look at later). However, the runtime views IDisposable as just another interface, without any special meaning; and this reduction in status to second-class citizen causes some difficulties.

In C#, a "poor man's deterministic deallocation" may be implemented through the use of try and finally, or the not-much-cleaner equivalent of using. There was a lot of debate in Microsoft whether to have reference-counted references or not, and I personally believe they made the wrong decision. As a result, deterministic resource deallocation requires the use of either the awkward finally/using, or the error-prone direct call to IDisposable.Dispose. Neither of these are particularly attractive to the programmer of reliable software who is used to, e.g., C++'s shared_ptr<T>.

The IDisposable Solution

IDisposable is Microsoft's solution for allowing deterministic resource deallocation. It is intended to be implemented in the following use cases:

Any type that owns managed (IDisposable) resources. Note that the containing type must own the resources, not just refer to them. One difficulty here is that it is not obvious which classes implement IDisposable, requiring programmers to constantly look up each class in the documentation. FxCop is a help in this situation, and will flag the programmer's code if they forget.

Any type that owns unmanaged resources.

Any type that owns both managed and unmanaged resources.

Any type that derives from an IDisposable class. I don't recommend deriving from types that own unmanaged resources; it is usually a cleaner object-oriented design to have such types as fields rather than base types.

IDisposable does provide for deterministic deallocation; however, it also comes with its own set of problems.

IDisposable's Difficulties - Usability

IDisposable objects are cumbersome to use correctly in C#. Anyone who uses an IDisposable object locally must know to wrap it in a using construct. What makes this awkward is that C# does not allow using to wrap an object that isn't IDisposable. So, for every object being used in a deterministic program, the coder must determine if using is necessary, either from continual documentation lookups, or by wrapping everything in using and removing the ones causing compilation errors.

C++ is slightly better in this regard. They support stack semantics for reference types, which has a logic equivalent to inserting a using only if necessary. C# would benefit from allowing using to wrap non-IDisposable objects.

This problem with IDisposable is an end-user problem: it can be mitigated through code analysis tools and coding conventions, although there is no perfect solution. Adding to this problem is the fact that if resources are disposed at an indeterminate time (i.e., the coder forgot using when it was required), then the code may work fine during testing, and inexplicably fail in the field.

Depending on IDisposable instead of reference-counted references also brings up the problem of ownership. When C++'s reference-counted shared_ptr<T>'s last reference goes out of scope, then its resources are disposed immediately. In contrast, IDisposable-based objects place the burden on the end-user to explicitly define which code "owns" the object, and is therefore responsible for disposing of its resources. Sometimes, ownership is obvious: when an object is owned by another object, the container object also implements IDisposable and is, in turn, disposed by its owner. In another very common case, the lifetime of an object can be defined by its code scope at some point in the program, and the end-user places a using construct to define the block of code that "owns" that object. However, there are a few other cases where the object lifetime may be shared by different code paths, and these are more difficult for the end-user to code correctly (whereas a reference-counted reference would give this problem a simple solution).

IDisposable's Difficulties - Backwards Compatibility

Adding IDisposable to or removing IDisposable from an interface or class is a breaking change. Proper design implies that the client code will primarily use interfaces, so IDisposable could be added to an internal class in that situation, bypassing the interface. However, this can still cause a problem with older client code.

Microsoft ran into this problem themselves. IEnumerator did not derive from IDisposable; however, IEnumerator<T> did. Now, when older client code expecting the IEnumerable collections is given IEnumerable<T> collections instead, their enumerators are not correctly disposed.

Is this the end of the world? Probably not, but it does open up some questions about how the second-class citizen status of IDisposable influences design choices.

IDisposable's Difficulties - Design

The single greatest drawback caused by IDisposable in the area of code design is this: every interface designed must predict whether or not their derived types will need IDisposable. The actual quote is, "Implement the Dispose design pattern on a base type that commonly has derived types that hold onto resources, even if the base type does not" (from Implementing Finalize and Dispose to Clean Up Unmanaged Resources). From a design perspective, interfaces need to predict whether implementations of that interface will need IDisposable.

If an interface does not inherit from IDisposable, yet one of its implementations needs it (e.g., an implementation from a third-party vendor), then the end-user code encounters a "slicing" problem. The end-user code does not realize that the class it is using needs to be disposed. The end-user code may cope with this problem by testing objects it is using for the IDisposable interface explicitly; however, this changes the awkward using into a truly horrible-looking finally construct for each and every abstract local object. Placing this kind of burden on the end user is, in my opinion, unacceptable (at least without language support).

This "slicing" problem is the generalization of the problem Microsoft had when updating IEnumerator; they decided that enumerators often need to free resources, so they added IDisposable to IEnumerator<T>. However, adding IDisposable to a derived type can result in slicing when the end-user code uses the old IEnumerator interface.

For some interfaces, it's rather obvious whether their derived types will need IDisposable. For other interfaces, however, it's not nearly as clear, and yet the decision must be made when the interface is published. In the general case, this is a real problem.

In short, IDisposable prevents designing reusable software. The underlying problem causing all this difficulty is the violation of one of the central principles of Object Oriented Design: keep the interface separate from the implementation. The type of resources allocated and deallocated internally by an implementation of an interface should be an implementation detail. However, when Microsoft made the decision to only support IDisposable as a second-class citizen, they were deciding to view the deallocation of resources as an interface instead of an implementation detail. They were wrong, and these difficulties are the result of violating the principle of separation.

There is one rather unattractive solution: have every interface and every class descend from IDisposable. Since IDisposable may be implemented as a noop, it really only means that the derived interface or class may have an implementation, derived class, or future version that does deallocate resources. I personally have not been brave enough to embrace this as a design guideline — yet.

The final difficulty regarding the IDisposable design is how they interact with collections. Since IDisposable is an interface, either collections must behave differently when they "own" their items, or the end-user must remember to invoke IDisposable.Dispose explicitly, when necessary. If this responsibility is placed on the collection classes, then that would imply a new set of collection classes that "own" their items; and duplicating a class hierarchy is a red flag to any designer, indicating that there is something wrong. If .NET supported reference-counted references (i.e., IDisposable as a first-class citizen), then none of this would be a problem.

IDisposable's Difficulties - Additional Error State

Another difficulty with IDisposable is that it is explicitly invokeable, and not tied to the lifetime of the object. In particular, this adds a new "disposed" state to every disposable object. With the addition of this state, Microsoft recommends that every type implementing IDisposable will check in every method and property accessor to see if it has been disposed, and throw an exception if it has. Ewww... This reminds me of a co-worker I once had who insisted that we run memory checksum algorithms every time we allocate memory, "just in case" the RAM is about to fail. In my opinion, checking for the disposed state is just a waste of cycles, and would only be useful in debug code. If end users can't follow even the most basic software contracts, they won't ever produce working code anyway.

Instead of checking for the disposed state and throwing an exception, I recommend supporting "undefined behaviour". Accessing a disposed object is the modern equivalent of accessing deallocated memory.

IDisposable's Difficulties - No Guarantees

Since IDisposable is just an interface, objects implementing IDisposable only support deterministic deallocation; they cannot require it. Since it is considered perfectly acceptable for the end-user to not dispose of an object (a convention I disagree with), any IDisposable object must support extra logic to handle indeterministic deallocation as well as deterministic deallocation. Once again, indeterministic deallocation is the standard that every .NET object must support, and deterministic deallocation is just an optional addition that cannot be enforced. True enforcement of deterministic resource deallocation would require reference-counted references.

IDisposable's Difficulties - Complexity of Implementation

Microsoft has a code pattern for implementing IDisposable. Not many coders fully understand this code (e.g. why a call to GC.KeepAlive is necessary, and why synchronization of the disposed field is not). There are several other articles that describe in detail how to implement this code pattern. The following are the reasons behind its somewhat obscure design:

It's possible that IDisposable.Dispose may never be called, so the disposable object must include a finalizer that disposes resources. In other words, deterministic deallocation must support indeterministic deallocation.

IDisposable.Dispose may be called multiple times without adverse side-effects, so any real disposal code needs to be protected by a check that it has not already run.

Because finalizers are run on all unreachable objects in an arbitrary order, finalizers may not access managed objects. Therefore, the resource deallocation method must be able to handle both a "normal" dispose (when called from IDisposable.Dispose) and an "unmanaged-only" dispose (when called from Object.Finalize).

Since finalizers run on a separate thread(s), there is the possibility that the finalizer may be called before IDisposable.Dispose returns. Judicial use of GC.KeepAlive and GC.SuppressFinalize may be used to prevent race conditions.

In addition, the following facts are often overlooked:

Finalizers are called if constructors throw an exception (another convention I disagree with); therefore, the disposal code must gracefully handle partially-constructed objects.

Implementing IDisposable on a CriticalFinalizerObject-derived type is tricky because void Dispose(bool disposing) is virtual, yet it must run within a Constrained Execution Region. This may require an explicit call to RuntimeHelpers.PrepareMethod.

The naming convention for the recommended IDisposable code pattern is confusing at best. The object is implementing the interface IDisposable, and needs a boolean field disposed. So far so good, but then: as part of the code pattern, the implementation of IDisposable.Dispose calls the overloaded Dispose with a boolean disposing parameter that indicates what type of disposing can take place (not whether disposing is taking place). The naming conventions for this code pattern are confusing, even for veteran C# programmers, if they aren't constantly reviewing the code pattern. Any deviation is flagged by FxCop as a violation of this code pattern.

This is one area where C++ again has a bit of an edge on C#. The C++ compiler does much of the work of implementing IDisposable due to its destructor syntax. C# is a pure .NET language, so in many ways, has a more natural syntax than C++. However, C++ does have two helpful advantages when it comes to deterministic resource deallocation: destructor syntax, and stack semantics for reference types.

Even for the perfect programmer, the complexity of the recommended IDisposable code pattern increases the possibility of incorrect code written in libraries or by coworkers. It is a natural impulse to place shutdown logic within a Dispose method. However, since we cannot touch managed objects when in finalizers, and since IDisposable only supports deterministic deallocation rather than enforcing it, this is often a mistake. The recommended IDisposable code pattern can only be used to deallocate resources; it cannot be used to support general shutdown logic. This fact is all too often forgotten.

Sometimes, for simplicity or by accident, IDisposable is just forgotten. FxCop catches some of these violations (such as the common case where an object contains an IDisposable object), but it misses other cases. Microsoft coders themselves have fallen into this trap: WeakReference does not implement IDisposable, and it certainly should. Programmers who do not believe in the necessity of deterministic resource deallocation may simply ignore IDisposable altogether.

Shutdown logic is a common need in any real-world application. This is particularly true in asynchronous programming models. For example, a class that has its own child thread would wish to stop that thread by setting a ManualResetEvent. While it is perfectly reasonable and expected to do this when IDisposable.Dispose is called directly, this would be disastrous if called from a finalizer. Since IDisposable does not enforce deterministic deallocation, there is not even a warning to the end-user who forgets or neglects to call IDisposable.Dispose; their program just slowly leaks resources. This raises the question: is there any way for the finalizer code to access managed objects?

In order to better understand the restrictions on finalizers, we must understand the garbage collector. [Note: The description of garbage collection and finalization given here is a simplification; the actual implementation is considerably more complicated. Generations, resurrection, weak references, and several other topics are ignored. For the purpose of this article, however, this logical description is correct and reasonably complete.]

The .NET garbage collector utilizes a mark/sweep algorithm. Specifically, it does the logical equivalent of the following:

Suspends all threads (except the finalizer thread(s)).

Creates a set of "root" objects. If the AppDomain is unloading or the CLR is shutting down, then there are no root objects. For normal garbage collections, the root objects are:

Static fields.

Method parameters and local variables for the whole call stack of each thread, unless the current CLI instruction has already moved past the point of their last access (e.g. if a local variable is only used for the first half of a function, then it is eligible for garbage collection for the second half of the function) - note that the this pointer is included here.

Normal and pinned GCHandle table entries (these are used for Interop code, so the GC doesn't remove objects that are referenced only by unmanaged code).

Recursively marks each root object as "reachable": for each reference field in the reachable object, the object referred to by the field is recursively marked as reachable (if it isn't already).

Identifies the remaining, unmarked objects as "unreachable".

Recursively marks each unreachable object with a finalizer (that hasn't had GC.SuppressFinalize called on it) as reachable, and places them on the "finalization reachable queue" in a mostly-unpredictable order.

In parallel with the garbage collection above, finalization is also constantly running in the background:

A finalizer thread takes an object from the finalization reachable queue, and executes its finalizer - note that multiple finalizer threads may be executing finalizers for different objects at any given time.

The object is then ignored; if it is still reachable from another object on the finalization reachable queue, then it will be kept in memory; otherwise, it will be considered unreachable, and will be collected at the next garbage collection sweep.

The reason that finalizers may not access managed objects is because they do not know what other finalizers have been run. Any object that has fields to other objects may access those fields, since the other objects will still be in memory, but nothing may be done with them, since they may have already had their finalizers run (thus disposing them). Even calling Dispose would be an error, since Dispose may already be running in the context of another finalizer thread. Calling Dispose would be meaningless anyway, since those objects are either reachable from live code, or are already on the finalization reachable queue. Also note that in the case of the AppDomain unloading or the CLR shutting down, all objects become eligible for garbage collection, including CLR runtime support objects and static reference fields; not even static methods such as EventLog.WriteEntry may be called in this situation.

There are a handful of exceptions where finalizers may access managed objects:

The finalization reachable queue is partially ordered: finalizers for CriticalFinalizerObject-derived types are called after finalizers for non-CriticalFinalizerObject-derived types. This means that, e.g. a class with a child thread may call ManualResetEvent.Set for a contained ManualResetEvent, as long as the class does not derive from CriticalFinalizerObject.

The Console object and some methods on the Thread object are given special consideration. This explains why example programs can create an object calling Console.WriteLine in its finalizer and then exit, but the same program won't work with EventLog.WriteEntry.

Generally speaking, finalizers may not access managed objects. However, support for shutdown logic is necessary for reasonably-complex software. The Windows.Forms namespace handles this with Application.Exit, which initiates an orderly shutdown. When designing library components, it is helpful to have a way of supporting shutdown logic integrated with the existing logically-similar IDisposable (this avoids having to define an IShutdownable interface without any built-in language support). This is usually done by supporting orderly shutdown when IDisposable.Dispose is invoked, and an abortive shutdown when it is not. It would be even better if the finalizer could be used to do an orderly shutdown whenever possible.

Microsoft came up against this problem, too. The StreamWriter class owns a Stream object; StreamWriter.Close will flush its buffers and then call Stream.Close. However, if a StreamWriter was not closed, its finalizer cannot flush its buffers. Microsoft "solved" this problem by not giving StreamWriter a finalizer, hoping that programmers will notice the missing data and deduce their error. This is a perfect example of the need for shutdown logic.

A Brief Hiatus - Where We Are

"But, of course, this is all far too messy. It runs counter to the goals for our new managed platform to force developers to worry about this sort of thing." (Chris Brumme, "Lifetime, GC.KeepAlive, handle recycling", blog post 2003-04-19)

<sarcasm strength="mild">Ahh, yes. .NET sure has made it easy. Why, unmanaged destructors in C++ are so much more complicated than all of this.</sarcasm> Seriously, this article could be much longer if it included the really complex issues such as resurrection and the restrictions on finalizers that descend from CriticalFinalizerObject.

I'd like to take a moment to sing some praises of .NET and C#. Although I do disagree with a couple of Microsoft's decisions, on the whole, they've done an excellent job. I'm a huge fan of any language that brings the procedural and functional families together in a more synergetic union, and C# does an excellent job of that. The .NET Framework and runtime have a few rough corners, but overall, they're better than what has come before, and they're obviously the way things are going. So far, I've been pointing out the problems caused by IDisposable, and from this point forward, I'll start looking at solving a couple of these problems.

The fact is, IDisposable is now built-in to .NET languages (though not the runtime), and any solution needs to make use of this interface. We're stuck with it, so to speak, so let's make the best of it.

One reason for the complexity of Microsoft's recommended IDisposable code pattern is because they try to cover too many use cases. A bit of discipline in the design of IDisposable classes will go a long way:

For each unmanaged resource, create exactly one (possibly internal) IDisposable class that is responsible for freeing it. Microsoft followed this principle thoroughly in the BCL implementation. Note that a wrapper type for an unmanaged resource is considered a managed resource.

Never derive from an unmanaged resource wrapper type.

Create other managed IDisposable types that either own managed resources and/or derive from a type that owns managed resources.

Under no circumstances create a type that has to consider both managed and unmanaged resources, when implementing IDisposable. This greatly simplifies the implementation, reducing possible errors.

Level 1 types are types that derive from Level 1 types and/or contain field members that are Level 0 or Level 1 types.

To expound on this design principle, the small private or internal Level 0 classes that wrap unmanaged resources should be as close to the native API as possible, and should only concern themselves with disposing the resource correctly. All other APIs should be provided in a Level 1 class that has a Level 0 field member. This would result in two loosely-related classes (or class hierarchies): one is only responsible for wrapping the unmanaged resource, and the other only has to refer to a managed resource. This reduces our use cases for IDisposable to only two:

Level 0 types: only deal with unmanaged resources.

Level 1 types: only deal with managed resources (defined by a base type and/or in fields).

Implementing IDisposable on Level 1 types is rather simple: just implement IDisposable.Dispose as calling Dispose on any IDisposable field, and then, if this type is derived from an IDisposable type, call base.Dispose. This is not the place for general shutdown logic. Note the following for this simple implementation:

Dispose is safe to be called multiple times because it is safe to call IDisposable.Dispose multiple times, and that's all it does.

Level 1 type should not have finalizers; they wouldn't be able to do anything anyway, since managed code cannot be accessed.

It is not necessary to call GC.KeepAlive(this) at the end of Dispose. Even though it is possible for the garbage collector to collect this object while Dispose is still running, this is not dangerous since all the resources being disposed are managed, and neither this type nor any derived types have finalizers.

Calling GC.SuppressFinalize(this) is likewise unnecessary because neither this type nor any derived types have finalizers.

However, IDisposable is still difficult to implement correctly for the first use case. Due to the complexities of properly implementing IDisposable for unmanaged resources, it's actually best if we don't implement it altogether. This can be accomplished through the diligent use of base types that handle the common logic, or through the use of helper classes that often remove the need for IDisposable.

It is very common to have to write an unmanaged resource wrapper class for an unmanaged resource that is a pointer to some data structure. For this common use case, a higher-level abstraction is available through Microsoft-provided helper classes. System.Runtime.InteropServices.SafeHandle, System.Runtime.InteropServices.CriticalHandle, and the classes in Microsoft.Win32.SafeHandles allow writing very simple unmanaged resource wrappers if the unmanaged resource may be treated as an IntPtr. However, these are not supported on the .NET Compact Framework; on that platform, I recommend writing your own version of these extremely useful classes.

Level 0 types, in the Disposable Design Principle, should always derive from SafeHandle, if it is available on the target platform. SafeHandle and its derived classes have special P/Invoke support, which helps prevent leaking resources in some rare situations. Interop code should define function parameters, and return types as SafeHandle (or derived types) rather than IntPtr. The CriticalHandle class, in spite of the name, is actually less safe to use than SafeHandle, and should generally be avoided.

The relationship between SafeWaitHandle and WaitHandle is a perfect example of the Disposable Design Principle: SafeWaitHandle is the Level 0 class, and WaitHandle is the Level 1 class that provides the normal end-user API. SafeWaitHandle is in the SafeHandle hierarchy, implementing SafeHandle.ReleaseHandle as a call to the Win32 CloseHandle function; it only concerns itself with how to free the resource. The Level 1 WaitHandle class, in contrast, is not in the SafeHandle hierarchy; and its hierarchy exposes a full API for waitable handles, such as WaitOne.

This means there are four possibilities when having to write a new unmanaged resource wrapper (in the order of ease of implementation):

There is already a Level 0 type for the unmanaged resource. In other words, the unmanaged resource is a pointer type that is already covered by a class derived from SafeHandle. Microsoft has supplied several classes already, including SafeFileHandle, SafePipeHandle, and SafeWaitHandle, among others. In this case, the programmer only needs to create a new Level 1 type.

The unmanaged resource is a pointer type, but doesn't have a suitable Level 0 type already defined. In this case, the programmer needs to create two classes, one Level 0 and one Level 1.

The unmanaged resource that needs wrapping is a simple pointer type along with some additional information (such as a secondary pointer or integral "context" value). In this case, the programmer must also create two classes, but the implementation details of the Level 0 type are more complex.

The unmanaged resource is not a pointer type at all. In this case, the programmer must create two classes, and the implementation details of both are much more complex.

Note that when creating hierarchies of Level 1 types, it is common practice to declare a protected property in the (possibly abstract) base Level 1 type, and this field should have the type and name of the related Level 0 type. For example, the Level 1 abstract base type WaitHandle establishes the Level 1 hierarchy for waitable handles, and it has a protected property named SafeWaitHandle of type SafeWaitHandle.

To define a new Level 1 type that uses a Level 0 type, extend an existing Level 1 hierarchy, if possible.

The example for using existing Level 0 (SafeHandle-derived) types is ManualResetTimer (named to match the existing ManualResetEvent). Of the many timers provided by the .NET framework, they did not include a WaitHandle-based timer that gets signalled when the timer goes off. This "Waitable Timer", as it is called by the SDK, is commonly used by asynchronous programs. For simplicity, this sample does not support periodic timers or timers with asynchronous callback functions.

Note that ManualResetTimer derives from WaitHandle (the Level 1 hierarchy) because the Level 0 SafeWaitHandle already correctly disposes of the unmanaged resource. Because of the Level 0/Level 1 class hierarchy division already in place, implementing ManualResetTimer is quite straightforward.

Always use SafeHandle or derived types as parameters and return values for interop functions. For example, this sample code uses SafeWaitHandle instead of IntPtr. This prevents resource leaks if a thread is unexpectedly aborted.

Since a Level 1 hierarchy is already in place, ManualResetTimer doesn't have to deal with disposing, even of its managed resources. This is all handled by the WaitHandle base type.

They should be decorated with a ReliabilityContractAttribute and a PrePrepareMethodAttribute.

Both IsInvalid and ReleaseHandle may be run from a finalizer during system shutdown, so they may not access any managed objects whatsoever.

Since a Level 0 type's ReleaseHandle only P/Invokes its resource cleanup function and returns, the Constrained Execution Region and finalizer restraints are not troublesome in practice. The only awkwardness is in the additional attributes that are necessary.

Once the Level 0 type is completed, then the Level 1 type may be defined:

SafeWindowStationHandle is now a protected property. This should be a set by derived classes, usually in their constructors. Note that this may also be a public property (e.g. Microsoft chose to make WaitHandle.SafeWaitHandlepublic); however, I believe protected is the better choice.

When implementing IDisposable in the base class, I assume the Disposable Design Principle instead of using Microsoft's IDisposable code pattern. As a result:

Types derived from WindowStationBase may not directly own unmanaged resources, i.e., they must be Level 1 types. Note that they may own Level 0 types, which may own unmanaged resources; they just can't be Level 0 types themselves.

There is no need for WindowStationBase (or any derived type) to have a finalizer. Implementing Microsoft's IDisposable code pattern requires a finalizer.

I chose to name the resource disposing function DisposeManagedResources, which is logically equivalent to the Dispose(true) of Microsoft's IDisposable code pattern.

Sometimes an unmanaged API requires additional context information in order to deallocate a resource. This requires a Level 0 type that has some additional information attached to it, and this always requires more complex interop code.

The example for defining advanced Level 0 types is allocating memory in the context of another process. The other process' handle needs to be associated with the allocated memory, and it needs to be passed to the deallocation function. First, the Level 0 type:

This is very similar to the Level 0 type defined earlier; only this class also keeps a SafeHandle reference to the remote process, which must be passed to VirtualFreeEx.

A Level 0 type may contain a reference to another Level 0 type (in this example, SafeRemoteMemoryHandle has a field of type SafeHandle). However, it must explicitly control the field's reference count, which requires an additional boolean field (ReleaseSafeProcessHandle).

The process handle is held as a SafeHandle, not an IntPtr. This is because SafeHandle internally implements reference counting to prevent premature deallocation. This is useful both while being held as a field in SafeRemoteMemoryHandle and being passed to VirtualFreeEx.

Since SafeProcessHandle may be accessed during CERs, its accessors need the ReliabilityContract and PrePrepareMethod attributes.

There is also an additional method, SafeRemoteMemoryHandle.SetHandle, which is designed to execute within a Constrained Execution Region, so it can atomically set both the remote process handle and the unmanaged handle together.

Once again, proper enumerations are skipped for simplicity.

Also, a more proper handling of the remote process handle would require defining a SafeProcessHandle, and using that in place of the SafeHandle in this sample. This sample has completely correct behavior, but does not provide full type safety.

The first thing that should stand out is how much more complicated the allocation function is. NativeMethods.VirtualAllocEx is designed to partially run within an explicit Constrained Execution Region. Specifically:

It does all the necessary allocations before the CER. In this example, it only needs to allocate the returned SafeRemoteMemoryHandle object.

The call to RuntimeHelpers.PrepareConstrainedRegions followed by the empty try block is the way of declaring the finally block to be an explicit Constrained Execution Region. See MSDN for more details on this method.

It performs error checking, including throwing exceptions (which may allocate memory) after the CER.

The CER provides atomic execution: It guarantees that the IntPtr returned from the unmanaged VirtualAllocEx is wrapped in a SafeRemoteMemoryHandle object, even in the presence of asynchronous exceptions (e.g., if Thread.Abort is called on a thread in a CER, the CLR will wait until the CER is completed before asynchronously raising the ThreadAbortException).

CERs were not necessary for the simpler examples because SafeHandle is treated specially when returned from an unmanaged function: the returned value (actually an IntPtr) is used to construct a new SafeHandle atomically. In other words, the CLR supports this behavior for SafeHandle automatically, but now we have to force the same behavior using CERs.

Another important note is that the interop code should continue to reference the Level 0 type (e.g., SafeRemoteMemoryHandle) instead of just an IntPtr; this keeps SafeHandle's reference counting involved. Passing the context data (e.g., SafeHandle or SafeProcessHandle) along with a plain IntPtr would be incorrect.

The RemoteMemory Level 1 type does expose the additional context property (as RemoteMemory.SafeProcessHandle). This is not required, but often useful.

Notes on how this example is simplified:

For simplicity, this example only provides a single Level 1 class instead of a class hierarchy. See the previous example for an example of the Level 1 hierarchy pattern.

Again, the process SafeHandle should really be a SafeProcessHandle, and proper enumerations have been omitted.

This sample also does not expose a very user-friendly API; it should include both reading and writing at various offsets, and should accept byte arrays instead of pre-pinned memory.

Exceptions of type Exception should not be thrown directly; this should be of a more specific type.

There are a handful of unmanaged APIs whose handle types are not pointers. Each of these handle types may either be converted to an IntPtr (if they are smaller or equal to the IntPtr type) or treated as additional context data for a fake IntPtr.

The example for non-pointer Level 0 types is the local atom table. There is no real reason to use this antiquated API in a modern program, but this example will illustrate how to handle APIs of this nature. The ATOM type is an unsigned 16-bit integer, and for illustration purposes, the sample is implemented twice: once widening the ushort to IntPtr, and the other treating the ushort as context data for a fake IntPtr.

The only difference of note is the addition of the Handle property, which provides access to the handle, treating it as a ushort. Note the necessity of the ReliabilityContract and PrePrepareMethod attributes on the property accessors. The IsInvalid and ReleaseHandle implementations use Handle instead of handle for ease of implementation.

The additional complexity comes into play with the interop code used with the Level 1 class:

The primary difference between this example and the last one is the need for CERs in every single interop call. The automatic reference counting from SafeHandle is no longer automatic, so it must be done by hand. Every time the underlying unmanaged handle needs to be passed to an unmanaged function, the example of NativeMethods.GetAtomName should be followed:

Use a CER to atomically increment the SafeHandle reference count, call the unmanaged function, and decrement the SafeHandle count. Note that incrementing the SafeHandle reference count may fail, which should abort the call. [Alternatively, the incrementing and unmanaged function call may be placed within the try block, but the decrementing must remain in the finally block.]

Perform all error testing: both the SafeHandle increment as well as the unmanaged function result must be considered. Remember that throwing Exception is not recommended in production code; a more specific type should be selected instead.

The second implementation (using context values instead of casting to/from IntPtr) may be chosen if the casting would be awkward, or if the unmanaged handle type won't fit into a single IntPtr field. It is possible to make the SafeHandle.handle field almost meaningless by only assigning it 0 (for invalid handle values) or -1 (indicating the handle - including the context values - is valid):

The IsInvalid property tests the handle, which now only has values of 0 or -1, but the ReleaseHandle method still uses Handle for convenience.

The Handle setter has been replaced by the SetHandle method. This is done in the example to reflect the fact that most of the time contexts are used, SetHandle will need to take more than one argument.

The only change necessary in the rest of the example is the change in how the handles are set in the NativeMethods.AddAtom constructor method:

ret.Handle = atom;

should be:

ret.SetHandle(atom);

Remember that in a real-world situation, SetHandle would be taking more than one argument.

Summary

To summarize, prefer using the Disposable Design Principle. The DDP splits up resource management responsibilities into Level 0 types (which handle unmanaged resources), and Level 1 types (which are still small wrapper classes that closely resemble the native API, but only handle managed resources):

Level 0 types directly wrap unmanaged resources, and are only concerned with deallocation of their resource.

Level 0 types are either abstract or sealed.

Level 0 types must be designed to execute completely within an atomic execution region.

For Constrained Execution Regions, this means that Level 0 types must derive from SafeHandle (which derives from CriticalFinalizerObject).

For finally blocks, this means that Level 0 types must derive from a separately-defined SafeHandle type which implements IDisposable to deallocate the unmanaged resource explicitly (possibly called in the context of a finally block) or from a finalizer.

Constructors for Level 0 types must be called from within an atomic execution region.

The special full framework interop handling of SafeHandle return values is considered unmanaged code (and therefore an atomic execution region of the strongest guarantee).

Level 0 types may refer to other Level 0 types, but must increment the count of the referred-to object as long as the reference is needed.

Level 1 types only deal with managed resources.

Level 1 types are generally sealed unless they are defining a base Level 1 type for a Level 1 hierarchy.

Level 1 types derive from Level 1 types or from IDisposable directly; they do not derive from CriticalFinalizerObject or Level 0 types.

Level 1 types may have fields that are Level 0 or Level 1 types.

Level 1 types implement IDisposable.Dispose by calling Dispose on each of its Level 0 and Level 1 fields, and then calling base.Dispose if applicable.

Level 1 types do not have finalizers.

When defining a Level 1 type hierarchy, the abstract root base type should define a protected property with the name and type of the associated Level 0 type.

Using the Disposable Design Principle (instead of Microsoft's IDisposable code pattern) will make software more reliable and easier to use.

Afterword

In a future article, I hope to address one further drawback to IDisposable: the lack of support for shutdown logic; and provide a (partial) solution. This was originally intended to be part of this article, but it's already too long. I also hope to look at the SafeHandle alternatives for the .NET Compact Framework, which sadly does not support SafeHandle or Constrained Execution Regions.

I'd like to thank my loving almost-wife Mandy Snell, for patiently proofreading this article. On October 4th, 2008, she will officially become Mandy Cleary. I also must state that everything good in my life comes from Jesus Christ; He is the source of all wisdom, and I thank Him for all His gifts. "For God giveth to a man that is good in his sight wisdom, and knowledge, and joy" (Ecc. 2:26).

History

2008-09-27 - Fixed bug in the advanced sample, rewrote the summary of the DDP, and added the reference to Microsoft's rationale to not support reference counting

2008-09-22 - Added the References and History sections

2008-09-21 - Initial publication

License

This article, along with any associated source code and files, is licensed under The BSD License

I have to say, I really like this approach. I've done it in a number of my objects as well; it enables more complex possibilities like managed shutdown logic as long as GC.KeepAlive is called at the end of Dispose. It's also more efficient, since the finalizer can be removed completely in release builds.

Given the complexities of the IDisposable <-> Finalizer situation, it may very well be that in a few years, everyone will migrate to requiring explicit disposing. I sure wouldn't mind! It would be nicer if we had something like stack reference semantics in C#, though, so the explicit disposing could be handled auto-magically by the compiler.

Last I checked, SQL Server dropped CFM, but I'm not a regular user of the full product.

Of course, anyone can write their own fiber-based CLR host. Though to be honest, I'm not convinced of any advantages of doing so. The fiber API was never meant to serve as a base for a thread implementation... more suited to co-routines and the like. At least that's how I always felt.

The reasons for choosing to use CFM are largely because when it migrated from SDI to conventional MFC based MDI, there was a very large base of existing core code, as well as tens of thousands of third party and customer applications and scripting that would have broken otherwise.

It seems to me that there are two common uses for iDispose; one of them you've touched on, but the other has to do with managed objects that can become useless and yet still exist, most notably those supporting events.

For example, if I have an object X that wants to receive an event when something happens to object Y, then object Y will contain a delegate that points to object X. Even if the effect of the event is to update fields in object X which can only be relevant if there exist references to X other than the event delegate (implying that if all such references go out of scope, Y's delegate pointing to X should be erased and X should be garbage-collected), no automatic disposal will take place in such instance.

Adding a .dispose method to object X allows for unsubscription in such a scenario. To be sure, I think other methods of handling that situation would be better if .net supported them (e.g. delegates, which could be invoked by a function which would return true if the target was alive and false if the target had been garbage-collected) but .dispose certainly seems like the most sensible approach given the way .net is actually set up.

If you're using IDisposable in this situation, you'll have to make sure it doesn't unsubscribe when called from your finalizer. This is a special case of the finalizer dependency problem; I may propose my partial solution for this problem in a future article.

Take a look at WeakReference[^] if you haven't already considered it. It has a small bug (it doesn't implement IDisposable), but if you keep that in mind it should be usable.

Is there any requirement that an idisposable object even have a finalizer? I can think of plenty of situations where it may be helpful to have an object function logically in a 'Using' clause even when it does not use any unmanaged resources. For example, if I want to count how many times a particular piece of code causes an object's "Paint" event to be fired, I could create an object whose constructor would subscribe to the event and whose dispose method would unsubscribe. Then I could code something like:

Using PM as New PaintMonitor(theControl)
... do stuff
somevariable = PM.count
End Using

In such a scenario, the PaintMonitor's dispose routine would be essential, but there would be no need for a finalizer. Should the normal Dispose pattern be used, including the useless finalizer, should be used except for the finalizer, which may be omitted, or what?

This is an example of shutdown logic (e.g., when an object is destroyed, code needs to run other than just freeing resources). I'll be tackling this in more detail in my next two articles, which won't be done for 2-3 weeks.

I believe the proper approach for your situation is to implement IDisposable without a Finalizer (note: event unsubscription cannot be done in a Finalizer), and make it clear in the documentation that Dispose must be called. Also evaluate the worst-case scenarios: what if the user doesn't wrap it in a Using? In your case, the best solution might be to have a Finalizer that only checks to see if Dispose was called and calls Debug.Assert if it wasn't.

This type of object (requiring Dispose and not implementing a Finalizer) is what I'm calling a Level 3 type in my future articles, and they have some non-obvious pitfalls regarding construction. Specifically, allocation within a using statement, e.g.:
Using PM as New PaintMonitor(theControl) ...
is only valid on Windows Forms, Services, and Console applications that "play nice". By "play nice", I mean that Thread.Abort is never called on an individual thread, and thread abortion is never cancelled (unless it's done only to return threads to an AppDomain host). Allocation within a using statement is definitely not valid in ASP.NET or SQL Server-hosted code. I'm talking in the general case, here; the simple example of event unsubscription should be OK even in ASP.NET/SQLServer.

...is only valid on Windows Forms, Services, and Console applications that "play nice".

In something like an event-subscription model, where object disposal is used to unsubscribe events, an occasional failure to dispose an object may create a memory leak which will persist for as long as the object which produces the events, but that would be the extent of the ill effect. So a somewhat rude shutdown wouldn't be the end of the world, though a pattern of such shutdowns could cause trouble.

Would there be any practical way of having an object keep a copy of the stack trace that was in effect when it was created, and then supply that to a debugger in the finalizer? I know that stack traces aren't terribly fast to generate, but in debug mode I would think it might be more useful to be informed "The BozFoo that was created in line 45 of module FooBar called from line 94 of ZebraQuack went out of scope without proper disposal" would be more helpful than "A BozFoo was allocated once upon a time and never disposed properly". Unfortunately, I don't know of any practical way to ensure that the stack trace wouldn't be garbage-collected before the BozFoo finalizer had a chance to run. Perhaps each allocation could be assigned a Long index number and create a log entry? Since the Long would be a value type, it wouldn't be subject to premature garbage collection. Have you seen anyone use such an approach?

In something like an event-subscription model, where object disposal is used to unsubscribe events, an occasional failure to dispose an object may create a memory leak which will persist for as long as the object which produces the events, but that would be the extent of the ill effect. So a somewhat rude shutdown wouldn't be the end of the world, though a pattern of such shutdowns could cause trouble.

True. If one assumes that event-producing objects are long-lived (compared to event subscription objects), then it would cause problems. I'm assuming that the event subscription object would keep a reference to its delegate - and therefore to all objects in that call stack.

supercat9 wrote:

I don't know of any practical way to ensure that the stack trace wouldn't be garbage-collected before the BozFoo finalizer had a chance to run. Perhaps each allocation could be assigned a Long index number and create a log entry? Since the Long would be a value type, it wouldn't be subject to premature garbage collection. Have you seen anyone use such an approach?

I'm not familiar with anyone doing this specifically, though it sounds reasonable. There *is* a way to force object lifetimes to extend past the finalizers of other objects, but it doesn't work if the AppDomain is being unloaded or the CLR is shutting down. Also, it's tricky to get right, and places a lot more strain on the garbage collector. I'm considering addressing this approach in my next article, but I will probably end up just mentioning it as a mere curiosity. If you're interested, the basic idea is to have the containing object (in this case, the event subscription object) keep a normal GCHandle reference to its contained object (the stack trace) - note that the event subscription object then *must* have a finalizer that will do nothing if the AppDomain is shutting down or the CLR is exiting, otherwise safely logs the stack trace and frees the GCHandle.

I really can't recommend this approach, though, due to its complexity and incompleteness (in an abortive shutdown, you won't get the stack trace anyway).

Unmanaged resources at least need some kind of flag, to prevent releasing the resource multiple times if Dispose is called more than once.

I believe your "resusable" comment is referring to every property and method checking if the object has been disposed, and throwing ObjectDisposedException if it has. Again, this is part of Microsoft's recommended IDisposable code pattern - it's not required (and in the article, I recommend that it not be done this way).