It's that time again: time for more programming craziness. Dallas Brown suggested that I talk about how NSAutoreleasePool works behind the scenes. I decided that the best way to do that would be to simply reimplement it, and that is what I'll discuss today.

High Level OverviewThe ball gets rolling when the autorelease message is sent to an object. autorelease is implemented on NSObject, and just calls through to [NSAutoreleasePool addObject: self]. This is a class method, which then needs to track down the right instance to talk to.

NSAutoreleasePool instances are stored in a per-thread stack. When a new pool is created, it gets pushed onto the top of the stack. When a pool is destroyed, it's popped off the stack. When the NSAutoreleasePool class method needs to look up the current pool, it grabs the stack for the current thread and grabs the pool at the top.

Once the right pool is found, the addObject: instance method is used to add the object to the pool. When an object is added to the pool, it's just added to a list of objects kept by the pool.

When a pool is destroyed, it goes through this list of objects and sends release to each one. This is just about all there is to it. There is one additional small complication: if a pool is destroyed which is not at the top of the stack of pools, it also destroys the other pools which sit above it. In short, NSAutoreleasePool instances nest, and if you fail to destroy an inner one, the outer one will take care of it when it gets destroyed.

Garbage CollectionNSAutoreleasePool exists under garbage collection and is even slightly functional. If you use the drain message rather than the release message, destroying a pool under garbage collection signals to the collector that this might be a good time to run a collection cycle. Aside from this, however, NSAutoreleasePool does nothing under garbage collection and isn't very interesting to consider there. For this article I will ignore garbage collection and concentrate on traditional memory management.

10.7 and ARC10.7 got a major internal overhaul of autorelease pools, and Apple is in the middle of introducing a whole new automatic reference counting system. Although the details have changed a great deal, the concepts of how autorelease pools work have not, so everything here is still valid.

InterfaceMy version of an autorelease pool class will be called MAAutoreleasePool. If you'd like to look at the code in its entirety, it's available on GitHub.

This class has class and instance methods for addObject:, as well as a CFMutableArray to hold the list of autoreleased objects. CFMutableArray is used instead of NSMutableArray because the automatic retain and release behavior of NSMutableArray will interfere with what we're trying to do here. There's also the possibility that NSMutableArray would use autorelease internally, which would really mess things up. CFMutableArray can be configured not to do any memory management of its contents, which is what we're after here.

Pool StackAutorelease pools are kept in a stack. Each thread has its own stack of pools, which is used to determine which pool to put an object in when it gets autoreleased.

To encapsulate the management of the stack, I wrote a private method, +_threadPoolStack which returns the current thread's stack of MAAutoreleasePool instances, creating it if necessary. Like each pool's list of contained objects, the pool stack is a CFMutableArray in order to prevent NSMutableArray's automatic memory management from screwing things up.

The simplest way to handle thread-local storage in Cocoa is with the threadDictionary method on NSThread. This method returns an NSMutableDictionary which is unique to the current thread, and automatically destroyed when the thread terminates.

The first thing this method does, then, is fetch that dictionary and declare a unique key to associate with the pool stack:

Finally, with the array either retrieved or newly created, it's returned:

returnarray;}

Now that this is in place, the other methods can be implemented. The +addObject: method essentially just calls the above and then forwards the addObject: to the pool at the top of the stack. I added a bit of paranoia to make sure that the stack isn't empty, and to print an error if it is:

+(void)addObject:(id)object{CFArrayRefstack=[self_threadPoolStack];CFIndexcount=CFArrayGetCount(stack);if(count==0){fprintf(stderr,"Object of class %s autoreleased with no pool, leaking\n",class_getName(object_getClass(object)));}else{MAAutoreleasePool*pool=(id)CFArrayGetValueAtIndex(stack,count-1);[pooladdObject:object];}}

Instance MethodsThe -init method for this class is really simple. Initialize the _objects array, add self to the top of the pool stack, and return:

Next, it removes self from the pool stack. Additionally, it also needs to remove any pools which sit above it in the pool stack. Because this process needs to iterate backwards and needs to be able to modify the stack while iterating, it uses a manual index-based loop:

If pool is self, then the loop is done. All that needs to be done is to remove the entry from the stack and then break out of the loop. All pools in the stack below the current one are left alone:

if(pool==self){CFArrayRemoveValueAtIndex(stack,index);break;}

If it's some other pool, then it needs to be destroyed. This is done by simply sending a release to it. Everything else that needs to happen will be done automatically:

else{[poolrelease];}}

This may be a little hard to understand at first, and it took me a little while to realize how this code needed to be written. When this line is hit, pool is necessarily at the top of the stack. When it's released, it will be deallocated (it's not legal to retain autorelease pools). When it gets deallocated, it will call into its own -dealloc which enters into this same loop again. This loop immediately hits the pool == self condition, removing the pool from the stack and exiting. Thus, all pools on the stack above the one currently being destroyed also get destroyed and removed from the stack.

The loop is now done, and all that's left to do is call through to super:

[superdealloc];}

The class is now complete!

LessonsThere are some good lessons to be learned from this exercise. Most importantly is that NSAutoreleasePool is a pretty straightforward class without much in the way of hidden gotchas. There's nothing complex going on behind the scenes. People new to Cocoa memory management often imagine that autorelease is much more complex than it really is, and ask questions like "How can I tell if an object has been autoreleased?" or "What happens if I autorelease an object twice?" Specifically, we can now see:

There's no way to tell if an object has been autoreleased. The pool is a fairly dumb container with only the barest idea of what it contains. It really just keeps a list for the purposes of sending release to those objects later. This is perfectly fine, because your code should never care whether an object has already been autoreleased.

Objects that are autoreleased twice just get added to the pool twice, and then when the pool is destroyed they get released twice.

Autoreleased objects get released when the current autorelease pool is destroyed. Pools are destroyed when the code that created them explicitly destroys them. If you aren't managing your own pools, then autoreleased objects will survive at least until you return to code you don't own (like Cocoa).

If you autorelease an object on one thread and then pass it to another thread, nothing special happens. The object is still released when the first thread's pool gets destroyed, regardless of what's happening on the new thread. If you need an object to survive the passage, it needs to be retained before sending and then released after receiving. (Fortunately, the cross-thread messaging mechanisms you're likely to use with objects, like GCD/blocks and Cocoa's perform... methods do this for you.)

ConclusionThat wraps up today's exploration of Cocoa internals. Now you know approximately how NSAutoreleasePool gets its job done and how it works. The implementation specifics vary (especially on Lion), but the basic ideas are the same. By knowing how memory management internals work, you can write better and less error-prone code.

Unless you're completely new to this blog, you probably already know that Friday Q&A is driven by reader submissions. On that note, if you have a topic that you'd like to see covered, please send it in!

Did you enjoy this article? I'm selling whole books full of them! Volumes II and III are now out! They're available as ePub, PDF, print, and on iBooks and Kindle. Click here for more information.

Thank you for this great explanation. As a programmer trying to understand Cocoa, this explanation is really all you need to know. It is how NSAutoreleasePool conceptually works.

Of course, in reality, Apple has optimized the hell out of this class. AFAIR, it doesn't really create a new instance of the array. There is one shared CFMutableArrayRef per thread, and it just puts a marker on that when a new autorelease pool starts, and when you release a pool, releases down to that marker.

I think it even plays tricks to avoid creating a new object for every pool, but I'm not sure anymore. After all, when you release an outer pool, all the inner pools get released as well, and it couldn't do that by returning the same singleton per thread from alloc and init each time.

In Lion, autorelease pools are implemented in the ObjC runtime (in objc-arr.mm). A single stack is maintained per thread, and manipulated through “pool tokens”. A pool token is simply a pointer to the top of an autorelease pool. (A NULL entry is kept a a boundary sentinel for verification.)

The @autorelease keyword generates calls to objc_autoreleasePoolPush() and objc_autoreleasePoolPop(), which use pool tokens directly. I presume an NSAutoreleasePool is now a simple wrapper for a pool token.

Another interesting optimization for ARC is runtime elision of autoreleases. An ARC function which semantically returns an autoreleased value can do so using objc_autoreleaseReturnValue(). If the calling function immediately calls objc_retainAutoreleasedReturnValue(id obj) – checked by looking for a specific instruction sequence – the object isn’t autoreleased, but instead stashed in a per-thread global, which causes objc_retainAutoreleasedReturnValue(id obj) to not retain.

Yes, NSThread's threadDictionary works fine on GCD queues. Remember that, from a plain multithreading standpoint, GCD queues aren't special in any way. They're just regular pthreads running regular code. Anything that works on a pthread works on a GCD queue, because you are on a pthread. The only trick is that you need to leave the thread as you found it, because code you don't own will run on it after you're done.

All pthreads are implicitly NSThreads as well. NSThread will create an instance on demand the first time you ask for one on a thread it doesn't know about.

Thanks Mike! I know most of that, but wasn't sure that the threadDictionary would be valid for threads not started by NSThread. This will be super handy to know when I need thread local storage in the future!

This is great! In this implementation, I think it might be more appropriate to release inner pools before releasing the objects in _objects.

It would be legal for an object 'owned' by an inner pool to send a message (or retain via perform... side effects) to a parent object 'owned' by an outer pool (that it might only have a weak reference to).

Jason: That's an interesting point. I think that, strictly speaking, ordering of releases is not guaranteed to have any relation to ordering of autoreleases, so this code is correct. Mostly I would expect people not to rely on the ordering, however across pool boundaries is probably a likely spot for people to do so, whether accidentally or on purpose. If one were doing it for real, it probably would be a good idea to make your change.

Terrence: Hey, at least you'll still know if they like to broaden their horizons and are capable of remembering stuff. Some advanced warning: if you liked to quiz people on how they'd implement reference counting, you may want to get that out of your system soon.

You state that nothing is changed with APC, however can you confirm if this is also true under ios5? We are subclassing NSAutoreleasePool and overriding the functions to use c arrays as the collection class. This has a MASSIVE improvement in performance, and worked well under ios4.2. With ios5 the overridden addObject is no longer being called so our 'enhanced' memory management is now broken. A change in the method signature perhaps?

Conceptually, as far as "autorelease pools are arrays of stuff that get released later", everything is the same. But the details have changed a great deal. In particular, there's not really an NSAutoreleasePool class anymore. The class still exists, but is now a thin wrapper around deeper runtime functionality. That functionality is, I think, directly accessed by the -autorelease method. In short, there's just enough left that you can still use NSAutoreleasePool objects in your code (when not using ARC), but you can't subclass or override things anymore and expect it to work.

I'd suggest ditching the custom class, at least on iOS 5. The new autorelease pool implementation is extremely fast, probably very difficult to improve upon, and I'd wager faster than what you have.

In particular, it uses a single array for the entire thread. Individual pools are just pointers into that array, and when popping a pool, that pointer is just used as a fence to know when to stop. Page-granular allocation and clever recycling means that no copying is needed when expanding the array (which doesn't need to be contiguous). For details, check out AutoreleasePoolPage and related code here: http://opensource.apple.com/source/objc4/objc4-493.9/runtime/objc-arr.mm

Thanks very much for your response Mike. I hadn't been able to find any information on what the change was under the covers and knowing that they're no longer storing everything in a Dictionary makes me very happy!! We overrode the autorelease function on a few of our base class objects to put themselves directly on our autorelease pools and as you said found it to be slower than using IOS5 autorelease pools. Looks like the NSAutoreleasePools are definitely the way to go now.