Does anyone have a good resource on implementing a shared object pool strategy for a limited resource in vein of Sql connection pooling? (ie would be implemented fully that it is thread safe).

To follow up in regards to @Aaronaught request for clarification the pool usage would be for load balancing requests to an external service. To put it in a scenario that would probably be easier to immediately understand as opposed to my direct situtation. I have a session object that functions similarly to the ISession object from NHibernate. That each unique session manages it's connection to the database. Currently I have 1 long running session object and am encountering issues where my service provider is rate limiting my usage of this individual session.

Due to their lack of expectation that a single session would be treated as a long running service account they apparently treat it as a client that is hammering their service. Which brings me to my question here, instead of having 1 individual session I would create a pool of different sessions and split the requests up to the service across those multiple sessions instead of creating a single focal point as I was previously doing.

Hopefully that background offers some value but to directly answer some of your questions:

Q: Are the objects expensive to create?A: No objects are a pool of limited resources

Q: Will they be acquired/released very frequently?A: Yes, once again they can be thought of NHibernate ISessions where 1 is usually acquired and released for the duration of every single page request.

Q: Will a simple first-come-first-serve suffice or do you need something more intelligent, i.e. that would prevent starvation?A: A simple round robin type distribution would suffice, by starvation I assume you mean if there are no available sessions that callers become blocked waiting for releases. This isn't really applicable since the sessions can be shared by different callers. My goal is distribute the usage across multiple sessions as opposed to 1 single session.

I believe this is probably a divergence from a normal usage of an object pool which is why I originally left this part out and planned just to adapt the pattern to allow sharing of objects as opposed to allowing a starvation situation to ever occur.

Q: What about things like priorities, lazy vs. eager loading, etc.?A: There is no prioritization involved, for simplicity's sake just assume that I would create the pool of available objects at the creation of the pool itself.

Can you tell us a bit about your requirements? Not all pools are created equal. Are the objects expensive to create? Will they be acquired/released very frequently? Will a simple first-come-first-serve suffice or do you need something more intelligent, i.e. that would prevent starvation? What about things like priorities, lazy vs. eager loading, etc.? Anything you can add would help us (or at least me) to come up with a more thorough answer.
–
AaronaughtApr 2 '10 at 2:02

Chris - just looking at your 2nd and 3rd paragraphs, and wondering whether these sessions should really be kept alive indefinitely? It sounds like that's what your service provider doesn't like (long-running sessions), and so you might be looking for a pool implementation that spins up new sessions as necessary and shuts them down when not in use (after some specified period). This can be done, but is a little more complicated, so I'd like to confirm.
–
AaronaughtApr 2 '10 at 20:44

I'm not sure if I need that robust of a solution or not yet as my solution is merely hypothetical. It's possible that my service provider is just lying to me and that their service is over sold and have merely found an excuse for a way to blame the user.
–
Chris MarisicApr 2 '10 at 21:44

6 Answers
6

This question is a little trickier than one might expect due to several unknowns: The behaviour of the resource being pooled, the expected/required lifetime of objects, the real reason that the pool is required, etc. Typically pools are special-purpose - thread pools, connection pools, etc. - because it is easier to optimize one when you know exactly what the resource does and more importantly have control over how that resource is implemented.

Since it's not that simple, what I've tried to do is offer up a fairly flexible approach that you can experiment with and see what works best. Apologies in advance for the long post, but there is a lot of ground to cover when it comes to implementing a decent general-purpose resource pool. and I'm really only scratching the surface.

A general-purpose pool would have to have a few main "settings", including:

Resource loading strategy - eager or lazy;

Resource loading mechanism - how to actually construct one;

Access strategy - you mention "round robin" which is not as straightforward as it sounds; this implementation can use a circular buffer which is similar, but not perfect, because the pool has no control over when resources are actually reclaimed. Other options are FIFO and LIFO; FIFO will have more of a random-access pattern, but LIFO makes it significantly easier to implement a Least-Recently-Used freeing strategy (which you said was out of scope, but it's still worth mentioning).

The concept here is simple - we'll let the public Pool class handle the common issues like thread-safety, but use a different "item store" for each access pattern. LIFO is easily represented by a stack, FIFO is a queue, and I've used a not-very-optimized-but-probably-adequate circular buffer implementation using a List<T> and index pointer to approximate a round-robin access pattern.

All of the classes below are inner classes of the Pool<T> - this was a style choice, but since these really aren't meant to be used outside the Pool, it makes the most sense.

I could have picked a number of different approaches, but the bottom line is that resources should be accessed in the same order that they were created, which means that we have to maintain references to them but mark them as "in use" (or not). In the worst-case scenario, only one slot is ever available, and it takes a full iteration of the buffer for every fetch. This is bad if you have hundreds of resources pooled and are acquiring and releasing them several times per second; not really an issue for a pool of 5-10 items, and in the typical case, where resources are lightly used, it only has to advance one or two slots.

Remember, these classes are private inner classes - that is why they don't need a whole lot of error-checking, the pool itself restricts access to them.

Throw in an enumeration and a factory method and we're done with this part:

The next problem to solve is loading strategy. I've defined three types:

public enum LoadingMode { Eager, Lazy, LazyExpanding };

The first two should be self-explanatory; the third is sort of a hybrid, it lazy-loads resources but doesn't actually start re-using any resources until the pool is full. This would be a good trade-off if you want the pool to be full (which it sounds like you do) but want to defer the expense of actually creating them until first access (i.e. to improve startup times).

The loading methods really aren't too complicated, now that we have the item-store abstraction:

The size and count fields above refer to the maximum size of the pool and the total number of resources owned by the pool (but not necessarily available), respectively. AcquireEager is the simplest, it assumes that an item is already in the store - these items would be preloaded at construction, i.e. in the PreloadItems method shown last.

AcquireLazy checks to see if there are free items in the pool, and if not, it creates a new one. AcquireLazyExpanding will create a new resource as long as the pool hasn't reached its target size yet. I've tried to optimize this to minimize locking, and I hope I haven't made any mistakes (I have tested this under multi-threaded conditions, but obviously not exhaustively).

You might be wondering why none of these methods bother checking to see whether or not the store has reached the maximum size. I'll get to that in a moment.

Now for the pool itself. Here is the full set of private data, some of which has already been shown:

Answering the question I glossed over in the last paragraph - how to ensure we limit the total number of resources created - it turns out that the .NET already has a perfectly good tool for that, it's called Semaphore and it's designed specifically to allow a fixed number of threads access to a resource (in this case the "resource" is the inner item store). Since we're not implementing a full-on producer/consumer queue, this is perfectly adequate for our needs.

As explained earlier, we're using the Semaphore to control concurrency instead of religiously checking the status of the item store. As long as acquired items are correctly released, there's nothing to worry about.

The purpose of that IsDisposed property will become clear in a moment. All the main Dispose method really does is dispose the actual pooled items if they implement IDisposable.

Now you can basically use this as-is, with a try-finally block, but I'm not fond of that syntax, because if you start passing around pooled resources between classes and methods then it's going to get very confusing. It's possible that the main class that uses a resource doesn't even have a reference to the pool. It really becomes quite messy, so a better approach is to create a "smart" pooled object.

This just proxies all of the "real" methods to its inner IFoo (we could do this with a Dynamic Proxy library like Castle, but I won't get into that). It also maintains a reference to the Pool that creates it, so that when we Dispose this object, it automatically releases itself back to the pool. Except when the pool has already been disposed - this means we are in "cleanup" mode and in this case it actually cleans up the internal resource instead.

This is a very good thing to be able to do. It means that the code which uses the IFoo (as opposed to the code which creates it) does not actually need to be aware of the pool. You can even injectIFoo objects using your favourite DI library and the Pool<T> as the provider/factory.

I've put the complete code on PasteBin for your copy-and-pasting enjoyment. There's also a short test program you can use to play around with different loading/access modes and multithreaded conditions, to satisfy yourself that it's thread-safe and not buggy.

One of the most complete, helpful, and interesting answers I've read on SO.
–
Josh SmeatonApr 3 '10 at 23:27

I couldn't agree more with @Josh about this response, especially for the PooledFoo part as releasing the objects always seemed to be handled in a very leaky way and I had imagined that it would make most sense to have it possible to use the using construct like you showed I just hadn't sat down and tried to build that where as your answer gives me all of the information I could need to solve my problem. I think for my specific situation I'll be able to simplify this quite a bit mostly since I can share the instances among threads and don't need to release them back to the pool.
–
Chris MarisicApr 3 '10 at 23:55

However if the simple approach doesn't work first, I have a few ideas in my head of how I could intelligently handle release for my case. I think most specifically I would establish the release to be able to determine that session itself faulted and to dispose of it and replace a new one into the pool. Regardless this post at this point is pretty much the definitive guide on object pooling in C# 3.0, I'm looking forward to see if anyone else has more comments on this.
–
Chris MarisicApr 4 '10 at 0:01

@Chris: If you're talking about WCF client proxies then I have a pattern for that as well, although you need a dependency injector or method interceptor to use it effectively. The DI version uses the kernel with a custom provider to obtain a new-if-faulted version, the method interception version (my preference) just wraps an existing proxy and inserts a fault check before each. I'm not sure how easy it would be to integrate it into a pool like this (haven't really tried, since I just wrote this!) but it would definitely be possible.
–
AaronaughtApr 4 '10 at 0:07

3

Very impressive, although a little over-engineered for most situations. I would expect something like this to be part of a framework.
–
ChaosPandionApr 4 '10 at 0:22

Scratch that earlier comment. I think I just found it strange because this pool doesn't seem to have any thresholds, and maybe it doesn't need to, it would depend on the requirements.
–
AaronaughtApr 2 '10 at 2:04

@Aaronaught - Is it really that odd? I wanted to create a lightweight pool that offers just the functionality needed. It is up to the client to use the class properly.
–
ChaosPandionApr 2 '10 at 2:07

+1 for a very simple solution that can be adapted to my purposes by just changing the backing type to be a List/HashTable etc and changing the counter to roll over. Random question how do you handle management of the pool object itself? Do you just stick it into an IOC container defining it to be singleton there?
–
Chris MarisicApr 2 '10 at 19:05

Should that be static readonly? But I find it odd that you would have the put inside a finally statement, if there's an exception wouldn't it be probable that the object it self is faulted? Would you handle that inside the Put method and left it out for simplicity some type of checking of whether the object is faulted and to create a new instance to be added to the pool instead of inserting the previous?
–
Chris MarisicApr 3 '10 at 19:55

@Chris - I am simply offering a simple tool that I have found useful in the past. The rest is up to you. Modify and use the code as you see fit.
–
ChaosPandionApr 3 '10 at 21:42

I really like Aronaught's implementation -- especially since he handles the waiting on resource to become available through the use of a semaphore. There are several additions I would like to make:

Change sync.WaitOne() to sync.WaitOne(timeout) and expose the timeout as a parameter on Acquire(int timeout) method. This would also necessitate handling the condition when the thread times out waiting on an object to become available.

Add Recycle(T item) method to handle situations when an object needs to be recycled when a failure occurs, for example.

Back in the day Microsoft provided a framework through Microsoft Transaction Server (MTS) and later COM+ to do object pooling for COM objects. That functionality was carried forward to System.EnterpriseServices in the .NET Framework and now in Windows Communication Foundation.

+1 for showing me that the IInstanceProvider interface exists as I will implement this for my solution. I'm always a fan of stacking my code behind a Microsoft provided interface when they provide a fitting definition.
–
Chris MarisicApr 2 '10 at 19:02