HG4 and Object Lifetime Management Models

As the title implies, I have been thinking quite a bit more about the object lifetime management models and how they will be used in Hieroglyph 4. In the past (i.e. Hieroglyph 3), I more or less used heap allocated objects as a default, and only generally didn't use stack allocation all that much. The only real exception to this was when working within a very confined scope, such as within a small method or something along those lines. This had the unintentional side effect that I generally used pointers (or smart pointers) for many of the various objects floating around in my rendering engine.

A small background story may be in line here - I am a self taught C++ developer, and also a self taught graphics programmer. In general, I am a voracious reader and so when I was starting out in development, I picked up many of the habits that were demonstrated in various programming books - most of which were graphics programming related. This was fine at the time, and I certainly learned a lot over the years, but I never really took too much time to dive into C++ or some of the nuances of its deterministic memory model.

As I mentioned a couple posts ago, I have really been digging in deep into C++11/14, and trying to gain a deeper insight into why and how certain paradigms are considered good practice, while others are horribly bad practice. The primary problem with all of the pointers that I described above is that I was unwittingly defaulting to using reference semantics everywhere... As far as correctness goes, that isn't really such a big deal - you can of course write correct programs that use only reference semantics (C# and Java use reference semantics almost exclusively). However, C++ gives you a bit more freedom to choose how and when your objects are treated as references or values, so it is worthwhile to really consider how your classes will be used before you write them.

The obvious choice in reference vs. value semantics is determined when you declare the variables for your objects. If you use pointers or (to a lesser extent) references, then you are clearly choosing reference semantics. However, your actual class design itself also plays a big role in defining the semantics of how it gets used. All of the copy control methods (copy constructor, copy assignment operator, destructor, and the move constructor/move assignment operator) essentially prescribe how your class instances are moved around in a program. And as an engine author, you have to choose ahead of time how that should look. I think this single design choice has one of the biggest impacts on how you (and your users) work with your software.

Once again returning to the pointer default... This default is what most beginning C++ developers choose, and it works fine. However, once you start expanding to multithreaded programming, always using pointers can begin to complicate things. If you have the chance to use value semantics, making a copy of a value for a second thread becomes trivially easy - but if you are using reference semantics it is a bit more tricky. You can make a copy of your references, but all the references still point to the same instance - so multithreaded programming gets even more complex than it already is.

In addition, on modern processors, memory access is THE main bottleneck when trying to fully utilize a processing core. If you are using reference semantics, you are inherently less cache coherent than if you use value semantics (I know, I know, there are some cases where it is better to use reference semantics, but in general this is the exception and not the rule). Creating an array of value objects is simple - you just make it, and the objects are initialized how you indicate during their instantiation. They are destroyed when they go out of scope. On the other hand, creating an array of reference objects is more complex... Are they valid references? Can I dereference them now? When do they get initialized? When can they be destroyed? Did I already destroy them? Lifetime management is just plain easy with value semantics, and not as easy with references.

So to get started with HG4, I am taking some extra time to consider a general set of guidelines for object management with this new understanding in mind. Sometimes reference semantics are necessary (i.e. COM pointers force certain reference semantics, and Direct3D is full of COM...) so the real key is to figure out when you can use values, and when you should use references. It truly seems like a bit of an artistic process - the good C++ developers are very good at making these types of design choices. I still find it a bit taxing to come to a good solution, but when you get there, it sure does shine through and makes your API much easier to work with. Let's hope that I can get the initial design right, and then grow organically from there.

I actually use handles in my current HG3 implementation for resources (i.e. textures + buffers). These are indeed values in the sense that they are easily copyable, but they actually use reference semantics as well. You can have multiple users of your engine with the same handle value, thus having two references to the same object. There are two things you have to watch out for here:

1. As long as the lifetime of the object is managed by your engine, there will be less issues dealing with handle validity. But if you have a delete/release method that takes a handle as an argument, you are heading for trouble unless you track valid handles somehow internally!

2. In multithreaded scenarios, you still have to be careful if any of the operations you can perform with your handle modify the contents of the referenced object. If they are only for 'const' operations, you are golden, but if not then you can still quickly run into trouble.

Both of these areas are easily overcome with a bit of care, so I hope it works out for you!

Edit: I see you are trying to move away from smart pointers as your "handle"? I'm interested to hear why you are making that decision. There are several arguments I would make in favor of using standard shared_ptr and the occasional weak_ptr over the use of integer id based handles.

Not sure if y2kiah was talking to me or to Jason, but anyway, since I'm in the same situation as Jason, I will write my opinion on the subject.

First of all, I'm trying to go away as much as possible from any ref-counted system; with this philosophy in mind, I've to think very well about who own what and who is responsable for freeing/releasing a resource.

With smart pointer or any similar ref-counted system, it's easy to pass a pointer all around and simple incRefCount(). This is often not a problem, but when you're dealing with an high multithreaded engine, it's sometimes source of subtle bugs that are very hard to spot and reproduce. Also, incrementing a refCount means you've to lock the resource first, increment the counter, unlock the resource and, if you're doing this a lot, it could lower performance.

Secondly, having an handle that "point" to something, means that you're free to reorganize memory without problem. For example, if you've a pointer to a resource, then if you need to move that resource in memory for any reason, you can't because who knows how many ref-counted pointer are around, all pointing to the same memory location? Moving that memory would imply a sort of notification to all the pointers telling that the memory has moved...almost impossible.

On the other side, if you've an handle that "point" to something, you can freely move that something. Nobody will notice because the handle remains the same, so from the outside everybody will still use the same unmodified handle, while internally you are free to move things all around.

It's not so easy as it seems, but it works. It requires a bit of thinking and you've to learn how to think this way, but I think it's worth the effort, expecialy with multithreading becoming so familiar

I agree with the points that Giallanon mentioned, but I am not opposed to shared pointers. In fact, in some cases they are the only good option. But they should not be baked into your API - you should let the user decide if they want to use them or not.

If you strive to use value semantics, you have much less issues with multi threading, and the ownership problem is automatically handled for you. If you watch Sean Parent's talk at Going Native 2013, he touches on some of the negative aspects of shared pointers.

One other thing that might interest you is the bitsquid blog, which discusses some of the benefits (and implementation details) of a handle based system. Sorry I'm not directly linking to these, but I'm writing on a tablet....

Meant for everyone with an opinion who wants to engage in the conversation :-)

To be clear, I am not arguing the merit of heap memory allocation over stack allocation. I agree that stack allocation is generally preferred over dynamic allocation when practical. I'm talking about the method used to share dynamically allocated resources across your engine's subsystems. My contention is that if you're going to use a handle, that a smart pointer already IS the handle that you are looking for.

First of all, I'm trying to go away as much as possible from any ref-counted system

That appears to be the exact opposite direction that the modern c++ language is progressing. You're swimming upstream on this one.

This is often not a problem, but when you're dealing with an high multithreaded engine, it's sometimes source of subtle bugs that are very hard to spot and reproduce. Also, incrementing a refCount means you've to lock the resource first, increment the counter, unlock the resource and, if you're doing this a lot, it could lower performance.

This is kind of a hand-wavy argument since lots of things can cause subtle bugs in multi-threaded code, but I don't see evidence of how ref counted pointers alone can be implicated. If you use a condition variable with a master/slave task queue, you protect the concurrent queue itself, there is no need to mutex every individual memory access contained within a task. If you are paranoid, use a unique_ptr to guarantee only one reference, then use the pointer's "move" semantics to hand it back to the master thread when done, or possibly convert it into a shared_ptr at that point if it is indeed a "shared" resource.

Secondly, having an handle that "point" to something, means that you're free to reorganize memory without problem.

Why are you reorganizing memory? Copying large chunks of memory around should be avoided. Use a memory pool or placement new to allocate it in the right location in the first place. But please, at least let yourself live in a world where a resource stays at one place in memory for its lifetime.

Cache coherency has very little to do with value vs. reference semantics and more about how you lay out your allocations in memory, and your access patterns over that memory later. Align your allocations to the order that you iterate over items in a loop. Copying a handle with value semantics is moot if the resources behind the handle are spread all over memory anyway. A handle is going to be a slower way to get at that memory vs. a pointer for the obvious reason that it will always involve some kind of container lookup by the "owner". If you're careful, it will be an O(1) operation but still relatively slow. This also means that the handle needs some behavior built in to know how to locate / request the resource. So instead of just delivering the actual dependency to the object that uses it, you're delivering this thing that instead knows how to go and find the thing that you really need - a pointless indirection. Then you have to handle the case where that thing is not actually available because the central authority disposed of it early because it had no idea (no ref count) that you still needed it - this adds cruft to your code. It's much "cleaner" to live in a world where you are guaranteed that the resource you need still exists in memory as long as you are holding on to a reference. You are indeed a shared owner of that thing as long as that thing is needed for you to function.

For example, if you've a pointer to a resource, then if you need to move that resource in memory for any reason, you can't because who knows how many ref-counted pointer are around, all pointing to the same memory location?

The ref-counted smart pointer knows... that's the reason for a ref count, so you KNOW exactly how many references are around. In any sane system, it should be trivial to locate those pointers and replace them if you have a situation where some central "authority" can rip the proverbial rug from under your feet. In such a situation, I would use a weak_ptr / shared_ptr pair to share the resource. If the weak_ptr fails to convert, at that point you can go and "ask" for the resource again and grab a new pointer. But honestly, in my 16 years of c++ development, I can't think of any case where I needed to "notify" a bunch of pointers that the memory has moved. That kind of thing would be a red flag to me of poor design.

It's not so easy as it seems, but it works. It requires a bit of thinking and you've to learn how to think this way, but I think it's worth the effort, expecialy with multithreading becoming so familiar

It's a solution to a problem that doesn't need to exist anymore, if you use modern c++ constructs. Ten years ago I was using integer IDs to spread around my resources thinking it was cleaner and easier than using naked pointers (and it was). I used a handle that knew how to go and grab the resource from a "manager/owner" every time I needed a resource... hundreds or thousands of times every loop doing lookups, some slower than others. With multi-threading in the mix, it's all the more reason to abandon the "single owner" concept because access to that owner object's container would have to be mutexed, thus adding extra contention to that container throughout your system.

Once a programmer has "seen the light" of the shared ownership concept, I believe they shed the fear of "losing control" over memory and realize that they still retain full control. You DO still control your object's lifetime. You DO still control the memory layout and you CAN get good cache coherency. Those are not matters of which language construct you use. Stop fighting the concept by shoehorning dated patterns into your code - it will result in you having cleaner, more readable, more self-documenting, more testable, more portable code.

To summarize my argument, the most important trade off you make by adopting an integer ID / handle + owner lookup model is that you have dirtier, more tightly coupled code:

1) you spread around a type-unsafe representation of your resource - this amorphous integer that really gives you no guarantees at all

2) you outright lie about the true nature of your dependencies, at the same time adding the requirement of a framework to make this thing useful

3) your code is loaded with self-rolled cruft- it's less readable by other programmers, less testable, and less reusable. Have you heard the expression "kill your darlings"?

4) an integer ID / handle is a wolf in sheep's clothing. A handle is a pointer, and a pointer is a handle if they both represent a way to get at a resource. I think it was George Carlin that said something like "don't change the word and expect the definition to change with it." A pointer lets you just use the thing that you need. A handle lets you add some tight coupling to an externality, a "global" state from the perspective of the client object, that can slowly look up a resource based on an id and give it to you. If you are not plugged into this "framework," you can't operate because your tricky mechanism for converting arbitrary integers into real useful things is broken. Why not just inject the thing you actually need to use, instead of making all of your objects responsible for going out and looking for the resources they need?

5) If I see a parameter of (const shared_ptr<Texture> &texPtr) rather than (int textureId), I am being more straightforward and clear about the true nature of my dependency. Without a single comment, I can ascertain a ton of self-documented information. For one, I know this function needs A TEXTURE. I don't need to know or worry about where to get an ID for some texture to pass in, and wonder whether the ID is going to be valid or not and what is controlling it, and pass it along hoping that the function I'm calling knows how to use the ID to get the thing it really needs.

I see "shared_ptr" and immediately I know "oh, I am not the only user of this thing, so I will make sure to only hold this reference in scope for as long as I actually need it!".

Also notice that I use "value semantics" to pass this shared_ptr object around. For all intents and purposes it is a value being copied. But I pass a const reference so that I don't incur any additional copy / ref counting overhead unless I actually store a local copy of it.

Well I think my point is made (probably to a fault). I don't expect to change any minds, programmers' brains aren't easily swayed by opinions on forums. I'm just putting it out there.

I agree with the points that Giallanon mentioned, but I am not opposed to shared pointers. In fact, in some cases they are the only good option. But they should not be baked into your API - you should let the user decide if they want to use them or not.

If you strive to use value semantics, you have much less issues with multi threading, and the ownership problem is automatically handled for you. If you watch Sean Parent's talk at Going Native 2013, he touches on some of the negative aspects of shared pointers.

One other thing that might interest you is the bitsquid blog, which discusses some of the benefits (and implementation details) of a handle based system. Sorry I'm not directly linking to these, but I'm writing on a tablet....

I will admit I have not seen those talks, and I will definitely check them out. I try to consume as many credible opinions as possible.

One bit of wisdom (if it can be called that) I have picked in my career is to try and not be swayed too far one way or the other by individual articles, talks, or journal comments. If I based my opinions on certain books from the early 2000's, I would think that using global Singleton managers everywhere is "good design."

I have heard many compelling arguments for ideas that go against the grain. But, I try to aggregate and average the ideas from many sources of influence over a long period of time to recognize real trends in thinking. If you look at the evidence over the last decade or so, managed memory languages have exploded in popularity, most of which use ref counting mechanisms to faciliate garbage collection. And the C++ language itself, steered by giants of the industry, is aligning itself in many ways with those managed language patterns. You can basically completely factor the "delete" keyword out of the c++ language these days.

First of all, I'm not against shared ptr, I still use them.
Here we are talking about a high performance graphics engine, a piece of software that is highly optimized in order to achieve the best performances. It's my opinion that, in order to reach this goal, you've to program you software as close as possible to the wire, for example by avoiding inheritance, by using custom memory allocators and so on and, obviously, by using multithreading in a smart way.
So, just to be very clear, I'm not talking of general programming, I'm specifically talking about game engines.

First of all, I'm trying to go away as much as possible from any ref-counted system

That appears to be the exact opposite direction that the modern c++ language is progressing. You're swimming upstream on this one.

I find myself going back from c++ to a more c-like approach. I find c is way closer to how the machine works, and let me express better and clearly what I want to. As an example, I prefer to work with char* or const char* instead of a std::string. I find that a char* is simple, elegant and can fulfill many roles, while a std::string is somewhat limited.
There are cases where std::string is the best choice, but in my engine, most of the function takes const char* as input, instead of a std::string&.
Also, it seems to me that many programmers involved in game engine programming are doing this kind of switch. On gamedev forums I've read many post going this way. Anyway, it could just be a matter or preference.

This is kind of a hand-wavy argument since lots of things can cause subtle bugs in multi-threaded code, but I don't see evidence of how ref counted pointers alone can be implicated. If you use a condition variable with a master/slave task queue, you protect the concurrent queue itself, there is no need to mutex every individual memory access contained within a task. If you are paranoid, use a unique_ptr to guarantee only one reference, then use the pointer's "move" semantics to hand it back to the master thread when done, or possibly convert it into a shared_ptr at that point if it is indeed a "shared" resource.

You DO need to mutex every incRefCount() or release().
If your shared resource is processed concurrently by more than a thread, this imply that to increment/decrement a counter, you need to mutex it, otherwise many thread could access the same counter at the same time reading incorrect values.
Also there's a problem arising if you release() from thread1, just before thread2 call incRefCount(). Thread1 is releasing, so the resource ref count goes to 0 before thread2 is able to incRefCount(). Thread2 will find himself with a shared resource that just was released before he could even use it, resulting in an invalid pointer.
Obviously there are ways around it, for example you should incRefCount() before passing the resource to the threads, but this is just an example of invalid pointers going around, even if you think you have a bullet-proof mechanism.

For example, if you've a pointer to a resource, then if you need to move that resource in memory for any reason, you can't because who knows how many ref-counted pointer are around, all pointing to the same memory location?

The ref-counted smart pointer knows... that's the reason for a ref count, so you KNOW exactly how many references are around. In any sane system, it should be trivial to locate those pointers and replace them...

Yes, you know HOW MANY pointers are sharing the resource, but you don't have a list of all pointers pointing to it. I'd like you to show me how could be trivial to find all classes that hold a shared pointer to a specific resource.
If I have a texture shared between 10 models, which in turn are shared between several rendering queue, are you saying that it's trivial to find all 10 models given the texture? I don't think so.

...if you have a situation where some central "authority" can rip the proverbial rug from under your feet. In such a situation, I would use a weak_ptr / shared_ptr pair to share the resource. If the weak_ptr fails to convert, at that point you can go and "ask" for the resource again and grab a new pointer. But honestly, in my 16 years of c++ development, I can't think of any case where I needed to "notify" a bunch of pointers that the memory has moved. That kind of thing would be a red flag to me of poor design.

Say you have an array of models to render:

3DModel modelList[10];
int nModel = 8;

We knows that modelList can hold no more than 10 models, and by now it holds 8 models, from modelList[0] to modelList[7].
So, if I want to render them:

for (int i=0; i<nModel; i++) modelList[i].Render()

You also have some shared_ptr<3DModel*> around in your code; you are using this pointers for example to change model position whenever you need.
Ok, let's say now that model number 3 has to be deleted, because the game no longer needs it, and we want to free resources.
At this point, modelList[2] has to be "killed". The usual way of doing this is to copy modelList[7] into modelList[2] and decrementing nModel. This way, we still have a contiguous array of models to render, and also we know that we have to render 7 models instead of 8.
What about your pointers pointing to &modelList[2] ?
They still point there, but now in modelList[2] we have modelList[7] so your pointer is not pointing to what he should point.
And the pointer pointing to modelList[7] is now invalid, because modelList[7] no longer exists.
In this situation, a subsystem moved some memory in order to achieve better performances and an handle will treat it in a transparent way, while a pointer won't

I will try to answer on all your other points, but now it's time to go sleep for me, will do it tomorrow

You have some pretty compelling points, but I think you are both oversimplifying the question. There are use cases for both value and reference semantics, and you should not rule out either solution. They are simply different tools for different jobs.

With that said, I do take exception to some of your points above:

1. The type safety of an integer handle is dependent on the situation. Some/most handles have type info included in the handle, so for better or worse, it is type safe if the system implements it properly.

2. If the int handle is the only way to reference an object, and you don't actually directly manipulate the object that is represented by the object, then there is no lying about dependencies. The object implementation is then encapsulated into the subsystem, which is a desirable feature.

3. This one is perhaps true, although if you encapsulate your systems properly then testing is no more easy or complex with shared pointers.

4. The ability to control access to an object through a handle abstraction puts access control at the subsystem level. If you pass out pointers, you can't stop someone from using that pointer. That may be important in some contexts, such as multithreaded scenarios.

5. I agree with your point here - the intent is much clearer in your example. However, a shared pointer (whether passed by value or reference) is using reference semantics - multiple copies of the shared pointer refer to the same resource, so it is a reference. As I mentioned in my first reply above, handles are also references.

I enjoy the discussion, so please keep it going! I don't blow in the wind about the programming choices I make, but I also don't wait for the industry to decide on an idiom before I will use it. You need to evaluate things in your own context - that is why developers exist!

You need to pass a TextureHandle to setTexture(), you can't pass a plain int

2) you outright lie about the true nature of your dependencies, at the same time adding the requirement of a framework to make this thing useful

I don't understand this statement, can you elaborate a bit more?

3) your code is loaded with self-rolled cruft- it's less readable by other programmers, less testable, and less reusable. Have you heard the expression "kill your darlings"?

Why less readable? I prefer to type (and see) TextureHandle instead of shared_ptr<gfx::Texture*>
Also the word "handle" let me understand that I'm dealing with "an int", so I don't need to take care of it when memcpying, it's just a plain int.
Less testable? I would say the opposite. If you have a pointer, you can never be sure that what the pointer is pointing to is a valid object. As in my example above, if you have a pointer to ModelList[7] and modelList[7] is swapped with modelList[2], your pointer points to an invalid object.
On the other side, a subsystem can always tell you if an handle is valid or not. If the subsystems "kills" an handle, whoever tries to use that handle will be notified that the handle is no more valid.
Less reusable? I don't see the point here. We are supposed to build a framework, not a general purpose library to share with the world

4) an integer ID / handle is a wolf in sheep's clothing. A handle is a pointer, and a pointer is a handle if they both represent a way to get at a resource. I think it was George Carlin that said something like "don't change the word and expect the definition to change with it." A pointer lets you just use the thing that you need. A handle lets you add some tight coupling to an externality, a "global" state from the perspective of the client object, that can slowly look up a resource based on an id and give it to you. If you are not plugged into this "framework," you can't operate because your tricky mechanism for converting arbitrary integers into real useful things is broken. Why not just inject the thing you actually need to use, instead of making all of your objects responsible for going out and looking for the resources they need?

This is not true. A pointer is a pointer to a specific memory location and once you've assigned to it, you can never change. An handle is a "index" to be used with a subsystem. You can query the subsystem with the handle to get a real pointer to a resource if you really need to (but that should be avoided).
The handle can never change, like the pointer, but the subsystem can spot if an handle is valid or not and notify you of this whenever you try to use it.
The handle does not point to anything. The handle is memcpyable, while a shared pointer is not.
To me, they look very differents.

5) If I see a parameter of (const shared_ptr<Texture> &texPtr) rather than (int textureId), I am being more straightforward and clear about the true nature of my dependency. Without a single comment, I can ascertain a ton of self-documented information. For one, I know this function needs A TEXTURE. I don't need to know or worry about where to get an ID for some texture to pass in, and wonder whether the ID is going to be valid or not and what is controlling it, and pass it along hoping that the function I'm calling knows how to use the ID to get the thing it really needs.

If I see const TextureHandle id, I know that this functions needs a texture. I don't see any difference in a fn taking a const shared_prt<Texture> and a fn taking a const TextureHandle .
You can assume the same meaning, in both case.
As a bonus, the fn can understand if the handle is a valid one, while can't understand if the pointer is (still) pointing to a valid object

5 bis) I see "shared_ptr" and immediately I know "oh, I am not the only user of this thing, so I will make sure to only hold this reference in scope for as long as I actually need it!".
Also notice that I use "value semantics" to pass this shared_ptr object around. For all intents and purposes it is a value being copied. But I pass a const reference so that I don't incur any additional copy / ref counting overhead unless I actually store a local copy of it.

You're using the value semantics with a shared ptr and this could lead to problems in a multithreading environment.

You're assuming that the object that the pointer is pointing to, will remains the same for the whole duration of the function. This is not the case. Other threads may modify the objects while the function is running or even worst, the memory pointed by the pointer could become invalid as in my example above.

Say you call this function to render a model, then you're assuming the the pointer will point to a immutable 3d model; maybe there are threads that are performing operations on the same model, changing position, texture or whatever.

If you don't protect the resource with a lock, you can have a fail. If you protect the resource with a lock, you've a performance hit because you're doing this every frame for every model.

With the "handle way of doing things", you have to work on local copy of the resource.

So the main idea is the following (rough example, in real life I acquire a LocalModel only once, and keep using it until something told me to update it with a new local copy)