I was just watching the "Going Native 2012" streams and I noticed the discussion about std::shared_ptr. I was a bit surprised to hear Bjarne's somewhat negative view on std::shared_ptr and his comment that it should be used as a "last resort" when an object's life-time is uncertain (which I believe, according to him, should infrequently be the case).

Would anyone care to explain this in a bit more depth? How can we program without std::shared_ptr and still manage object life-times in a safe way?

This question came from our site for professional and enthusiast programmers.

8

Not using pointers? Having a distinct owner of the object, that manages the lifetime?
–
Bo PerssonFeb 4 '12 at 14:52

2

what about explicitly shared data? It is hard to not use pointers. Also std::shared_pointer would do the dirty "manage lifetime" in that case
–
Kamil KlimekFeb 4 '12 at 14:55

6

Have you considered listening less to the advice presented and more to the argument behind that advice? He explains pretty well the kind of system in which this sort of advice would work.
–
Nicol BolasFeb 4 '12 at 15:04

@NicolBolas: I listened to the advice and the argument but obviously I didn't feel I understood it well enough.
–
ronagFeb 4 '12 at 15:05

At what time does he say "last resort"? Watching the bit at 36 minutes in (channel9.msdn.com/Events/GoingNative/GoingNative-2012/…) he says that he's wary of the use of pointers, but he means pointers in general, not just shared_ptr and unique_ptr but even 'regular' pointer. He implies that objects themselves (and not pointers to objects allocated with new) should be preferred. Was the bit you were thinking of later in the presentation?
–
PharapApr 17 '14 at 8:58

9 Answers
9

If you can avoid shared ownership then your application will be simpler and easier to understand and hence less susceptible to bugs introduced during maintenance. Complex or unclear ownership models tend to lead to difficult to follow couplings of different parts of the application through shared state that may not be easily trackable.

Given this, it is preferable to use objects with automatic storage duration and to have "value" sub-objects. Failing this, unique_ptr may be a good alternative with shared_ptr being - if not a last resort - some way down the list of desirable tools.

+1 for poiting out that the issue is not the techno itself (shared ownership), but the difficulties it introduces for us mere humans who then have to decipher what's going on.
–
Matthieu M.Feb 4 '12 at 15:39

However, taking such an approach will severely limit a programmer's ability to apply concurrency programming patterns on most non-trivial OOP classes (due to non-copyability.) This issue is raised in the "Going Native 2013".
–
rwongJul 12 '13 at 16:57

The world that Bjarne lives in is very... academic, for want of a better term. If your code can be designed and structured such that objects have very deliberate relational hierarchies, such that ownership relationships are rigid and unyielding, code flows in one direction (from high-level to low-level), and objects only talk to those lower in the hierarchy, then you won't find much need for shared_ptr. It's something you use on those rare occasions where someone has to break the rules. But otherwise, you can just stick everything in vectors or other data structures that uses value semantics, and unique_ptrs for things you have to allocate singly.

While that's a great world to live in, it's not what you get to do all of the time. If you cannot organize your code in that way, because the design of the system you're trying to make means that it is impossible (or just deeply unpleasant), then you're going to find yourself needing shared ownership of objects more and more.

In such a system, holding naked pointers is... not dangerous exactly, but it does raise questions. The great thing about shared_ptr is that it provides reasonable syntactic guarantees about the lifetime of the object. Can it be broken? Of course. But people can also const_cast things; basic care and feeding of shared_ptr should provide reasonable quality of life for allocated objects who's ownership must be shared.

Then, there are weak_ptrs, which cannot be used in the absence of a shared_ptr. If your system is rigidly structured, then you can store a naked pointer to some object, safe in the knowledge that the structure of the application ensures that the object pointed to will outlive you. You can call a function that returns a pointer to some internal or external value (find object named X, for example). In properly structured code, that function would only be available to you if the object's lifetime were guaranteed to exceed your own; thus, storing that naked pointer in your object is fine.

Since that rigidity is not always possible to achieve in real systems, you need some way to reasonably ensure the lifetime. Sometimes, you don't need full ownership; sometimes, you just need to be able to know when the pointer is bad or good. That's where weak_ptr comes in. There have been cases where I could have used a unique_ptr or boost::scoped_ptr, but I had to use a shared_ptr because I specifically needed to give someone a "volatile" pointer. A pointer who's lifetime was indeterminate, and they could query when that pointer was destroyed.

A safe way to survive when the state of the world is indeterminate.

Could that have been done by some function call to get the pointer, instead of via weak_ptr? Yes, but that could more easily be broken. A function who returns a naked pointer has no way of syntactically suggesting that the user not do something like store that pointer long-term. Returning a shared_ptr also makes it way too easy for someone to simply store it and potentially prolong the life-span of an object. Returning a weak_ptr however strongly suggests that storing the shared_ptr you get from lock is a... dubious idea. It won't stop you from doing it, but nothing in C++ stops you from breaking code. weak_ptr provides some minimal resistance from doing the natural thing.

Now, that's not to say that shared_ptr can't be overused; it certainly can. Especially pre-unique_ptr, there were many cases where I just used a boost::shared_ptr because I needed to pass a RAII pointer around or put it into a list. Without move semantics and unique_ptr, boost::shared_ptr was the only real solution.

And you can use it in places where it is quite unnecessary. As stated above, proper code structure can eliminate the need for some uses of shared_ptr. But if your system cannot be structured as such and still do what it needs to, shared_ptr will be of significant use.

+1: Look e.g. at boost::asio. I think the idea extends to many areas, you may not know at compile time which UI widget or asynchronous call is the last to relinquish an object, and with shared_ptr you don't need to know. It obviously doesn't apply to every situation, just another (very useful) tool in the tool-box.
–
Guy SirtonFeb 4 '12 at 19:57

2

A bit late comment; shared_ptr is great for systems where c++ integrated with scripting language such as python. Using boost::python, reference counting on the c++ and python side cooperates greatly; any object from c++ can be still held in python and it won't die.
–
eudoxosJun 16 '12 at 9:13

1

+1 for talking in the context of a "real world"
–
YZ.learnerSep 20 '13 at 19:22

Most of the time, an object is associated with some collection, to which it belongs for its entire lifetime. In which case you can just use whatever_collection<o_type> or whatever_collection<std::unique_ptr<o_type>>, that collection being a member of an object or an automatic variable. Of course, if you didn't need a dynamic number of objects, you could just use an automatic array of fixed-size.

Neither iteration through the collection or any other operation on the object requires a helper function to share ownership... it uses the object, then returns, and the caller guarantees that the object stays alive for the entire call. This is by far the most used contract between caller and callee.

Nicol Bolas commented that "If some object holds onto a naked pointer and that object dies... oops." and "Objects need to ensure that the object lives through that object's life. Only shared_ptr can do that."

I don't buy that argument. At least not that shared_ptr solves this problem. What about:

If some hash table holds onto an object and that object's hashcode changes... oops.

If some function is iterating a vector and an element is inserted into that vector... oops.

Like garbage collection, default use of shared_ptr encourages the programmer not to think about the contract between objects, or between function and caller. Thinking about correct preconditions and postconditions is needed, and object lifetime is just a tiny piece of that bigger pie.

Objects don't "die", some piece of code destroys them. And throwing shared_ptr at the problem instead of figuring out the call contract is a false safety.

@ronag: I suspect that you've started using it where a raw pointer would have been better, because "raw pointers are bad". But raw pointers aren't bad. Only making the first, owning pointer to an object a raw pointer is bad, because then you have to manually manage memory, which is non-trivial in the presence of exceptions. But using raw pointers as handles or iterators is just fine.
–
Ben VoigtFeb 4 '12 at 15:09

4

@BenVoigt: Of course, the difficulty with passing around naked pointers is that you don't know the lifetime of objects. If some object holds onto a naked pointer and that object dies... oops. That's exactly the kind of thing shared_ptr and weak_ptr were designed to avoid. Bjarne tries to live in a world were everything has a nice, explicit lifetime, and everything's built around that. And if you can build that world, great. But that's not how it is in the real world. Objects need to ensure that the object lives through that object's life. Only shared_ptr can do that.
–
Nicol BolasFeb 4 '12 at 15:13

5

@NicolBolas: If a function creates an object and returns it by pointer, it should be a unique_ptr, expressing that only one pointer to the object exists and it has ownership.
–
Ben VoigtFeb 4 '12 at 15:31

6

@Nicol: If it's looking up up a pointer in some collection, it probably ought to use whatever pointer type is in that collection, or a raw pointer if the collection holds values. If it is creating an object and the caller wants a shared_ptr, it should still return a unique_ptr. Conversion from unique_ptr to shared_ptr is easy, but the reverse is logically impossible.
–
Ben VoigtFeb 4 '12 at 16:09

4

@BenVoigt: I agree with your first sentence, though I agree with @NicolBolas in "real life". Perhaps its a difference in domain, or unless you can confidently say "reference counting is always harmful", there are often cases you want to do reference counting. (One can get philosophical and impractical discussing this. That's not the point. Suffice to say, large proportional of Firefox and Windows (XPCOM/COM respectively) rely on refcount as a fundamental concept). When you do need it, you're better off using shared_ptr than inventing your own, and that's exactly it's purpose.
–
kizzx2Feb 5 '12 at 6:57

I prefer not thinking in absolute terms (like "last resort") but relative to the problem domain.

C++ can offer a number of different ways to manage lifetime. Some of them try to re-conduce the objects in a stack-driven way. Some other try to escape this limitation.
Some of them are "literal", some other are approximations.

Actually you can:

use pure value semantics. Works for relatively small objects where what is important are "values" and not "identities", where you can assume that two Person having an same name are the same person (better: two representation of a same person). Lifetime is granted by the machine stack, end -essentially- deosn't matter to the program (since a person is it s name, no matter what Person is carrying it)

use stack allocated objects, and related reference or pointers: allows polymorphism, and grants object lifetime. No need of "smart pointers", since you ensure no object can be "pointed" by structures that leave in the stack longer than the object they point to (first create the object, then the structures that refer to it).

use stack managed heap allocated objects: this is what std::vector and all containers do, and wat std::unique_ptr does (you can think to it as a vector with size 1). Again, you admit the object begin to exist (and end their existence) before (after) the data structure they refer to.

The weakness of this mehtods is that object types and quantities cannot vary during the execution of deeper stack level calls in respect to where they are created.
All this techniques "fail" their strength in all the situation where creation and deletion of object are consequence of user activities, so that the runtime-type of the object is not compile-time known and there can be over-structures referring to objects the user is asking to remove from a deeper stack-level function call.
In this cases, you have to either:

introduce some discipline about managing object and related referring structures or ...

go somehow to the dark side of "escape the pure stack based lifetime": object must leave independently of the functions that created them.
And must leave ... until they're needed.

C++ isteslf doesn't have any native mechanism to monitor that event (while(are_they_needed)), hence
you have to approximate with:

use shared ownership: objects life is bound to a "reference counter": works if "ownership" can be hierarchically organized, fails where ownership loops may exist. This is what std::shared_ptr does. And weak_ptr can be used to break the loop. This works the most of the time but fails in large design, where many designer works in different teams and there is no clear reason (something coming from a somewhat requirement) about who musty own what (the typical example are dual liked chains: is the previous owing the next referring the previous or the next owning the previous referring the next? In absebce of a requirement the tho solutions are equivalent, and in large project you risk to mix them up)

Use a garbage collecting heap: You simply don't care bout lifetime. You run the collector time to time and what is unreachabe is considered "not anymore needed" and ... well ... ahem ... destroyed? finalized? frozen?. There are a number of GC collector, but I never find one that is really C++ aware. The most of them free memory, not caring about object destruction.

Use a C++ aware garbage collector, with a proper standard methods interface. Good luck to find it.

Going to the very first solution to the last one, the amount of auxiliary data structure required to manage object lifetime increase, as the time spend to organize it and maintain it.

Garbage collector have cost, shared_ptr have less, unique_ptr even less, and stack managed objects have very few.

Is shared_ptr the "last resort"?.
No, it's not: the last resort are garbage collectors.
shared_ptr is actually the std:: proposed last resort. But may be the righ solution, if you are in the situation I explained.

The one thing mentioned by Herb Sutter in a later session is that every time you copy a shared_ptr<> there's an interlocked increment / decrement that has to happen. On multi-threaded code on a multi-core system, the memory synchronization is not insignificant. Given the choice it's better to use either a stack value, or a unique_ptr<> and pass around references or raw pointers.

The point is, don't just use shared_ptr like it's the silver bullet that will solve all your memory leak problems just because it's in the standard. It's a tempting trap, but it's still important to be aware of resource ownership, and unless that ownership is shared, a shared_ptr<> is not the best option.
–
EclipseFeb 4 '12 at 18:11

To me this is the least important detail. See premature optimization. In most cases this shouldn't drive the decision.
–
Guy SirtonFeb 4 '12 at 20:02

1

@gbjbaanb: yes they're at the cpu level, but on a multi-core system you're invalidating caches and forcing memory barriers.
–
EclipseAug 3 '12 at 2:26

2

In a game project I worked on we found that the performance difference was very significant, to the point where we needed 2 different types of ref-counted pointer, one which was threadsafe, one which was not.
–
KylotanNov 23 '12 at 17:37

I don't remember if last "resort" was the exact word he used, but I believe that the actual meaning of what he said was last "choice": given clear ownership conditions; unique_ptr, weak_ptr, shared_ptr and even naked pointers have their place.

One thing they all agreed upon is that we're (developers, book authors, etc) all in the "learning phase" of C++11 and patterns and styles are being defined.

As an example, Herb explained we should expect new editions of some of the seminal C++ books, such as Effective C++ (Meyers) and C++ Coding Standards (Sutter & Alexandrescu), a couple of years out while the industry's experience and best practices with C++11 pans out.

I think what he's getting at is that it's becoming common for everyone to write shared_ptr whenever they might have written a standard pointer (like a kind of global replacement), and that it's being used as a cop-out instead of actually designing or at least planning for object creation and deletion.

The other thing that people forget (besides the locking /updating/unlocking bottleneck mentioned in the material above), is that shared_ptr alone doesn't solve cycle problems. You can still leak resources with shared_ptr:

Object A, contains a shared pointer to another Object A
Object B creates A a1 and A a2, and assigns the a1.otherA = a2; and a2.otherA = a1;
Now, object B's shared pointers it used to create a1,a2 go out of scope (say at the end of a function). Now you have a leak- no one else refers to a1 and a2, but they refer to each other so their ref counts are always 1, and you've leaked.

That's the simple example, when this occurs in real code it happens usually in complicated ways. There is a solution with weak_ptr, but so many people now just do shared_ptr everywhere and don't even know of the leak problem or even of weak_ptr.

To wrap it up: I think the comments referenced by the OP boil down to this:

No matter what language you're working in (managed, unmanaged, or something in-between with reference counts like shared_ptr), you need to understand and intentionally decide on object creation, lifetimes, and destruction.

edit: even if that means "unknown, I need to use a shared_ptr," you've still thought of it and are doing so intentionally.

I'll answer from my experience with Objective-C, a language where all objects are reference counted and allocated on the heap. Because of having one way to treat objects, things are a lot easier for the programmer. That has allowed for standard rules to be defined that, when adhered, guarantee code robustness and no memory leaks. It also made possible for clever compiler optimizations to emerge like the recent ARC (automatic reference counting).

My point is that shared_ptr should be your first option rather than the last resort. Use reference counting by default and other options only if you are sure of what you are doing. You will be more productive and your code will be more robust.

How can we program without std::shared_ptr and still manage object lifetimes in safe way?

C++ has large number of different ways to do memory, for example:

Use struct A { MyStruct s1,s2; }; instead of shared_ptr in class scope. This is only for advanced programmers because it requires that you understand how dependencies work, and requires ability to control dependencies enough to restrict them to a tree. Order of classes in header file is important aspect of this. It seems this usage is already common with native builtin c++ types, but it's use with programmer-defined classes seems to be less used because of these dependency and order of classes problems. This solution also have problems with sizeof. Programmers see problems in this as a requirement to use forward declarations or unnecessary #includes and thus many programmers will fall back to inferior solution of pointers and later to shared_ptr.

Use MyClass &find_obj(int i); + clone() instead of shared_ptr<MyClass> create_obj(int i);. Many programmers want to create factories for creating new objects. shared_ptr is ideally suited for this kind of usage. The problem is that it already assumes complex memory management solution using heap/free store allocation, instead of simpler stack or object based solution. Good C++ class hierarchy supports all memory management schemes, not just one of them. The reference based solution can work if the returned object is stored inside the containing object, instead of using local function scope variable. Passing ownership from factory to user code should be avoided. Copying the object after using find_obj() is good way to handle it -- normal copy-constructors and normal constructor (of different class) with refrerence parameter or clone() for polymorphic objects can handle it.

Use of references instead of pointers or shared_ptrs. Every c++ class has constructors, and each reference data member needs to be initialized. This usage can avoid many uses of pointers and shared_ptrs. You just need to choose if your memory is inside the object, or outside of it, and choose the struct solution or reference solution based on the decision. Problems with this solution are usually related to avoiding constructor parameters which is common but problematic practise and misunderstanding how interfaces for classes should be designed.

"Passing ownership from factory to user code should be avoided." And what happens when that isn't possible? "Use of references instead of pointers or shared_ptrs." Um, no. Pointers can be reseated. References cannot. This forces construction-time restrictions on what is stored in a class. That's not practical for a lot of things. Your solution seems to be very rigid and inflexible to the needs of a more fluid interface and use pattern.
–
Nicol BolasFeb 4 '12 at 22:17

@Nicol Bolas: Once you follow the rules above, the refs are going to be used for dependencies between objects, and not for data storage like you suggested. The dependencies are more stable than the data, so we never get into the problem you were considering.
–
tp1Feb 4 '12 at 22:28

Here's a very simple example. You have a game entity, which is an object. It needs to refer to another object, which is a target entity it needs to talk to. However, targets can change. Targets can die at various points. And the entity needs to be able to handle these circumstances. Your rigid no-pointers approach can't handle even something as simple as changing targets, let alone the target dying.
–
Nicol BolasFeb 4 '12 at 22:34

@nicol bolas: oh, that is handled differently; the interface of the class supports more than one "entity". Instead of 1:1 mapping between objects and entities, you'll use entityarray. Then entities die very easily by just removing it from the array. There is only small number of entityarrays in the whole game and dependencies between arrays do not change very often :)
–
tp1Feb 4 '12 at 22:40

1

No, unique_ptr is best suited for factories. You can turn a unique_ptr into a shared_ptr, but it's logically impossible to go the other direction.
–
Ben VoigtFeb 5 '12 at 13:16