Hi all,
I know this has been discussed million times before, but there is so many posts
on the mailing lists on this subject
and all of them different, its kind of hard to find what's authoritative and
whats not. TDPL doesn't really talk much
about this subject either, it just says the delete keyword is deprecated, and
that clear() doesn't free memory, and
that there is GC.free() which explicitly frees memory.
Say we have a class C, which is constructed and allocated using: C* c = new
C(). I understand c is allocated on the GC
heap. What if I do want to explicitly free it, and not wait for the GC to kick
in? Assuming that delete is gone,
1) Do i do clear(c) and then GC.free(c)? What would happen if i skipped clear()?
2) What is D's equivalent of C++ std::memory? If there's none, what are the
implications of using C's malloc and free
in D as opposed to in C, if any?
Thanks,
Petr

2) What is D's equivalent of C++ std::memory? If there's none, what are
the implications of using C's malloc and free
in D as opposed to in C, if any?

There isn't really one - there is core.memory, but that's mostly just a
shim atop the GC.
If you choose to use malloc and free, you yourself are responsible for
cleaning up, and for registering things with the GC.
There is also GC.malloc, which does basically the same. It has some bells
on it, but I know not for sure what they do.
--
Simen

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake
of customizability appears absurd to me. Why not move the implementation
of the delete statement to the standard library (if it's not there
already) and get it to do the same as the fancy new clear() thing?

Semantically speaking, 'delete' is just a function, and not a
particularly common one. This is not the case with most other language
features.
Turning 'delete' into a real function simplifies the language grammar,
frees a keyword, and makes code that uses it more consistent with the
rest of D, all at essentially no cost.
--
Rainer Deyke - rainerd eldwood.com

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake
of customizability appears absurd to me. Why not move the implementation
of the delete statement to the standard library (if it's not there
already) and get it to do the same as the fancy new clear() thing?

Semantically speaking, 'delete' is just a function, and not a
particularly common one. This is not the case with most other language
features.
Turning 'delete' into a real function simplifies the language grammar,
frees a keyword, and makes code that uses it more consistent with the
rest of D, all at essentially no cost.

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake
of customizability appears absurd to me. Why not move the implementation
of the delete statement to the standard library (if it's not there
already) and get it to do the same as the fancy new clear() thing?

Semantically speaking, 'delete' is just a function, and not a
particularly common one. This is not the case with most other language
features.
Turning 'delete' into a real function simplifies the language grammar,
frees a keyword, and makes code that uses it more consistent with the
rest of D, all at essentially no cost.

We could put it in object.di.
All of this stuff has been discussed between (mainly) Walter, Sean, and
myself. Essentially the plan is to slowly deprecate delete and foster
use of clear() as safe resource reclamation.
Andrei

A std lib function is not set in stone, later it can be improved,
modified, etc.

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake
of customizability appears absurd to me. Why not move the implementation
of the delete statement to the standard library (if it's not there
already) and get it to do the same as the fancy new clear() thing?

delete shouldn't have been a keyword in the first place - it's only
justified historically (it was defined before templates existed etc).
Anyway, changing its meaning at this point is bound to confuse C++ comers.
I don't think writing clear(obj) is more taxing that writing delete obj.
Andrei

I agree, deprecating a fundamental operator like delete should not be
taken lightly. A lot of us prefer to manage the memory manually and do
RAII for our own reasons. By all means, add clear(obj) to the
language/standard library, but both methods should always be available.
There is no valid reason to coerce the rest of us into depending on the
garbage collector. If I wanted to program in a language that imposes
that on me, I would be programming in Java.

I agree, deprecating a fundamental operator like delete should not be
taken lightly.

I agree deprecation shouldn't be taken lightly, but delete is not a
fundamental operator - it could and should be defined at most as a
simple function.

A lot of us prefer to manage the memory manually and do
RAII for our own reasons. By all means, add clear(obj) to the
language/standard library, but both methods should always be available.
There is no valid reason to coerce the rest of us into depending on the
garbage collector. If I wanted to program in a language that imposes
that on me, I would be programming in Java.

Agreed. What would be a good name for a function that does what delete does?
Andrei

I agree deprecation shouldn't be taken lightly, but delete is not a
fundamental operator - it could and should be defined at most as a
simple function.

delete is the antipod of new. With this logic, I don't see why new
shouldn't be replaced with a function as well - after all, all it does
is allocation and initialization, as opposed to delete's destruction and
deallocation.

Not quite. New is different because it is a primitive - it can't be
implemented as a function (well it could if it user placement new, but
we're back to square one). In contrast, delete already knows the type of
the object it's destroying and can call the destructor manually so it is
easy to implement as a regular function.
Anyway, I'd be glad to provide a function that calls malloc() and then
the emplace(), and also a function that calls the destructor and then
free(). But manual deletion has no business in the garbage collected
heap. That currently druntime provides it is an accident caused by the
current implementation; most GC's cannot provide efficient manual
reclamation. And they shouldn't anyway.
(There's a longer discussion underneath this concerning what memory
really is and what GCs really are and do.)
Andrei

But manual deletion has no business in the garbage collected
heap. That currently druntime provides it is an accident caused by the
current implementation; most GC's cannot provide efficient manual
reclamation. And they shouldn't anyway.

The structure of memory blocks is like a graph, generally each memory zone can
have inbound references and outbound ones.
In unsafe D if I am "sure" a GC-managed memory zone (like a large array of
doubles or a large associative array from int to double) has no inbound
references, then I want a way to deallocate it quickly.
In SafeD that can't be allowed, because if I am wrong that operation can
generate some broken references.
So in SafeD the delete operation can be replaced by something safer, a way to
tell the GC to deallocate just a specific object, walking the graph and setting
to null all the inbound pointers (this is a slower operation).
A problem is that the GC is conservative, so some of those references can be
something different, and setting them to null can damage the program state. So
it seems even less safe.
Bye,
bearophile

But manual deletion has no business in the garbage collected
heap. That currently druntime provides it is an accident caused by the
current implementation; most GC's cannot provide efficient manual
reclamation. And they shouldn't anyway.

The structure of memory blocks is like a graph, generally each memory zone can
have inbound references and outbound ones.
In unsafe D if I am "sure" a GC-managed memory zone (like a large array of
doubles or a large associative array from int to double) has no inbound
references, then I want a way to deallocate it quickly.
In SafeD that can't be allowed, because if I am wrong that operation can
generate some broken references.
So in SafeD the delete operation can be replaced by something safer, a way to
tell the GC to deallocate just a specific object, walking the graph and setting
to null all the inbound pointers (this is a slower operation).
A problem is that the GC is conservative, so some of those references can be
something different, and setting them to null can damage the program state. So
it seems even less safe.

The problem is that many of today's GC are structured in ways that make
efficient reclamation of individual block extremely inefficient.
Andrei

The problem is that many of today's GC are structured in ways that make
efficient reclamation of individual block extremely inefficient.

If the GC is conservative it can think some reference to a memory block exists,
despite it's a false reference, so it can delay even for a lot of time its
reclamation. A manual delete can be useful to force the release of this memory
block if the programmer knows no references are present. False pointers are
present in large memory blocks, that are the memory blocks that you most want
to free when you don't need them anymore.
Bye,
bearophile

Replacing delete or forbidding it in detriment of a safer alternative
function indeed makes sense when compiling in the SafeD language subset.
However, the delete operation has a well defined role in "unsafe" D.
The most advantageous would be adding clear() as well as a possible
"destroy()" function, but keeping the delete operation legal, and forbid
it in SafeD mode only.

Not quite. New is different because it is a primitive - it can't be
implemented as a function (well it could if it user placement new, but
we're back to square one). In contrast, delete already knows the type
of the object it's destroying and can call the destructor manually so
it is easy to implement as a regular function.

Sorry, not following you here. Why can't new be implemented as a
templated function which takes the type of the object to create as a
template parameter? Isn't it just allocation, copying over T.init, then
calling the constructor with whatever arguments the user passed?

In D it's possible to call the constructor by issuing __ctor(). In C++
there is no way, hence the presence of a language primitive.

But manual deletion has no business in the garbage collected heap.

Isn't that making the assumption that all D programs are garbage-collected?

No. It is not making that assumption.

That currently druntime provides it is an accident caused by
thecurrent implementation; most GC's cannot provide efficient manual
reclamation. And they shouldn't anyway.

Why not? And what if I don't use the GC (just the
allocation/deallocation aspects of it)?

You should use malloc() and free() for manual memory management and the
GC for automatic memory management. Each of these two is ill-suited for
carrying the other's job.

(There's a longer discussion underneath this concerning what memory
really is and what GCs really are and do.)

I understand your points regarding leaving deallocation to happen on a
GC run being more efficient than manually deallocating objects
individually,

I didn't make that point.

but this doesn't cover all use-cases. Also, what if I need
to deallocate a large block of memory right now? I'd be forced to use
the more verbose and less safe "free" functions.

To manually manage a large memory block, you may want to use malloc()
and free(). Again: manual free() should NOT be counted on in a garbage
collector.
Andrei

Not quite. New is different because it is a primitive - it can't be
implemented as a function (well it could if it user placement new, but
we're back to square one). In contrast, delete already knows the type
of the object it's destroying and can call the destructor manually so
it is easy to implement as a regular function.

Sorry, not following you here. Why can't new be implemented as a
templated function which takes the type of the object to create as a
template parameter? Isn't it just allocation, copying over T.init, then
calling the constructor with whatever arguments the user passed?

In D it's possible to call the constructor by issuing __ctor(). In C++
there is no way, hence the presence of a language primitive.

But manual deletion has no business in the garbage collected heap.

Isn't that making the assumption that all D programs are
garbage-collected?

No. It is not making that assumption.

That currently druntime provides it is an accident caused by
thecurrent implementation; most GC's cannot provide efficient manual
reclamation. And they shouldn't anyway.

Why not? And what if I don't use the GC (just the
allocation/deallocation aspects of it)?

You should use malloc() and free() for manual memory management and the
GC for automatic memory management. Each of these two is ill-suited for
carrying the other's job.

(There's a longer discussion underneath this concerning what memory
really is and what GCs really are and do.)

I understand your points regarding leaving deallocation to happen on a
GC run being more efficient than manually deallocating objects
individually,

I didn't make that point.

Apologies. I now understand how you were referring to that point.
To further clarify: on certain GC implementation, the cost of one
individual object deletion is comparable to the cost of a full collection.
Andrei

Not quite. New is different because it is a primitive - it can't be
implemented as a function (well it could if it user placement new, but
we're back to square one). In contrast, delete already knows the type
of the object it's destroying and can call the destructor manually so
it is easy to implement as a regular function.

Sorry, not following you here. Why can't new be implemented as a
templated function which takes the type of the object to create as a
template parameter? Isn't it just allocation, copying over T.init, then
calling the constructor with whatever arguments the user passed?

In D it's possible to call the constructor by issuing __ctor(). In C++
there is no way, hence the presence of a language primitive.

Doesn't that only support my argument that new *can* be implemented as a
function?

It does. I was explaining how come new was a keyword in C++ followed
then by imitation by Java (where it's completely unneeded) and D.

You should use malloc() and free() for manual memory management and
the GC for automatic memory management. Each of these two is
ill-suited for carrying the other's job.

Thanks, that was really what I was missing from the picture.
However, D allows overloading new/delete for custom allocators[1] - this
can be used for a cleaner way to allocate objects in unmanaged memory. I
don't see why this use of the delete keyword should be deprecated.
[1]: http://digitalmars.com/d/2.0/memory.html#newdelete

That's gone too. It was a thoroughly mistaken feature, and it's an
accident that it still compiles and runs.
We need to plug all the holes in the dam(n).
Andrei

free(). But manual deletion has no business in the garbage collected
heap. That currently druntime provides it is an accident caused by the
current implementation; most GC's cannot provide efficient manual
reclamation. And they shouldn't anyway.
(There's a longer discussion underneath this concerning what memory
really is and what GCs really are and do.)
Andrei

Why not simply make 'delete' an operation that states "I assert that
there are no references to this object, and it will no longer be used."
The GC is then free to do whatever it wants with the object, claim its
memory now, or later, etc.. Surely then, 'delete' is never an
unefficient operation?
Is that what clear() does? I'm not sure what exactly clear() does, I
wasn't able to find definite information on it.
--
Bruno Medeiros - Software Engineer

A std lib function is not set in stone, later it can be improved,
modified, etc.

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake
of customizability appears absurd to me. Why not move the implementation
of the delete statement to the standard library (if it's not there
already) and get it to do the same as the fancy new clear() thing?

delete shouldn't have been a keyword in the first place - it's only
justified historically (it was defined before templates existed etc).
Anyway, changing its meaning at this point is bound to confuse C++
comers.
I don't think writing clear(obj) is more taxing that writing delete obj.

I'm sorry, but what if I manage my memory manually and don't use the GC?
Or will that soon be deprecated as well?

No. Only inherently mistaken features such as delete are deprecated.
Andrei

There is also GC.malloc, which does basically the same. It has some bells
on it, but I know not for sure what they do.

GC.malloc will allocate managed memory (it will be unallocated when the GC
will not see any references to it).
To allocate unmanaged memory, use malloc from core.stdc.stdlib.
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

Hi all,
I know this has been discussed million times before, but there is so
many posts on the mailing lists on this subject and all of them
different, its kind of hard to find what's authoritative and whats not.
TDPL doesn't really talk much about this subject either, it just says
the delete keyword is deprecated, and that clear() doesn't free memory,
and that there is GC.free() which explicitly frees memory.
Say we have a class C, which is constructed and allocated using: C* c =
new C(). I understand c is allocated on the GC heap. What if I do want
to explicitly free it, and not wait for the GC to kick in? Assuming that
delete is gone,
1) Do i do clear(c) and then GC.free(c)? What would happen if i skipped
clear()?

Strictly speaking, you have to call GC.free(cast(void*) c), since c is a
reference (C) and not a pointer (C*).
-Lars

Hi all,
I know this has been discussed million times before, but there is so
many posts on the mailing lists on this subject and all of them
different, its kind of hard to find what's authoritative and whats
not.
TDPL doesn't really talk much about this subject either, it just says
the delete keyword is deprecated, and that clear() doesn't free
memory,
and that there is GC.free() which explicitly frees memory.
Say we have a class C, which is constructed and allocated using: C* c
=
new C(). I understand c is allocated on the GC heap. What if I do
want
to explicitly free it, and not wait for the GC to kick in? Assuming
that
delete is gone,
1) Do i do clear(c) and then GC.free(c)? What would happen if i
skipped
clear()?

Strictly speaking, you have to call GC.free(cast(void*) c), since c is
a
reference (C) and not a pointer (C*).

It may change to a template function at some point because there's no
implicit conversion from shared(void)*. I haven't completely decided
whether the user should be expected to cast when freeing shared memory.

That can be a good thing to add to Phobos2 or the core. An overloaded
version with the same name for nonclasses can be added.

Can someone please explain to me how this is different and *better* than
the allegedly now-deprecated delete statement?
Also, what happens if the object instance was allocated on the stack?
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

Ok, so I now know how to explicitly free memory allocated to the GC heap. Now
the question is, should we have the usual C* c = new C(), and I wanted to
allocate it to unmanaged memory(just like it would in C++) and then at some
point call the destructor on it and free the memory, outside of the GC. How
would I do that?
Thanks for all your replies and suggestions,
Petr

Ok, so I now know how to explicitly free memory allocated to the GC
heap. Now the question is, should we have the usual C* c = new C(), and
I wanted to allocate it to unmanaged memory(just like it would in C++)
and then at some
point call the destructor on it and free the memory, outside of the GC.
How would I do that?

Yes, that's something I need to, because I need to allocate memory from
a special pool, the GC doesn't know about. And I need to handle freeing
of such a memory myself because it's persistent and surviving
application start-overs.
--
Robert M. Münch
http://www.robertmuench.de

Ok, so I now know how to explicitly free memory allocated to the GC
heap. Now the question is, should we have the usual C* c = new C(),
and I wanted to allocate it to unmanaged memory(just like it would in
C++) and then at some
point call the destructor on it and free the memory, outside of the
GC. How would I do that?

Yes, that's something I need to, because I need to allocate memory from
a special pool, the GC doesn't know about. And I need to handle freeing
of such a memory myself because it's persistent and surviving
application start-overs.

A std lib function is not set in stone, later it can be improved,
modified, etc.

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake of
customizability appears absurd to me. Why not move the implementation of
the delete statement to the standard library (if it's not there already)
and get it to do the same as the fancy new clear() thing?
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

A std lib function is not set in stone, later it can be improved,
modified, etc.

But the same could be said about any language feature! Deprecating the
delete statement, and increasing the verbosity of the code for the sake
of customizability appears absurd to me. Why not move the implementation
of the delete statement to the standard library (if it's not there
already) and get it to do the same as the fancy new clear() thing?

delete shouldn't have been a keyword in the first place - it's only
justified historically (it was defined before templates existed etc).
Anyway, changing its meaning at this point is bound to confuse C++
comers.
I don't think writing clear(obj) is more taxing that writing delete obj.

I'm sorry, but what if I manage my memory manually and don't use the GC?
Or will that soon be deprecated as well?
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

I agree deprecation shouldn't be taken lightly, but delete is not a
fundamental operator - it could and should be defined at most as a
simple function.

delete is the antipod of new. With this logic, I don't see why new
shouldn't be replaced with a function as well - after all, all it does is
allocation and initialization, as opposed to delete's destruction and
deallocation.
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

Not quite. New is different because it is a primitive - it can't be
implemented as a function (well it could if it user placement new, but
we're back to square one). In contrast, delete already knows the type of
the object it's destroying and can call the destructor manually so it is
easy to implement as a regular function.

Sorry, not following you here. Why can't new be implemented as a templated
function which takes the type of the object to create as a template
parameter? Isn't it just allocation, copying over T.init, then calling the
constructor with whatever arguments the user passed?

But manual deletion has no business in the garbage collected heap.

Isn't that making the assumption that all D programs are garbage-collected?

That currently druntime provides it is an accident caused by thecurrent
implementation; most GC's cannot provide efficient manual reclamation.
And they shouldn't anyway.

Why not? And what if I don't use the GC (just the allocation/deallocation
aspects of it)?

(There's a longer discussion underneath this concerning what memory
really is and what GCs really are and do.)

I understand your points regarding leaving deallocation to happen on a GC
run being more efficient than manually deallocating objects individually,
but this doesn't cover all use-cases. Also, what if I need to deallocate a
large block of memory right now? I'd be forced to use the more verbose and
less safe "free" functions.
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

So in SafeD the delete operation can be replaced by something safer, a
way to tell the GC to deallocate just a specific object, walking the
graph and setting to null all the inbound pointers (this is a slower
operation).

I'd just like to point out that, without some careful considerations, this
would allow any code to modify memory belonging to completely unrelated
code it knows nothing about. Without careful planning it could cause the
program to crash due to null pointer dereferences in completely unrelated
areas, leaving you scratching your head why is that pointer null in the
first place.
I think that a better idea is "safe deletion": make a precise GC examine
the entire graph and make sure that the calling code has the only
reference to the object before deleting it.
(This isn't practical anyway, because it'd probably be too slow to be
useful for most cases, and doesn't apply to current D implementations.)
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net

Not quite. New is different because it is a primitive - it can't be
implemented as a function (well it could if it user placement new, but
we're back to square one). In contrast, delete already knows the type
of the object it's destroying and can call the destructor manually so
it is easy to implement as a regular function.

Sorry, not following you here. Why can't new be implemented as a
templated function which takes the type of the object to create as a
template parameter? Isn't it just allocation, copying over T.init, then
calling the constructor with whatever arguments the user passed?

In D it's possible to call the constructor by issuing __ctor(). In C++
there is no way, hence the presence of a language primitive.

Doesn't that only support my argument that new *can* be implemented as a
function?

You should use malloc() and free() for manual memory management and the
GC for automatic memory management. Each of these two is ill-suited for
carrying the other's job.

Thanks, that was really what I was missing from the picture.
However, D allows overloading new/delete for custom allocators[1] - this
can be used for a cleaner way to allocate objects in unmanaged memory. I
don't see why this use of the delete keyword should be deprecated.
[1]: http://digitalmars.com/d/2.0/memory.html#newdelete
--
Best regards,
Vladimir mailto:vladimir thecybershadow.net