I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.
The design of TempAlloc is plenty adequate for its charter, but the
charter itself is too narrow. I'd like to design together with David a
more general allocator interface in which TempAlloc would be a player.
That requires massive changes to the interface, but relatively small
changes to the implementation.
If we don't do this, TempAlloc would be an outlier once we have
allocator, and would need to be adapted in order to mix and match with
other allocation strategies.
Please let me know if that's a possibility and I'll get back with details.
Thanks,
Andrei

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.
The design of TempAlloc is plenty adequate for its charter, but the
charter itself is too narrow. I'd like to design together with David a
more general allocator interface in which TempAlloc would be a player.
That requires massive changes to the interface, but relatively small
changes to the implementation.
If we don't do this, TempAlloc would be an outlier once we have
allocator, and would need to be adapted in order to mix and match with
other allocation strategies.
Please let me know if that's a possibility and I'll get back with details.
Thanks,
Andrei

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

I'm thinking of defining a general interface that catches malloc, the
GC, scoped allocators a la TempAlloc, and possibly compound allocators a
la reaps.
I'll start with an example:
struct Mallocator
{
/** Allocates a chunk of size s. */
static void * allocate(size_t s) {
return enforce(malloc(s), new OutOfMemory);
}
/** Optional: frees a chunk allocated with allocate */
static void free(void *p) {
.free(p);
}
/** Optional: frees all chunks allocated with allocate */
// static void freeAll();
/** Resizes a chunk allocated with allocate without moving.
Required, but may be implemented to always return false. */
static bool resize(void* p, size_t newSize) {
// Can't use realloc here
return false;
}
/** true if calls to free enforce the pointer is valid */
enum { freeIsChecked = false }
/** Returns the guaranteed alignment for allocation requests
of size s. Must be computable during compilation. */
size_t align(size_t s) { return (void*).sizeof; }
/** Returns the size effectively allocated if an allocation
of size s were requested. Must be computable during
compilation.*/
static size_t allocSize(size_t s) {
return ((s + align(s) - 1) / align(s)) * align(s);
}
/** true if memory is automatically reclaimed (GC) */
enum { isAutomatic = false }
/** true if allocator is scoped, i.e. all memory allocated
goes away when the original allocator goes out of scope */
enum { isScoped = false }
}
This is a symbolic interface. Some or all of the functions above may be
static. I think, without being yet positive, that the allocator should
have reference semantics, i.e. all copies of a given allocator should
manipulate the same memory. This should reduce unwitting errors caused
e.g. by passing an allocator by reference instead of by value.
For TempAlloc that means the primitives frameInit and frameFree are
replaced with an object, e.g. RegionAllocator. That object and all of
its copies manage a given frame. (BTW I think the name "region" is
consecrated and should replace "frame" throughout in TempAlloc. In fact
TempAlloc should be renamed to RegionAlloc.)
With this interface at hand I think we can tackle a variety of
heavy-duty allocation tasks in a relatively abstract manner. In
particular, we can compose several allocation strategies (e.g. a heap on
top of a region). A simpler task would be e.g. to define an aligned
allocator on top of an unaligned one.
Andrei

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

}
This is a symbolic interface. Some or all of the functions above may be
static. I think, without being yet positive, that the allocator should
have reference semantics, i.e. all copies of a given allocator should
manipulate the same memory. This should reduce unwitting errors caused
e.g. by passing an allocator by reference instead of by value.
For TempAlloc that means the primitives frameInit and frameFree are
replaced with an object, e.g. RegionAllocator. That object and all of
its copies manage a given frame. (BTW I think the name "region" is
consecrated and should replace "frame" throughout in TempAlloc. In fact
TempAlloc should be renamed to RegionAlloc.)
With this interface at hand I think we can tackle a variety of
heavy-duty allocation tasks in a relatively abstract manner. In
particular, we can compose several allocation strategies (e.g. a heap on
top of a region). A simpler task would be e.g. to define an aligned
allocator on top of an unaligned one.
Andrei

If core.memory.GC is to adopt the Mallocator interface, I think it's
better not to break backward compatibility without a good reason.

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

We can use either the malloc/free nomenclature or the
allocate/deallocate nomenclature. I personally favor the latter so as to
not create confusion. The semantics of allocation may be quite different
from that of malloc.

If core.memory.GC is to adopt the Mallocator interface, I think it's
better not to break backward compatibility without a good reason.

Good point. Generally the GC allocator was intended quite specifically
for GC, but the closer we get the APIs together, the better. Probably
the allocator interface should have a notion of type and type-specific
allocation attributes.
Andrei

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before
said
interface is decided upon. ;)
-Lars

We can use either the malloc/free nomenclature or the
allocate/deallocate nomenclature. I personally favor the latter so as to
not create confusion. The semantics of allocation may be quite different
from that of malloc.

If core.memory.GC is to adopt the Mallocator interface, I think it's
better not to break backward compatibility without a good reason.

Good point. Generally the GC allocator was intended quite specifically
for GC, but the closer we get the APIs together, the better. Probably
the allocator interface should have a notion of type and type-specific
allocation attributes.
Andrei

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

I'm thinking of defining a general interface that catches malloc, the
GC, scoped allocators a la TempAlloc, and possibly compound allocators a
la reaps.
I'll start with an example:
struct Mallocator
{
/** Allocates a chunk of size s. */
static void * allocate(size_t s) {
return enforce(malloc(s), new OutOfMemory);
}
/** Optional: frees a chunk allocated with allocate */
static void free(void *p) {
.free(p);
}
/** Optional: frees all chunks allocated with allocate */
// static void freeAll();
/** Resizes a chunk allocated with allocate without moving.
Required, but may be implemented to always return false. */
static bool resize(void* p, size_t newSize) {
// Can't use realloc here
return false;
}
/** true if calls to free enforce the pointer is valid */
enum { freeIsChecked = false }
/** Returns the guaranteed alignment for allocation requests
of size s. Must be computable during compilation. */
size_t align(size_t s) { return (void*).sizeof; }
/** Returns the size effectively allocated if an allocation
of size s were requested. Must be computable during
compilation.*/
static size_t allocSize(size_t s) {
return ((s + align(s) - 1) / align(s)) * align(s);
}
/** true if memory is automatically reclaimed (GC) */
enum { isAutomatic = false }
/** true if allocator is scoped, i.e. all memory allocated
goes away when the original allocator goes out of scope */
enum { isScoped = false }
}
This is a symbolic interface. Some or all of the functions above may be
static. I think, without being yet positive, that the allocator should
have reference semantics, i.e. all copies of a given allocator should
manipulate the same memory. This should reduce unwitting errors caused
e.g. by passing an allocator by reference instead of by value.
For TempAlloc that means the primitives frameInit and frameFree are
replaced with an object, e.g. RegionAllocator. That object and all of
its copies manage a given frame. (BTW I think the name "region" is
consecrated and should replace "frame" throughout in TempAlloc. In fact
TempAlloc should be renamed to RegionAlloc.)

It's ok to allow an object to replace frameInit and frameFree for
conformance to the general allocator interface, but I'd need to keep
frameInit and frameFree around. They're useful for functions that
return pointers to TempAlloc-allocated memory. I don't want to have to
pass in a RegionAllocator object to each of these because it's too
verbose. I want to be able to do:
void doStuff() {
TempAlloc.frameInit();
scope(exit) TempAlloc.frameFree();
auto arr = getArray();
}
uint[] getArray() {
return TempAlloc.newArray!(uint[])(5);
}
My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

It's ok to allow an object to replace frameInit and frameFree for
conformance to the general allocator interface, but I'd need to keep
frameInit and frameFree around. They're useful for functions that return
pointers to TempAlloc-allocated memory. I don't want to have to pass in
a RegionAllocator object to each of these because it's too verbose. I
want to be able to do:
void doStuff() {
TempAlloc.frameInit();
scope(exit) TempAlloc.frameFree();
auto arr = getArray();
}
uint[] getArray() {
return TempAlloc.newArray!(uint[])(5);
}
My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

I was actually glad of that particular outcome...

I suggest to think about it three times before giving FrameAlloc reference
semantics. I'd even like a variant of Appender without reference semantics.
Bye,
bearophile

It's ok to allow an object to replace frameInit and frameFree for
conformance to the general allocator interface, but I'd need to keep
frameInit and frameFree around. They're useful for functions that return
pointers to TempAlloc-allocated memory. I don't want to have to pass in
a RegionAllocator object to each of these because it's too verbose. I
want to be able to do:
void doStuff() {
TempAlloc.frameInit();
scope(exit) TempAlloc.frameFree();
auto arr = getArray();
}
uint[] getArray() {
return TempAlloc.newArray!(uint[])(5);
}
My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

It's ok to allow an object to replace frameInit and frameFree for
conformance to the general allocator interface, but I'd need to keep
frameInit and frameFree around. They're useful for functions that return
pointers to TempAlloc-allocated memory. I don't want to have to pass in
a RegionAllocator object to each of these because it's too verbose. I
want to be able to do:
void doStuff() {
TempAlloc.frameInit();
scope(exit) TempAlloc.frameFree();
auto arr = getArray();
}
uint[] getArray() {
return TempAlloc.newArray!(uint[])(5);
}
My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

I was actually glad of that particular outcome...

??? What outcome?

I was glad that one needs to pass a TempAlloc object down to the
function, instead of that function silently returning memory allocated
with TempAlloc. Generally I have a very strong stance against stuff
that's simultaneously terse, implied, and unsafe. In fact the moment you
mentioned that the one reason against passing TempAlloc objects down is
verboseness, I interpreted that as a good argument why it's _good_ to do
that.
I do agree with your choice of scan flags because your analysis of
costs, benefits, and onus put on the user is compelling. In this case,
however, it seems to me that functions that implicitly return stuff on
the TempAlloc stack are paving the way towards messed-up modules that
can't be reasoned about modularly.
Thanks,
Andrei

I do agree with your choice of scan flags because your analysis of
costs, benefits, and onus put on the user is compelling. In this case,
however, it seems to me that functions that implicitly return stuff on
the TempAlloc stack are paving the way towards messed-up modules that
can't be reasoned about modularly.
Thanks,
Andrei

There are two use cases for implicitly returning TempAlloc-allocated memory:
1. In a private API.
2. I have a few data structures that I may clean up and submit as a proposal
later (hash table, hash set, AVL tree) whose implementations are specifically
optimized for TempAlloc. For example, the hash table is provisionally called
StackHash. I'd really rather write:
auto table = StackHash!(uint, double)(10);
table[666] = 8675309;
rather than:
auto table = StackHash!(uint, double)(10);
table.addElement(666, 8675309, someLongVerboseAllocator);

I do agree with your choice of scan flags because your analysis of
costs, benefits, and onus put on the user is compelling. In this case,
however, it seems to me that functions that implicitly return stuff on
the TempAlloc stack are paving the way towards messed-up modules that
can't be reasoned about modularly.
Thanks,
Andrei

There are two use cases for implicitly returning TempAlloc-allocated memory:
1. In a private API.

If we provide an artifact good for private APIs but dangerous for true
modular code, I think this is a weak argument.

2. I have a few data structures that I may clean up and submit as a proposal
later (hash table, hash set, AVL tree) whose implementations are specifically
optimized for TempAlloc. For example, the hash table is provisionally called
StackHash. I'd really rather write:
auto table = StackHash!(uint, double)(10);
table[666] = 8675309;
rather than:
auto table = StackHash!(uint, double)(10);
table.addElement(666, 8675309, someLongVerboseAllocator);

Couldn't StackHash's constructor accept an allocator as an argument?
I think there are good ways to solve these issues if we start from a
shared view that implicit region allocation has a lot going against it.
In that case, I'm sure we'll find creative solutions. Right now it seems
we're engaging in a pattern of debating disadvantages of passing regions
explicitly, which ignores the disadvantages of having implicit ranges.
Thanks,
Andrei

I do agree with your choice of scan flags because your analysis of
costs, benefits, and onus put on the user is compelling. In this case,
however, it seems to me that functions that implicitly return stuff on
the TempAlloc stack are paving the way towards messed-up modules that
can't be reasoned about modularly.
Thanks,
Andrei

There are two use cases for implicitly returning TempAlloc-allocated
memory:
1. In a private API.

If we provide an artifact good for private APIs but dangerous for true
modular code, I think this is a weak argument.

Hmm, I'd be fine with leaving out any blatantly obvious way to get
implicit TempAlloc allocation as long as there's a backdoor in place so
that it can be done if you really want to. For example, when allocating
from RegionAllocator, you'd probably want to check that you're
allocating from the last created RegionAllocator anyhow, at least in
debug mode. Therefore, you'd probably want every RegionAllocator to
have a pointer to the previous RegionAllocator. Then you could have a
thread-local RegionAllocator.lastCreated. You could then allocate from
RegionAllocator.lastCreated if you really, really wanted to.

2. I have a few data structures that I may clean up and submit as a
proposal
later (hash table, hash set, AVL tree) whose implementations are
specifically
optimized for TempAlloc. For example, the hash table is provisionally
called
StackHash. I'd really rather write:
auto table = StackHash!(uint, double)(10);
table[666] = 8675309;
rather than:
auto table = StackHash!(uint, double)(10);
table.addElement(666, 8675309, someLongVerboseAllocator);

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

I am hoping we can arrange things such that a RegionAllocator created
from scratch initializes a new frame, whereas subsequent copies of it
use that same frame. Would that work?
Andrei

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

from scratch initializes a new frame, whereas subsequent copies of it
use that same frame. Would that work?
Andrei

No. I don't want every creation of a new frame to require a GC heap allocation.

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

from scratch initializes a new frame, whereas subsequent copies of it
use that same frame. Would that work?
Andrei

No. I don't want every creation of a new frame to require a GC heap
allocation.

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator. Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

from scratch initializes a new frame, whereas subsequent copies of it
use that same frame. Would that work?
Andrei

No. I don't want every creation of a new frame to require a GC heap
allocation.

I don't understand why such would be necessary.
Andrei

Maybe I'm missing something, but how else would a RegionAllocator have
reference semantics?

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator.
Since
TempAlloc is designed to avoid global GC locks/world stopping like the
plauge, this is obviously bad.

from scratch initializes a new frame, whereas subsequent copies of it
use that same frame. Would that work?
Andrei

No. I don't want every creation of a new frame to require a GC heap
allocation.

I don't understand why such would be necessary.
Andrei

Maybe I'm missing something, but how else would a RegionAllocator have
reference semantics?

I'm thinking along the lines of an Algebraic containing the actual state
and a pointer to the original object.
Andrei

My other concern is that giving RegionAllocator reference semantics
would, IIUC, require an allocation to allocate a RegionAllocator.
Since
TempAlloc is designed to avoid global GC locks/world stopping like
the
plauge, this is obviously bad.

from scratch initializes a new frame, whereas subsequent copies of it
use that same frame. Would that work?
Andrei

No. I don't want every creation of a new frame to require a GC heap
allocation.

I don't understand why such would be necessary.
Andrei

Maybe I'm missing something, but how else would a RegionAllocator have
reference semantics?

I'm thinking along the lines of an Algebraic containing the actual state
and a pointer to the original object.
Andrei

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.

As review manager, I don't mind postponing the review until some later
time.
As a community member and Phobos user, I think it would of course be
preferable if TempAlloc fits into a more general allocator interface.
As an active TempAlloc user, I hope it doesn't take too long before said
interface is decided upon. ;)
-Lars

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.
The design of TempAlloc is plenty adequate for its charter, but the
charter itself is too narrow. I'd like to design together with David a
more general allocator interface in which TempAlloc would be a player.
That requires massive changes to the interface, but relatively small
changes to the implementation.
If we don't do this, TempAlloc would be an outlier once we have
allocator, and would need to be adapted in order to mix and match with
other allocation strategies.
Please let me know if that's a possibility and I'll get back with details.
Thanks,
Andrei

Sorry for the late reply. I was on a camping trip without internet
access this weekend. This seems perfectly reasonable to me, and I'd
love to work on a general allocator interface and fit TempAlloc to it.

I'd like to kindly request the author, the review manager, and the
community that TempAlloc skips this round of reviews without inclusion
in Phobos, to be reviewed again later.
The design of TempAlloc is plenty adequate for its charter, but the
charter itself is too narrow. I'd like to design together with David a
more general allocator interface in which TempAlloc would be a player.
That requires massive changes to the interface, but relatively small
changes to the implementation.
If we don't do this, TempAlloc would be an outlier once we have
allocator, and would need to be adapted in order to mix and match with
other allocation strategies.
Please let me know if that's a possibility and I'll get back with
details.
Thanks,
Andrei

Sorry for the late reply. I was on a camping trip without internet
access this weekend. This seems perfectly reasonable to me, and I'd love
to work on a general allocator interface and fit TempAlloc to it.