erts_alloc

erts_alloc is an Erlang Run-Time System internal memory
allocator library. erts_alloc provides the Erlang
Run-Time System with a number of memory allocators.

Allocators

Currently the following allocators are present:

temp_alloc

Allocator used for temporary allocations.

eheap_alloc

Allocator used for Erlang heap data, such as Erlang process heaps.

binary_alloc

Allocator used for Erlang binary data.

ets_alloc

Allocator used for ETS data.

driver_alloc

Allocator used for driver data.

sl_alloc

Allocator used for memory blocks that are expected to be
short-lived.

ll_alloc

Allocator used for memory blocks that are expected to be
long-lived, for example Erlang code.

fix_alloc

A fast allocator used for some frequently used
fixed size data types.

std_alloc

Allocator used for most memory blocks not allocated via any of
the other allocators described above.

sys_alloc

This is normally the default malloc implementation
used on the specific OS.

mseg_alloc

A memory segment allocator. mseg_alloc is used by other
allocators for allocating memory segments and is currently only
available on systems that have the mmap system
call. Memory segments that are deallocated are kept for a
while in a segment cache before they are destroyed. When
segments are allocated, cached segments are used if possible
instead of creating new segments. This in order to reduce
the number of system calls made.

sys_alloc is always enabled and
cannot be disabled. mseg_alloc is always enabled if it is
available and an allocator that uses it is enabled. All other
allocators can be enabled or disabled.
By default all allocators are enabled.
When an allocator is disabled, sys_alloc is used instead of
the disabled allocator.

The main idea with the erts_alloc library is to separate
memory blocks that are used differently into different memory
areas, and by this achieving less memory fragmentation. By
putting less effort in finding a good fit for memory blocks that
are frequently allocated than for those less frequently
allocated, a performance gain can be achieved.

The alloc_util framework

Internally a framework called alloc_util is used for
implementing allocators. sys_alloc, and
mseg_alloc do not use this framework; hence, the
following does not apply to them.

An allocator manages multiple areas, called carriers, in which
memory blocks are placed. A carrier is either placed in a
separate memory segment (allocated via mseg_alloc), or in
the heap segment (allocated via sys_alloc). Multiblock
carriers are used for storage of several blocks. Singleblock
carriers are used for storage of one block. Blocks that are
larger than the value of the singleblock carrier threshold
(sbct) parameter are placed
in singleblock carriers. Blocks that are smaller than the value
of the sbct parameter are placed in multiblock
carriers. Normally an allocator creates a "main multiblock
carrier". Main multiblock carriers are never deallocated. The
size of the main multiblock carrier is determined by the value
of the mmbcs parameter.

Sizes of multiblock carriers
allocated via mseg_alloc are
decided based on the values of the largest multiblock carrier
size (lmbcs), the smallest
multiblock carrier size (smbcs),
and the multiblock carrier growth stages
(mbcgs) parameters. If
nc is the current number of multiblock carriers (the main
multiblock carrier excluded) managed by an allocator, the size
of the next mseg_alloc multiblock carrier allocated by
this allocator will roughly be
smbcs+nc*(lmbcs-smbcs)/mbcgs when
nc <= mbcgs,
and lmbcs when nc > mbcgs. If the value of the
sbct parameter should be larger than the value of the
lmbcs parameter, the allocator may have to create
multiblock carriers that are larger than the value of the
lmbcs parameter, though.
Singleblock carriers allocated via mseg_alloc are sized
to whole pages.

Sizes of carriers allocated via sys_alloc are
decided based on the value of the sys_alloc carrier size
(ycs) parameter. The size of
a carrier is the least number of multiples of the value of the
ycs parameter that satisfies the request.

Coalescing of free blocks are always performed immediately.
Boundary tags (headers and footers) in free blocks are used
which makes the time complexity for coalescing constant.

The memory allocation strategy
used for multiblock carriers by an
allocator is configurable via the as
parameter. Currently the following strategies are available:

Implementation: A balanced binary search tree is
used. The time complexity is proportional to log N, where
N is the number of sizes of free blocks.

Address order best fit

Strategy: Find the smallest block that satisfies the
requested block size. If multiple blocks are found, choose
the one with the lowest address.

Implementation: A balanced binary search tree is
used. The time complexity is proportional to log N, where
N is the number of free blocks.

Address order first fit

Strategy: Find the block with the lowest address that satisfies the
requested block size.

Implementation: A balanced binary search tree is
used. The time complexity is proportional to log N, where
N is the number of free blocks.

Address order first fit carrier best fit

Strategy: Find the carrier with the lowest address that
can satisfy the requested block size, then find a block within
that carrier using the "best fit" strategy.

Implementation: Balanced binary search trees are
used. The time complexity is proportional to log N, where
N is the number of free blocks.

Address order first fit carrier address order best fit

Strategy: Find the carrier with the lowest address that
can satisfy the requested block size, then find a block within
that carrier using the "adress order best fit" strategy.

Implementation: Balanced binary search trees are
used. The time complexity is proportional to log N, where
N is the number of free blocks.

Good fit

Strategy: Try to find the best fit, but settle for the best fit
found during a limited search.

Implementation: The implementation uses segregated free
lists with a maximum block search depth (in each list) in
order to find a good fit fast. When the maximum block
search depth is small (by default 3) this implementation
has a time complexity that is constant. The maximum block
search depth is configurable via the
mbsd parameter.

A fit

Strategy: Do not search for a fit, inspect only one free
block to see if it satisfies the request. This strategy is
only intended to be used for temporary allocations.

Implementation: Inspect the first block in a free-list.
If it satisfies the request, it is used; otherwise, a new
carrier is created. The implementation has a time
complexity that is constant.

As of erts version 5.6.1 the emulator will refuse to
use this strategy on other allocators than temp_alloc.
This since it will only cause problems for other allocators.

Apart from the ordinary allocators described above a number of
pre-allocators are used for some specific data types. These
pre-allocators pre-allocate a fixed amount of memory for certain data
types when the run-time system starts. As long as pre-allocated memory
is available, it will be used. When no pre-allocated memory is
available, memory will be allocated in ordinary allocators. These
pre-allocators are typically much faster than the ordinary allocators,
but can only satisfy a limited amount of requests.

System Flags Effecting erts_alloc

Warning!

Only use these flags if you are absolutely sure what you are
doing. Unsuitable settings may cause serious performance
degradation and even a system crash at any time during
operation.

Memory allocator system flags have the following syntax:
+M<S><P> <V>
where <S> is a letter identifying a subsystem,
<P> is a parameter, and <V> is the
value to use. The flags can be passed to the Erlang emulator
(erl) as command line
arguments.

System flags effecting specific allocators have an upper-case
letter as <S>. The following letters are used for
the currently present allocators:

Absolute max cache bad fit (in kilobytes). A segment in the
memory segment cache is not reused if its size exceeds the
requested size with more than the value of this
parameter. Default value is 4096.

+MMrmcbf <ratio>

Relative max cache bad fit (in percent). A segment in the
memory segment cache is not reused if its size exceeds the
requested size with more than relative max cache bad fit
percent of the requested size. Default value is 20.

+MMsco true|false

Set super carrier only flag. This
flag defaults to true. When a super carrier is used and this
flag is true, mseg_alloc will only create carriers
in the super carrier. Note that the alloc_util framework may
create sys_alloc carriers, so if you want all carriers to
be created in the super carrier, you therefore want to disable use
of sys_alloc carriers by also passing
+Musac false. When the flag
is false, mseg_alloc will try to create carriers outside
of the super carrier when the super carrier is full.

NOTE: Setting this flag to false may not be supported
on all systems. This flag will in that case be ignored.

NOTE: The super carrier cannot be enabled nor
disabled on halfword heap systems. This flag will be
ignored on halfword heap systems.

+MMscrfsd <amount>

Set super carrier reserved
free segment descriptors. This parameter defaults to 65536.
This parameter determines the amount of memory to reserve for
free segment descriptors used by the super carrier. If the system
runs out of reserved memory for free segment descriptors, other
memory will be used. This may however cause fragmentation issues,
so you want to ensure that this never happens. The maximum amount
of free segment descriptors used can be retrieved from the
erts_mmap tuple part of the result from calling
erlang:system_info({allocator, mseg_alloc}).

+MMscrpm true|false

Set super carrier reserve physical
memory flag. This flag defaults to true. When this flag is
true, physical memory will be reserved for the whole super
carrier at once when it is created. The reservation will after that
be left unchanged. When this flag is set to false only virtual
address space will be reserved for the super carrier upon creation.
The system will attempt to reserve physical memory upon carrier
creations in the super carrier, and attempt to unreserve physical
memory upon carrier destructions in the super carrier.

NOTE: What reservation of physical memory actually means
highly depends on the operating system, and how it is configured. For
example, different memory overcommit settings on Linux drastically
change the behaviour. Also note, setting this flag to false
may not be supported on all systems. This flag will in that case
be ignored.

NOTE: The super carrier cannot be enabled nor
disabled on halfword heap systems. This flag will be
ignored on halfword heap systems.

+MMscs <size in MB>

Set super carrier size (in MB). The super carrier size defaults to
zero; i.e, the super carrier is by default disabled. The super
carrier is a large continuous area in the virtual address space.
mseg_alloc will always try to create new carriers in the super
carrier if it exists. Note that the alloc_util framework may
create sys_alloc carriers. For more information on this, see the
documentation of the +MMsco
flag.

NOTE: The super carrier cannot be enabled nor
disabled on halfword heap systems. This flag will be
ignored on halfword heap systems.

+MMmcs <amount>

Max cached segments. The maximum number of memory segments
stored in the memory segment cache. Valid range is
0-30. Default value is 10.

The following flags are available for configuration of
sys_alloc:

+MYe true

Enable sys_alloc. Note: sys_alloc cannot be disabled.

+MYm libc

malloc library to use. Currently only
libc is available. libc enables the standard
libc malloc implementation. By default libc is used.

+MYtt <size>

Trim threshold size (in kilobytes). This is the maximum amount
of free memory at the top of the heap (allocated by
sbrk) that will be kept by malloc (not
released to the operating system). When the amount of free
memory at the top of the heap exceeds the trim threshold,
malloc will release it (by calling
sbrk). Trim threshold is given in kilobytes. Default
trim threshold is 128. Note: This flag will
only have any effect when the emulator has been linked with
the GNU C library, and uses its malloc implementation.

+MYtp <size>

Top pad size (in kilobytes). This is the amount of extra
memory that will be allocated by malloc when
sbrk is called to get more memory from the operating
system. Default top pad size is 0. Note: This flag
will only have any effect when the emulator has been linked
with the GNU C library, and uses its malloc
implementation.

The following flags are available for configuration of allocators
based on alloc_util. If u is used as subsystem
identifier (i.e., <S> = u) all allocators based on
alloc_util will be effected. If B, D, E,
F, H, L, R, S, or T is used as
subsystem identifier, only the specific allocator identified will be
effected:

+M<S>acul <utilization>|de

Abandon carrier utilization limit. A valid
<utilization> is an integer in the range
[0, 100] representing utilization in percent. When a
utilization value larger than zero is used, allocator instances
are allowed to abandon multiblock carriers. If de (default
enabled) is passed instead of a <utilization>,
a recomended non zero utilization value will be used. The actual
value chosen depend on allocator type and may be changed between
ERTS versions. Currently the default equals de, but this
may be changed in the future. Carriers will be abandoned when
memory utilization in the allocator instance falls below the
utilization value used. Once a carrier has been abandoned, no new
allocations will be made in it. When an allocator instance gets an
increased multiblock carrier need, it will first try to fetch an
abandoned carrier from an allocator instances of the same
allocator type. If no abandoned carrier could be fetched, it will
create a new empty carrier. When an abandoned carrier has been
fetched it will function as an ordinary carrier. This feature has
special requirements on the
allocation strategy used. Currently
only the strategies aoff, aoffcbf and aoffcaobf support
abandoned carriers. This feature also requires
multiple thread specific instances
to be enabled. When enabling this feature, multiple thread specific
instances will be enabled if not already enabled, and the
aoffcbf strategy will be enabled if current strategy does not
support abandoned carriers. This feature can be enabled on all
allocators based on the alloc_util framework with the
exception of temp_alloc (which would be pointless).

+M<S>as bf|aobf|aoff|aoffcbf|aoffcaobf|gf|af

Allocation strategy. Valid strategies are bf (best fit),
aobf (address order best fit), aoff (address order first fit),
aoffcbf (address order first fit carrier best fit),
aoffcaobf (address order first fit carrier address order best fit),
gf (good fit), and af (a fit). See
the description of allocation strategies in "the alloc_util framework" section.

+M<S>asbcst <size>

Absolute singleblock carrier shrink threshold (in
kilobytes). When a block located in an
mseg_alloc singleblock carrier is shrunk, the carrier
will be left unchanged if the amount of unused memory is less
than this threshold; otherwise, the carrier will be shrunk.
See also rsbcst.

Max block search depth. This flag has effect only if the
good fit strategy has been selected for allocator
<S>. When the good fit strategy is used, free
blocks are placed in segregated free-lists. Each free list
contains blocks of sizes in a specific range. The max block
search depth sets a limit on the maximum number of blocks to
inspect in a free list during a search for suitable block
satisfying the request.

+M<S>mmbcs <size>

Main multiblock carrier size. Sets the size of the main
multiblock carrier for allocator <S>. The main
multiblock carrier is allocated via sys_alloc and is
never deallocated.

+M<S>mmmbc <amount>

Max mseg_alloc multiblock carriers. Maximum number of
multiblock carriers allocated via mseg_alloc by
allocator <S>. When this limit has been reached,
new multiblock carriers will be allocated via
sys_alloc.

+M<S>mmsbc <amount>

Max mseg_alloc singleblock carriers. Maximum number of
singleblock carriers allocated via mseg_alloc by
allocator <S>. When this limit has been reached,
new singleblock carriers will be allocated via
sys_alloc.

+M<S>ramv <bool>

Realloc always moves. When enabled, reallocate operations will
more or less be translated into an allocate, copy, free sequence.
This often reduce memory fragmentation, but costs performance.

+M<S>rmbcmt <ratio>

Relative multiblock carrier move threshold (in percent). When
a block located in a multiblock carrier is shrunk,
the block will be moved if the ratio of the size of the returned
memory compared to the previous size is more than this threshold;
otherwise, the block will be shrunk at current location.

+M<S>rsbcmt <ratio>

Relative singleblock carrier move threshold (in percent). When
a block located in a singleblock carrier is shrunk to
a size smaller than the value of the
sbct parameter,
the block will be left unchanged in the singleblock carrier if
the ratio of unused memory is less than this threshold;
otherwise, it will be moved into a multiblock carrier.

+M<S>rsbcst <ratio>

Relative singleblock carrier shrink threshold (in
percent). When a block located in an mseg_alloc
singleblock carrier is shrunk, the carrier will be left
unchanged if the ratio of unused memory is less than this
threshold; otherwise, the carrier will be shrunk.
See also asbcst.

+M<S>sbct <size>

Singleblock carrier threshold. Blocks larger than this
threshold will be placed in singleblock carriers. Blocks
smaller than this threshold will be placed in multiblock
carriers. On 32-bit Unix style OS this threshold can not be set higher
than 8 megabytes.

Multiple, thread specific instances of the allocator.
This option will only have any effect on the runtime system
with SMP support. Default behaviour on the runtime system with
SMP support is NoSchedulers+1 instances. Each scheduler will use
a lock-free instance of its own and other threads will use
a common instance.

It was previously (before ERTS version 5.9) possible to configure
a smaller amount of thread specific instances than schedulers.
This is, however, not possible any more.

Currently the following flags are available for configuration of
alloc_util, i.e. all allocators based on alloc_util
will be effected:

+Muycs <size>

sys_alloc carrier size. Carriers allocated via
sys_alloc will be allocated in sizes which are
multiples of the sys_alloc carrier size. This is not
true for main multiblock carriers and carriers allocated
during a memory shortage, though.

+Mummc <amount>

Max mseg_alloc carriers. Maximum number of carriers
placed in separate memory segments. When this limit has been
reached, new carriers will be placed in memory retrieved from
sys_alloc.

+Musac <bool>

Allow sys_alloc carriers. By default true. If
set to false, sys_alloc carriers will never be
created by allocators using the alloc_util framework.

Instrumentation flags:

+Mim true|false

A map over current allocations is kept by the emulator. The
allocation map can be retrieved via the instrument
module. +Mim true implies +Mis true.
+Mim true is the same as
-instr.

+Mis true|false

Status over allocated memory is kept by the emulator. The
allocation status can be retrieved via the instrument
module.

+Mit X

Reserved for future use. Do not use this flag.

Note!

When instrumentation of the emulator is enabled, the emulator
uses more memory and runs slower.

Other flags:

+Mea min|max|r9c|r10b|r11b|config

min

Disables all allocators that can be disabled.

max

Enables all allocators (currently default).

r9c|r10b|r11b

Configures all allocators as they were configured in respective
OTP release. These will eventually be removed.

config

Disables features that cannot be enabled while creating an
allocator configuration with
erts_alloc_config(3).
Note, this option should only be used while running
erts_alloc_config, not when using the created
configuration.

+Mlpm all|no

Lock physical memory. The default value is no, i.e.,
no physical memory will be locked. If set to all, all
memory mappings made by the runtime system, will be locked into
physical memory. If set to all, the runtime system will fail
to start if this feature is not supported, the user has not got enough
privileges, or the user is not allowed to lock enough physical memory.
The runtime system will also fail with an out of memory condition
if the user limit on the amount of locked memory is reached.