16.2.2.7 Memory Allocation within memcached

When you first start memcached, the memory
that you have configured is not automatically allocated.
Instead, memcached only starts allocating and
reserving physical memory once you start saving information into
the cache.

When you start to store data into the cache,
memcached does not allocate the memory for
the data on an item by item basis. Instead, a slab allocation is
used to optimize memory usage and prevent memory fragmentation
when information expires from the cache.

With slab allocation, memory is reserved in blocks of 1MB. The
slab is divided up into a number of blocks of equal size. When
you try to store a value into the cache,
memcached checks the size of the value that
you are adding to the cache and determines which slab contains
the right size allocation for the item. If a slab with the item
size already exists, the item is written to the block within the
slab.

If the new item is bigger than the size of any existing blocks,
then a new slab is created, divided up into blocks of a suitable
size. If an existing slab with the right block size already
exists, but there are no free blocks, a new slab is created. If
you update an existing item with data that is larger than the
existing block allocation for that key, then the key is
re-allocated into a suitable slab.

For example, the default size for the smallest block is 88 bytes
(40 bytes of value, and the default 48 bytes for the key and
flag data). If the size of the first item you store into the
cache is less than 40 bytes, then a slab with a block size of 88
bytes is created and the value stored.

If the size of the data that you intend to store is larger than
this value, then the block size is increased by the chunk size
factor until a block size large enough to hold the value is
determined. The block size is always a function of the scale
factor, rounded up to a block size which is exactly divisible
into the chunk size.

The result is that you have multiple pages allocated within the
range of memory allocated to memcached. Each
page is 1MB in size (by default), and is split into a different
number of chunks, according to the chunk size required to store
the key/value pairs. Each instance has multiple pages allocated,
and a page is always created when a new item needs to be created
requiring a chunk of a particular size. A slab may consist of
multiple pages, and each page within a slab contains an equal
number of chunks.

The chunk size of a new slab is determined by the base chunk
size combined with the chunk size growth factor. For example, if
the initial chunks are 104 bytes in size, and the default chunk
size growth factor is used (1.25), then the next chunk size
allocated would be the best power of 2 fit for 104*1.25, or 136
bytes.

Allocating the pages in this way ensures that memory does not
get fragmented. However, depending on the distribution of the
objects that you store, it may lead to an inefficient
distribution of the slabs and chunks if you have significantly
different sized items. For example, having a relatively small
number of items within each chunk size may waste a lot of memory
with just few chunks in each allocated page.

You can tune the growth factor to reduce this effect by using
the -f command line option, which adapts the
growth factor applied to make more effective use of the chunks
and slabs allocated. For information on how to determine the
current slab allocation statistics, see
Section 16.2.4.2, “memcached Slabs Statistics”.

If your operating system supports it, you can also start
memcached with the -L
command line option. This option preallocates all the memory
during startup using large memory pages. This can improve
performance by reducing the number of misses in the CPU memory
cache.