5.3.14 Configuring MySQL Cluster Send Buffer Parameters

Formerly, the NDB kernel used a send buffer whose size was fixed
at 2MB for every node in the cluster, which was allocated when
the node started. Because the size of this buffer could not be
changed after the cluster was started, it was necessary to make
it large enough in advance to accommodate the maximum possible
load on any transporter socket. However, this was an inefficient
use of memory, since much of it often went unused, and could
result in large amounts of resources being wasted when scaling
up to many API nodes.

Beginning with MySQL Cluster NDB 7.0, this problem is solved by
employing a unified send buffer whose memory is allocated
dynamically from a pool shared by all transporters. This means
that the size of the send buffer can be adjusted as necessary.
Configuration of the unified send buffer can accomplished by
setting the following parameters:

TotalSendBufferMemory.
This parameter can be set for all types of MySQL Cluster
nodes—that is, it can be set in the
[ndbd], [mgm], and
[api] (or [mysql])
sections of the config.ini file. It
represents the total amount of memory (in bytes) to be
allocated by each node for which it is set for use among
all configured transporters. If set, its minimum is 256KB;
the maximum is 4294967039.

To be backward-compatible with existing configurations, this
parameter takes as its default value the sum of the maximum
send buffer sizes of all configured transporters, plus an
additional 32KB (one page) per transporter. The maximum
depends on the type of transporter, as shown in the
following table:

This enables existing configurations to function in close to
the same way as they did with MySQL Cluster NDB 6.3 and
earlier, with the same amount of memory and send buffer
space available to each transporter. However, memory that is
unused by one transporter is not available to other
transporters.

OverloadLimit.
This parameter is used in the
config.ini file
[tcp] section, and denotes the amount
of unsent data (in bytes) that must be present in the send
buffer before the connection is considered overloaded.
When such an overload condition occurs, transactions that
affect the overloaded connection fail with NDB API Error
1218 (Send Buffers overloaded in NDB
kernel) until the overload status passes. The
default value is 0, in which case the effective overload
limit is calculated as SendBufferMemory *
0.8 for a given connection. The maximum value
for this parameter is 4G.

SendBufferMemory.
In MySQL Cluster NDB 6.3 and earlier, this TCP
configuration parameter represented the amount of memory
allocated at startup for each configured TCP connection.
Beginning with MySQL Cluster NDB 7.0, this value denotes a
hard limit for the amount of memory that may be used by a
single transporter out of the entire pool specified by
TotalSendBufferMemory.
However, the sum of SendBufferMemory
for all configured transporters may be greater than the
TotalSendBufferMemory
that is set for a given node. This is a way to save memory
when many nodes are in use, as long as the maximum amount
of memory is never required by all transporters at the
same time.

ReservedSendBufferMemory.
This optional data node parameter, if set, gives an amount
of memory (in bytes) that is reserved for connections
between data nodes; this memory is not allocated to send
buffers used for communications with management servers or
API nodes. This provides a way to protect the cluster
against misbehaving API nodes that use excess send memory
and thus cause failures in communications internally in
the NDB kernel. If set, its the minimum permitted value
for this parameters is 256KB; the maximum is 4294967039.