Managing Broker System-Wide Memory

Once clients are connected to the broker, the routing and delivery
of messages can proceed. In this phase, the broker is responsible for creating
and managing different types of physical destinations, ensuring a smooth flow
of messages, and using resources efficiently. You can use the broker configuration
properties described under Routing and Delivery Properties to manage these tasks in a way that suits your application’s
needs.

The performance and stability of a broker depend on the system resources
(such as memory) available and how efficiently they are utilized. You can
set configuration properties to prevent the broker from becoming overwhelmed
by incoming messages or running out of memory. These properties function at
three different levels to keep the message service operating as resources
become scarce:

Systemwide message limits apply
collectively to all physical destinations on the system. These include the
maximum number of messages held by a broker (imq.system.max_count)
and the maximum total number of bytes occupied by such messages (imq.system.max_size). If either of these limits is reached, the broker will reject
any new messages until the pending messages fall below the limit. There is
also a limit on the maximum size of an individual message (imq.message.max_size) and a time interval at which expired messages are reclaimed (imq.message.expiration.interval).

Individual destination limits regulate
the flow of messages to a specific physical destination. The configuration
properties controlling these limits are described in Chapter 18, Physical Destination Property Reference. They include limits on the
number and size of messages the destination will hold, the number of message
producers and consumers that can be created for it, and the number of messages
that can be batched together for delivery to the destination.

The destination can be configured to respond to memory
limits by slowing down the delivery of message by message producers, by rejecting
new incoming messages, or by throwing out the oldest or lowest-priority existing
messages. Messages deleted from the destination in this way may optionally
be moved to the dead message queue rather than discarded outright; the broker
property imq.destination.DMQ.truncateBody controls whether
the entire message body is saved in the dead message queue, or only the header
and property data.

As a convenience during application development and testing, you can
configure a message broker to create new physical destinations automatically
whenever a message producer or consumer attempts to access a nonexistent destination.
The broker properties summarized in Table 17–3 parallel the ones just described, but apply to such auto-created
destinations instead of administratively created ones.

System memory thresholds define
levels of memory usage at which the broker takes increasingly serious action
to prevent memory overload. Four such usage levels are defined:

Green: Plenty of memory
is available.

Yellow: Broker memory is
beginning to run low.

Orange: The broker is low
on memory.

Red: The broker is out
of memory.

The memory utilization percentages defining these levels
are specified by the broker properties imq.green.threshold, imq.yellow.threshold , imq.orange.threshold,
and imq.red.threshold , respectively; the default values
are 0% for green, 80% for yellow, 90% for orange, and 98% for red.

As memory usage advances from one level to the next, the broker responds
progressively, first by swapping messages out of active memory into persistent
storage and then by throttling back producers of nonpersistent messages, eventually
stopping the flow of messages into the broker. (Both of these measures degrade
broker performance.) The throttling back of message production is done by
limiting the size of each batch delivered to the number of messages specified
by the properties imq.resourceState.count, where resourceState is green , yellow, orange, or red , respectively.

The triggering of these system memory thresholds is a sign that systemwide
and destination message limits are set too high. Because the memory thresholds
cannot always catch potential memory overloads in time, you should not rely
on them to control memory usage, but rather reconfigure the system-wide and
destination limits to optimize memory resources.