Managing Off-Heap Memory

GemFire can be configured to store region values in off-heap memory, which is memory within the JVM that is not subject to Java garbage collection.

Garbage collection (GC) within a JVM can prove to be a performance impediment. A server cannot exert control over when garbage collection within the JVM heap memory takes place, and the server has little control over the triggers for invocation. Off-heap memory offloads values to a storage area that is not subject to Java GC. By taking advantage of off-heap storage, an application can reduce the amount of heap storage that is subject to GC overhead.

Off-heap memory works in conjunction with the heap, it does not replace it. The keys are stored in heap memory space. GemFire’s own memory manager handles the off-heap memory with better performance than the Java garbage collector would for certain sets of region data.

The resource manager monitors the contents of off-heap memory and invokes memory management operations in accordance with two thresholds similar to those used for monitoring the JVM heap: eviction-off-heap-percentage and critical-off-heap-percentage.

On-heap and Off-heap Objects

The following objects are always stored in the JVM heap:

Region metadata

Entry metadata

Keys

Indexes

Subscription queue elements

The following objects can be stored in off-heap memory:

Values - maximum value size is 2GB

Reference counts

List of free memory blocks

WAN queue elements

Note:
Do not use functional range indexes with off-heap data, as they are not supported. An attempt to do so generates an exception.

Off-heap Recommendations

Off-heap storage is best suited to data patterns where:

Stored values are relatively uniform in size

Stored values are mostly less than 128K in size

The usage patterns involve cycles of many creates followed by destroys or clear

The values do not need to be frequently deserialized

Many of the values are long-lived reference data

Be aware that GemFire has to perform extra work to access the data stored in off-heap memory since it is stored in serialized form. This extra work may cause some use cases to run slower in an off-heap configuration, even though they use less memory and avoid garbage collection overhead. However, even with the extra deserialization, off-heap storage may give you the best performance. Features that may increase overhead include

frequent updates

stored values of widely varying sizes

deltas

queries

Implementation Details

The off-heap memory manager is efficient at handling region data values that are all the same size or are of fixed sizes. With fixed and same-sized data values allocated within the off-heap memory, freed chunks can often be re-used, and there is little or no need to devote cycles to defragmentation.

Region values that are less than or equal to eight bytes in size will not reside in off-heap memory, even if the region is configured to use off-heap memory. These very small size region values reside in the JVM heap in place of a reference to an off-heap location. This performance enhancement saves space and load time.

Controlling Off-heap Use with the Resource Manager

The GemFire resource manager controls off-heap memory by means of two thresholds, in much the same way as it does JVM heap memory. See Using the GemFire Resource Manager. The resource manager prevents the cache from consuming too much off-heap memory by evicting old data. If the off-heap memory manager is unable to keep up, the resource manager refuses additions to the cache until the off-heap memory manager has freed an adequate amount of memory.

The resource manager has two threshold settings, each expressed as a percentage of the total off-heap memory. Both are disabled by default.

Eviction Threshold. The percentage of off-heap memory at which eviction should begin. Evictions continue until the resource manager determines that off-heap memory use is again below the eviction threshold. Set the eviction threshold with the eviction-off-heap-percentage region attribute. The resource manager enforces an eviction threshold only on regions with the HEAP_LRU characteristic. If critical threshold is non-zero, the default eviction threshold is 5% below the critical threshold. If critical threshold is zero, the default eviction threshold is 80% of total off-heap memory.

The resource manager enforces eviction thresholds only on regions whose LRU eviction policies are based on heap percentage. Regions whose eviction policies based on entry count or memory size use other mechanisms to manage evictions. See Eviction for more detail regarding eviction policies.

Critical Threshold. The percentage of off-heap memory at which the cache is at risk of becoming inoperable. When cache use exceeds the critical threshold, all activity that might add data to the cache is refused. Any operation that would increase consumption of off-heap memory throws a LowMemoryException instead of completing its operation. Set the critical threshold with the critical-off-heap-percentage region attribute.

Critical threshold is enforced on all regions, regardless of LRU eviction policy, though it can be set to zero to disable its effect.

Specifying Off-heap Memory

To use off-heap memory, specify the following options when setting up servers and regions:

From gfsh, start each server that will support off-heap memory with a non-zero off-heap-memory-size value, specified in megabytes (m) or gigabytes (g). If you plan to use the resource manager, specify critical threshold, eviction threshold, or (in most cases) both.

Mark regions whose entry values should be stored off-heap by setting the off-heap region attribute to true Configure other region attributes uniformly for all members that host data for the same region. .

critical-off-heap-percentage=value
Specifies the percentage of off-heap memory at or above which the cache is considered in danger of becoming inoperable due to out of memory exceptions. See <resource-manager> for details.

eviction-off-heap-percentage=value
Specifies the percentage of off-heap memory at or above which eviction should begin. Can be set for any region, but actively operates only in regions configured for HEAP_LRU eviction. See <resource-manager> for details.

Tuning Off-heap Memory Usage

GemFire collects statistics on off-heap memory usage which you can view with the gfsh show metrics command. See Off-Heap (OffHeapMemoryStats) for a description of available off-heap statistics.

Off-heap memory is optimized, by default, for storing values of 128 KB in size. This figure is known as the “maximum optimized stored value size,” which we will denote here by maxOptStoredValSize. If your data typically runs larger, you can enhance performance by increasing the OFF_HEAP_FREE_LIST_COUNT system parameter to a number larger than maxOptStoredValSize/8, where maxOptStoredValSize is expressed in KB (1024 bytes). So, the default values correspond to:

To optimize for a maximum optimized stored value size that is twice the default, or 256 KB, the free list count should be doubled:

-Dgemfire.OFF_HEAP_FREE_LIST_COUNT=32768

During the tuning process, you can toggle the off-heap region attribute on and off, leaving other off-heap settings and parameters in place, in order to compare your application’s on-heap and off-heap performance.