Monitoring the Resource Capping Daemon

Use the rcapstat command to issue reports on lnodes
or projects capped by rcapd. You can set the sampling interval
and specify the report frequency.

interval

Specifies the sampling interval in seconds. The default interval is 5 seconds.

count

Specifies the number of times that the statistics are repeated. By default, rcapstat reports statistics until a termination signal is received
or until the rcapd process exits.

The paging statistics in the first report issued by rcapstat show the activity since the daemon was started. Subsequent
reports reflect the activity since the last report was issued.

The following list defines the column headings in an rcapstat report.

id

The lnode ID of the collection of processes or project ID of the processes
attached to a project.

lnode/project

The collection ID type, which is project or lnode.

nproc

The number
of processes in the collection.

vm

The total
virtual memory size of the collection of processes, including all mapped
files and devices, in kilobytes (K), megabytes (M), or gigabytes (G).

rss

The total
resident set size (RSS) of the collection of processes, in kilobytes (K),
megabytes (M), or gigabytes (G), not accounting for objects that are shared.

cap

The RSS cap
defined for a project or an lnode. See rcapd(1MSRM) for information
about how to specify memory caps.

at

The total
amount of memory that rcapd attempted to page out since
the last rcapstat sample.

avgat

The average
amount of memory that rcapd attempted to page out during
each sample cycle that occurred since the last rcapstat
sample. The rate at which rcapd samples collection RSS
can be set with rcapadm(1MSRM).

pg

The total
amount of memory that rcapd successfully paged out since
the last rcapstat sample. rcapd pages
out the most infrequently used pages.

avgpg

An estimate
of the average amount of memory that rcapd successfully
paged out during each sample cycle that occurred since the last rcapstat sample. The rate at which rcapd samples
process RSS sizes can be set with rcapadm.

Producing Reports With rcapstat

The examples in this section show you how to use rcapstat
to monitor the resource utilization of collections of processes that have
physical memory caps defined.

Using rcapstat to Report Cap and Lnode Information

Caps are defined for two lnodes associated with two users. user1 has a cap of 50 megabytes, and user2 has
a cap of 10 megabytes.

The following command produces reports at 5-second sampling intervals.
A report will be issued five times, once after each sample.

The first three lines of output constitute the first report, which contains
the cap and lnode information for the two lnodes, and paging statistics since rcapd was started. The at and pg
columns are a number greater than zero for user1 and zero
for user2, which indicates that at some time in the daemon's
history, user1 exceeded its cap but user2
did not.

The subsequent reports contain paging statistics since the prior intervals,
but show no significant activity.

Lowering the Cap of an Lnode

The limadm(1MSRM)
command can be used to lower the memory cap of an lnode, which makes the cap
more restrictive. rcapd will enforce the new cap after
the next configuration interval (see rcapadm(1MSRM)). A signal can also be sent, which
causes rcapd to enforce the new cap immediately.

admin# limadm set rss.limit=30M user1
admin# pkill -HUP rcapd

The following command produces reports at 5-second sampling intervals.
A report will be issued five times, once after each sample.

When the cap was lowered to 30 megabytes from 50 megabytes, rcapd responded by attempting to page out the 6-megabyte amount
of resident memory above the cap value. The goal was reached; it was exceeded
by a small amount.

Using rcapstat to Monitor the RSS of a Project

The following command produces reports at 5-second sampling intervals.
A report will be issued five times, once after each sample.

In this example, the project user1 has an RSS in
excess of its physical memory cap. The nonzero values in the pg
column indicate that rcapd is consistently paging out memory
as it attempts to meet the cap by lowering the physical memory utilization
of the project's processes. However, rcapd is unsuccessful,
as indicated by the varying rss values that do not show
a corresponding decrease. This means that the application's resident memory
is being actively used, forcing rcapd to affect the working
set. Under this condition, the system will continue to experience high page
fault rates, and associated I/O, until the working set size is reduced, the
cap is raised, or the application changes its memory access pattern.

Determining the Working Set Size of a Project

The following example is a continuation of the previous example, and
it uses the same project.

By inhibiting cap enforcement, either by raising the cap of a project
or by changing the minimum cap enforcement memory pressure value (see rcapadm(1MSRM)),
the resident set can become the working set. The rss column
might stabilize to show the project working set size, as shown in this example.
This is the minimum cap value that will allow the project's processes to operate
without perpetually incurring page faults.

Reporting Global Memory Caps

You can use the -g option of rcapstat
to report the following:

Global memory utilization cap set by rcapadm(1MSRM)

Current memory utilization as a percentage of all physical
memory installed on the system