Context Navigation

Memory utilities

This section describes functions that could be useful in tracking down
memory problems.

Heap objects

[Function]map-heap-objectsfunction &key area

Calls function on all objects in the heap. If area is specified,
it looks only in that area of the heap. area can be one of
:readonly, :static, :managed-static, or :dynamic, or a list of
areas to look in multiple areas. If area is nil or not specified,
looks in all the areas.

function should try to avoid allocating memory. While a little bit
of consing may be ok, excessive consing leading to garbage collection
will likely cause problems.

[Function]find-referencersobject &optional area

Returns a list of all objects in the heap that contain a direct
reference to object. area is as for map-heap-objects.

[Function]transitive-referencerslist-of-objects &key area (verbose t)

Finds all transitive referencers to any of the objects in
list-of-objects, i.e. all objects have a path to an object in
list-of-objects. Returns a hash table with the referencers
as keys and t as value. area may be specified as in find-referencers
to limit the referencers to particular areas of the heap. If verbose
is true, the default, prints short progress information to *standard-output*.

heap-utilization walks the lisp heap, collects information about the objects stored on the heap, and prints a report of the results to stream. It shows the number of objects of each type, the logical size (i.e. the size of the data part of the object) and the physical size (the logical size plus any header and padding bytes).

If gc-first is true (the default), does a full gc before scanning the heap.

If classes is true, classifies objects by class rather than just basic type.

area can be used to restrict the walk to one memory area or a list of areas. Some possible values are :dynamic, :static, :managed-static, :readonly. By default, all areas (including stacks) are examined.

sort can be one of :count, :logical-size, or :physical-size to sort output by count or size.

unit can be one of :kb:mb or :gb to show sizes in units other than bytes.

If start is non-nil, it should be an object returned by get-allocation-sentinel; only
objects at higher address are scanned (i.e. roughly, only objects allocated after it).

If threshold is non-nil, it should be a number between 0 and 1. All types whose share of the heap is less than threshold will be lumped together in an "All Others" line rather than being listed individually.

Memory maps

[Function]parse-proc-maps&optional pid

This function is only available on Linux. It returns a list of
currently mapped memory sections in the unix process with id pid,
which defaults to the current process. Each memory section is
represented by a list of 6 elements:

(address perms name section-size unmodified-size modified-size)

where address is the starting address of the section, name is
usually the name of the file the section is mapped from, and perms
is a 4-character string representing permissions:

first character is "r" if readable, "-" otherwise

second character is "w" if writable, "-" otherwise

third character is "x" if executable, "-" otherwise

fourth character is "s" if sharable, "p" if private (copy on write)

section-size is the total amount of address space assigned to the section,
unmodified-size is the total size of unmodified (clean) pages in the
section, and modified-size is the total size of modified (dirty) pages.
(unmodified-size and modified-size may be nil if the operating system
doesn't provide this information).

Note that unmodified pages mapped from a file are shared among all
processes using them, regardless of the sharable/private permission
setting. The permission setting only affects what happens when a page
is modified: in sharable sections, any changes are visible to all
processes, whereas private sections copy the page on write.

A permission string of "---p" indicates a range of reserved address
space using no system resources.

[Function]proc-maps-diffmap1 map2

Compares two lists as returned by parse-proc-maps and returns a list
of all changes, in the form of (map1-entry map2-entry) where
map1-entry and map2-entry represent the same section (the same
start address, permissions and filename) with different size values.
map1-entry can be nil indicating a new entry added in map2,
and map2-entry can be nil indicating an entry removed in map2.

Examining core files

CCL has a suite of functions for examining core dump files of CCL
applications. Currently these functions are only available for x8664
Linux (porting them to another architecture would be a fairly
straightfoward learn-CCL-internals project).

A core dump file contains a copy of the Lisp heap of a process. You
can open a core file with open-core, and get a summary of the heap
contents with core-heap-utilization or idom-heap-utilization.
Depending on how the core dump
was obtained, you might also be able to examine the stack with
core-print-call-history. If this is not enough, there are
many functions available that let you manually examine individual objects,
map over symbols, etc. Core file objects are represented as
integers, encoding the address and the tag of the object, and there
are functions available to decode their types and contents.

[Function]open-corepathname &key image (method :mmap)

Opens a core dump file contained in pathname and establishes it as
the current core file used by all the subsequent functions. If
image is specified, it should be the name of the CCL image file
corresponding to the core file. This is used to get information
(e.g. builtin symbol pnames) from read-only sections which are not
present in the core file. method determines how the core file is
accessed, it can be :mmap or :stream.

Examines the lisp heap stored in the core file and reports the types of objects in it. The arguments are as for heap-utilization above, except area can also be :tenured to restrict the computation to just the tenured part of the dynamic heap.

This function analyzes the core file heap based on dominance. It takes a while
to compute, but often gives more meaningful results than the regular heap
utilization report, especially when looking at memory leaks.

An object A is said to dominate another object B if all paths from gc roots
to B go through A. A is said to immediately dominate B if A dominates B and
every other object that dominates B also dominates A.

Since an object can only have one immediate dominator, all objects that are not
dominators themselves can be partitioned into disjoint sets, each consisting of
all objects with the same immediate dominator.

If B is in the partition owned by A, i.e. B is not a dominator of anything and
is immediately dominated by A, then for all intents and purposes, B is just a
part of A: B will be garbage collected whenever A is garbage collected, and there
are no pointers directly to B from outside the partition. Nothing about the behavior
of the heap would change if B was permanently appended to A instead of being a
separate object.

idom-heap-utilization attributes the total size of all the partition members
to the immediate dominator. I.e. it computes what core-heap-utilization would report
if the heap was flattened by attaching all objects to their immediate dominators,
and ignoring all types except the type of the immediate dominator. This
tends to emphasize the types of objects that are really responsible for the memory usage.

[Function]core-print-call-history&key (stream t) origin detailed-p

Attempts to find and describe the stack stored in the core file. The arguments are as for print-call-history.

[Function]map-core-areasfunction &key area

Calls function on all objects in the core heap. function receives one argument, an integer representing a lisp object in the core file. area can be used to restrict to certain areas in the heap. It can
the name of an area (:readonly, :static, :managed-static or :dynamic), or a list of area names, or the special name :tenured to only consider the tenured section.

[Function]core-printcore-object &optional (stream t) depth

Interpret core-object as an object in the core file, and output its printed
representation to stream stream. depth controls how deeply to descend.

[Function]core-listpcore-object

True if core-object is a list

[Function]core-nullpcore-object

True if core-object is null.

[Function]core-conspcore-object

True if core-object is a cons.

[Function]core-carcore-object

core-object must be a cons. Returns its car.

[Function]core-cdrcore-object

core-object must be a cons. Returns its cdr.

[Function]core-symbolpcore-object

True if core-object is a symbol

[Function]core-functionpcore-object

True if core-object is a function

[Function]core-uvector-pcore-object

True if core-object is a uvector.

[Function]core-uvtypecore-object

core-object must be a uvector. Returns a symbol representing core-object's internal vector type (internal
types are the types shown in heap-utilization, e.g. ccl::simple-unsigned-word-vector).

[Macro]core-uvtypepcore-objecttype

Returns true if core-object is a uvector of internal vector type type (internal
types are the types shown in heap-utilization, e.g. ccl::simple-unsigned-word-vector).

[Function]core-uvsizecore-object

core-object must be a uvector. Returns its size (number of elements).

[Function]core-uvrefcore-objectindex

core-object must be a uvector. Returns its indexth element.

[Function]core-object-typecode-typecore-object

Returns a symbol representing the internal type of core-object. The
same as core-uvtype if core-object is a uvector.

[Function]core-object-type-keycore-object

Returns either a symbol (for built-in types) or a core object representing
a symbol that names the type of core object. In either case, the value
returned is suitable for use in an eql hash table; use core-type-string
to get a consistent printable representation.

[Function]core-type-stringtype-key

Given a value returned by core-object-type-key, returns the type
name as a string.

[Function]copy-from-corecore-object &key (depth 1)

Attempts to create a lisp copy of the core object core-object, for
example if core-object is a string in the core heap, returns a string
with the same characters. depth limits how far to descend. At depth 0,
only immediate values such as characters and fixnums are copied.

[Function]core-listcore-object

core-object should be a list. Copies the top-level list, returning a list of core objects.

[Function]core-find-packagename &key error

Returns the core object representing a package named name in the core heap. name
should be a string or symbol, or a core object which is a string or symbol.
If there is no such package in the core heap, returns nil if error is null,
else signals an error.

[Function]core-keyword-package

Returns the address of the keyword package in the core heap. Equivalent
to but more efficient than (core-find-package :keyword).

[Function]core-find-symbolname &optional package

Returns the core object representing the symbol named name in
package package in the core heap. name can be a symbol, string,
or a core object representing a symbol or string. package can be a
symbol, string, package, or core object representing a symbol, string or
package. If name is a string or a string core object, then
package is required, otherwise package defaults to the package
of name.

[Function]core-package-namescore-object

Returns, as a list of strings, the name and nicknames of the core package core-object.

[Function]core-package-namecore-object

Returns, as a string, the name of the core package core-object. The same as (car (core-package-names core-object))
but without consing up the whole list.

[Function]core-map-symbolsfunction

Map function over all symbols in the core heap. function will be called with one argument, a
symbol core object . function will be invoked once for each symbol in the heap, including
keywords and uninterned symbols.

[Function]core-symbol-namecore-object

Returns, as a string, the name of the core symbol core-object.

[Function]core-symbol-valuecore-object

Returns the core object that is the global value of the core symbol core-object.

[Function]core-symbol-packagecore-object

Returns the core object that is the package of the core symbol core-object.

[Function]core-symbol-plistcore-object

Returns the core object that is the plist of the core symbol core-object.

[Function]core-gethashcore-objectcore-hash

Looks up core-object in the core hash table object core-hash. Returns the value,
a core object, if found, else nil.

[Function]core-hash-table-countcore-hash

Returns the hash-table-count of the core hash table object core-hash. Signals an
error if the table was in the middle of being rehashed.

[Function]core-lfun-namecore-function

Returns the core object that is the contents of the name slot of the core function object core-function.

[Function]core-lfun-bitscore-function

Returns the lfun bits of the core function object core-function.

[Function]core-nth-immediatecore-functionindex

Returns the core object that is the contents of the indexth immediate slot of the core function object core-function.

[Function]core-find-classname

name must be a symbol or a symbol core object. Returns a class core object named name, or nil if not found.

[Function]core-instance-classcore-instance

core-instance must be a CLOS instance (i.e. core-object-typecode-type'ccl::instance). Returns the class core object which is the class of the instance.

[Function]core-instance-pcore-objectcore-class

Returns true if core-object is a instance of the core class core-class.

[Function]core-string=core-string string

Returns true if the core string core-string and string are equal. Equivalent to but more efficient than
(string= (copy-from-core core-string) string).

[Function]core-all-processes

Returns a list of core objects representing all processes in the core heap.

[Function]core-process-namecore-process

Returns, as a string, the name of the process core object core-process.

[Function]core-find-process-for-idlwp

Returns a process core object whose native thread id is lwp, or nil if not found.

[Function]core-qaddress &optional offset

Return, as an integer, the pointer-sized contents of the core heap at address (+ address offset).

[Function]core-laddress &optional offset

Return the 32-bit contents of the core heap at address (+ address offset)

[Function]core-waddress &optional offset

Return the 16-bit contents of the core heap at address (+ address offset)

[Function]core-baddress &optional offset

Return the 8-bit contents of the core heap at address (+ address offset)