NAME

bup-memtest - test bup memory usage statistics

SYNOPSIS

bup memtest [options...]

DESCRIPTION

bupmemtest opens the list of pack indexes in your bup repository, then
searches the list for a series of nonexistent objects, printing memory
usage statistics after each cycle.
Because of the way Unix systems work, the output will usually show a
large (and unchanging) value in the VmSize column, because mapping the
index files in the first place takes a certain amount of virtual
address space. However, this virtual memory usage is entirely virtual;
it doesn’t take any of your RAM. Over time, bup uses parts of the
indexes, which need to be loaded from disk, and this is what causes an
increase in the VmRSS column.

OPTIONS

-n, —number=number
set the number of objects to search for during each cycle (ie.
before printing a line of output)
-c, —cycles=cycles
set the number of cycles (ie. the number of lines of output
after the first). The first line of output is always 0 (ie.
the baseline before searching for any objects).
—ignore-midx
ignore any .midx files created by bupmidx. This allows you to
compare memory performance with and without using midx.

DISCUSSION

When optimizing bup indexing, the first goal is to keep the VmRSS
reasonably low. However, it might eventually be necessary to swap in
all the indexes, simply because you’re searching for a lot of objects,
and this will cause your RSS to grow as large as VmSize eventually.
The key word here is eventually. As long as VmRSS grows reasonably
slowly, the amount of disk activity caused by accessing pack indexes is
reasonably small. If it grows quickly, bup will probably spend most of
its time swapping index data from disk instead of actually running your
backup, so backups will run very slowly.
The purpose of bupmemtest is to give you an idea of how fast your
memory usage is growing, and to help in optimizing bup for better
memory use. If you have memory problems you might be asked to send the
output of bupmemtest to help diagnose the problems.
Tip: try using bupmidx-a or bupmidx-f to see if it helps reduce
your memory usage.
Trivia: index memory usage in bup (or git) is only really a problem
when adding a large number of previously unseen objects. This is
because for each object, we need to absolutely confirm that it isn’t
already in the database, which requires us to search through all the
existing pack indexes to ensure that none of them contain the object in
question. In the more obvious case of searching for objects that do
exist, the objects being searched for are typically related in some
way, which means they probably all exist in a small number of
packfiles, so memory usage will be constrained to just those packfile
indexes.
Since git users typically don’t add a lot of files in a single run, git
doesn’t really need a program like bupmidx. bup, on the other hand,
spends most of its time backing up files it hasn’t seen before, so its
memory usage patterns are different.