Hessian Multiply Function for Lower Memory

The fminconinterior-point and trust-region-reflective algorithms,
and the fminunctrust-region algorithm
can solve problems where the Hessian is dense but structured. For
these problems, fmincon and fminunc do
not compute H*Y with the Hessian H directly,
because forming H would be memory-intensive. Instead,
you must provide fmincon or fminunc with
a function that, given a matrix Y and information
about H, computes W = H*Y.

In this example, the objective function is nonlinear and linear
equalities exist so fmincon is used. The description
applies to the trust-region reflective algorithm; the fminunctrust-region algorithm
is similar. For the interior-point algorithm, see the HessianMultiplyFcn option
in Hessian Multiply Function. The
objective function has the structure

f(x)=f^(x)−12xTVVTx,

where V is a 1000-by-2 matrix. The Hessian
of f is dense, but the Hessian of f^ is sparse. If the Hessian of is H^, then H,
the Hessian of f, is

H=H^−VVT.

To avoid excessive memory usage that could happen by working
with H directly, the example provides a Hessian
multiply function, hmfleq1. This function, when
passed a matrix Y, uses sparse matrices Hinfo,
which corresponds to , and V to compute the Hessian
matrix product

W = H*Y = (Hinfo - V*V')*Y

In this example, the Hessian multiply function needs and V to
compute the Hessian matrix product. V is a constant,
so you can capture V in a function handle to an
anonymous function.

However, is not a constant and must be computed at the
current x. You can do this by computing in the objective
function and returning as Hinfo in the third output
argument. By using optimoptions to set the 'Hessian' options
to 'on', fmincon knows to get
the Hinfo value from the objective function and
pass it to the Hessian multiply function hmfleq1.

Step 1: Write a file brownvv.m that computes the objective function, the gradient, and the sparse part of the Hessian.

The example passes brownvv to fmincon as
the objective function. The brownvv.m file
is long and is not included here. You can view the code with the command

type brownvv

Because brownvv computes the gradient as
well as the objective function, the example (Step
3) uses optimoptions to
set the SpecifyObjectiveGradient option to true.

Step 2: Write a function to compute Hessian-matrix products for H given a matrix Y.

Now, define a function hmfleq1 that uses Hinfo,
which is computed in brownvv, and V,
which you can capture in a function handle to an anonymous function,
to compute the Hessian matrix product W where W = H*Y = (Hinfo - V*V')*Y. This function must
have the form

W = hmfleq1(Hinfo,Y)

The first argument must be the same as the third argument returned
by the objective function brownvv. The second argument
to the Hessian multiply function is the matrix Y (of W
= H*Y).

Because fmincon expects the second argument Y to
be used to form the Hessian matrix product, Y is
always a matrix with n rows where n is
the number of dimensions in the problem. The number of columns in Y can
vary. Finally, you can use a function handle to an anonymous function
to capture V, so V can be the third argument to 'hmfleqq'.

Note

The function hmfleq1 is available in the optimdemos folder
as hmfleq1.m.

Step 3: Call a nonlinear minimization routine with a starting point and linear equality constraints.

Load the problem parameter, V, and the sparse
equality constraint matrices, Aeq and beq,
from fleq1.mat, which is available in the optimdemos folder.
Use optimoptions to set the SpecifyObjectiveGradient option
to true and to set the HessianMultiplyFcn option
to a function handle that points to hmfleq1. Call fmincon with
objective function brownvv and with V as
an additional parameter:

Preconditioning

In this example, fmincon cannot use H to
compute a preconditioner because H only exists
implicitly. Instead of H, fmincon uses Hinfo,
the third argument returned by brownvv, to compute
a preconditioner. Hinfo is a good choice because
it is the same size as H and approximates H to
some degree. If Hinfo were not the same size as H, fmincon would
compute a preconditioner based on some diagonal scaling matrices determined
from the algorithm. Typically, this would not perform as well.

MATLAB Command

You clicked a link that corresponds to this MATLAB command:

Run the command by entering it in the MATLAB Command Window.
Web browsers do not support MATLAB commands.

Was this topic helpful?

Select Your Country

Choose your country to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .