37.1 Overview

The problem of multidimensional minimization requires finding a point
x such that the scalar function,

f(x_1, …, x_n)

takes a value which is lower than at any neighboring point. For smooth
functions the gradient g = \nabla f vanishes at the minimum. In
general there are no bracketing methods available for the
minimization of n-dimensional functions. The algorithms
proceed from an initial guess using a search algorithm which attempts
to move in a downhill direction.

Algorithms making use of the gradient of the function perform a
one-dimensional line minimisation along this direction until the lowest
point is found to a suitable tolerance. The search direction is then
updated with local information from the function and its derivatives,
and the whole process repeated until the true n-dimensional
minimum is found.

Algorithms which do not require the gradient of the function use
different strategies. For example, the Nelder-Mead Simplex algorithm
maintains n+1 trial parameter vectors as the vertices of a
n-dimensional simplex. On each iteration it tries to improve
the worst vertex of the simplex by geometrical transformations. The
iterations are continued until the overall size of the simplex has
decreased sufficiently.

Both types of algorithms use a standard framework. The user provides a
high-level driver for the algorithms, and the library provides the
individual functions necessary for each of the steps. There are three
main phases of the iteration. The steps are,

initialize minimizer state, s, for algorithm T

update s using the iteration T

test s for convergence, and repeat iteration if necessary

Each iteration step consists either of an improvement to the
line-minimisation in the current direction or an update to the search
direction itself. The state for the minimizers is held in a
gsl_multimin_fdfminimizer struct or a
gsl_multimin_fminimizer struct.