It is reasonable to choose a search vector that will be a
descent direction; that is, a direction leading to function reduction.
A descent direction is defined as one along which the directional
derivative is negative:

When we write the approximation

we see that the negativity of the right-hand side guarantees that a lower
function value can be found along for a sufficiently
small .

Different methods are distinguished by their choice of search
directions.
Algorithms can be classified into nonderivative, gradient,
and second-derivative (Newton) methods depending on the technique
used to determine
in Algorithm 2.1. These classes will be discussed in turn
beginning in Section 2.3.