According to the formula
(
Orthogonality of residues
2
)
We
conclude
Therefore, when we conduct
-th
step of Gram-Schmidt
-orthogonalization:
only one term is non-zero in the
sum:
We would like to remove matrix multiplications from the above relationship. We
set
in the equation
:
and apply the operation
.
Then
According to
,
hence
According to the summary (
Conjugate
directions
),
We combine the last two
relationships:
thus
We substitute it into
:
By
-orthogonality
of
and
we also have
thus
We collect the results.

Algorithm

(Conjugate gradients) Start from any
.
Set
For
do
To avoid accumulation of round off errors, occasionally restart with
using last
as
.
Violation of
-orthogonality
of
is the criteria of error accumulation. Use condition of the type
to stop.