Background to the Problems

We wish to construct a set of nn vectors q1,q2, … ,qnq1,q2,…,qn such that:

– the vectors {qi}{qi} form an orthonormal set; that is,
qiTqj = 0qiTqj=0 for i ≠ ji≠j, and ‖qi‖2 = 1‖qi‖2=1;

– each aiai is linearly dependent on the set {qi}{qi}.

Gram–Schmidt Orthogonalization

The classical Gram–Schmidt orthogonalization process is described in many textbooks; see for example Chapter 5 of Golub and Van Loan (1996).

It constructs the orthonormal set progressively. Suppose it has computed orthonormal vectors q1,q2, … ,qkq1,q2,…,qk which orthogonalise the first kk vectors a1,a2, … ,aka1,a2,…,ak. It then uses ak + 1ak+1 to compute qk + 1qk+1 as follows:

zk + 1

=

k

ak + 1 −

∑

(qiTak + 1)qi

i = 1

qk + 1

=

zk + 1 / ‖zk + 1‖2.

zk+1=ak+1-∑i=1k(qiTak+1)qiqk+1=zk+1/‖zk+1‖2.

In finite precision computation, this process can result in a set of vectors {qi}{qi} which are far from being orthogonal. This is caused by |zk + 1||zk+1| being small compared with |ak + 1||ak+1|. If this situation is detected, it can be remedied by reorthogonalising the computed qk + 1qk+1 against q1,q2, … ,qkq1,q2,…,qk, that is, repeating the process with the computed qk + 1qk+1 instead of ak + 1ak+1. See Danial et al. (1976).

Let AA be the mm by nn matrix whose columns are the nn vectors to be orthogonalised. The QRQR factorization gives

A = QR

A=QR

where RR is an nn by nn upper triangular matrix and QQ is an mm by nn matrix, whose columns are the required orthonormal set.

Moreover, for any kk such that 1 ≤ k ≤ n1≤k≤n, the first kk columns of QQ are an orthonormal basis for the first kk columns of AA.

Householder's method requires twice as much work as the Gram–Schmidt method, provided that no re-orthogonalization is required in the latter. However, it has satisfactory numerical properties and yields vectors which are close to orthogonality even when the original vectors aiai are close to being linearly dependent.

Recommendations on Choice and Use of Available Functions

The single function in this chapter, nag_orthog_real_gram_schmidt (f05aa), uses the Gram–Schmidt method, with re-orthogonalization to ensure that the computed vectors are close to being exactly orthogonal. This method is only available for real vectors.