MATH 240

Using a calculator

Solving a system of linear equations on a computer or calculator
is surprisingly difficult.
Inverting a matrix or performing certain other matrix operations
can lead to numerical errors that require a lot of theory to understand.
Our department has an entire undergraduate course,
Math 434, Numerical Linear Algebra,
that covers numerical techniques in linear algebra.
We do not have enough time to discuss numerical algorithms in MATH 240,
and if you do not know the relevant theory,
you must be very cautious and skeptical about the answer
when you just press a button on your calculator.

Solving a system of equations

If you have been trained to believe that
a calculator will always give you the correct answer,
you may be in for a shock
if you try to solve a system of equations
by just plugging the coefficients into your calculator and pressing a button.
Even a system of two equations in two unknowns
can present problems for the program used by your calculator.
Here is one example.

We will try to solve this system of equations.

416785x + 415872y = 1
415872x + 414961y = 0

The coefficients in the problem have six significant digits.
Since the TI-85 calculator stores more than twice that many
significant digits internally,
solving the system would seem to present no problem.
Using the equation solver on the TI-85 gives the following "answer".

You can "check" the calculator answer by substituting it
back into the system.
Be sure to use the values stored in the calculator.
Then, to the limits of the calculator's accuracy,
everything checks out,
and, in fact, both solutions appear to be correct.

To find the exact solution,
we can use elementary row operations on the system of equations.
The first goal is to reduce the size of the numbers,
but retain integer values.
Here are the results.

This is not a problem unique to the TI-85.
Using MATLAB on a SUN workstation also gives
an answer that differs substantially from the correct one.
The difficulties are inherent in the problem.
To look at this problem from the geometric point of view,
we could compute the slope of each line.
To 12 decimal point accuracy, we get

416785 ÷ 415872 = -1.00219538704
415872 ÷ 414961 = -1.00219538704

To most calculators, the lines appear to be parallel,
and so there should be no solution at all!
Because the angle between the two lines is very small,
a small change in the coefficients caused by roundoff error
can make a very large difference in the solution.
From a geometric point of view,
shifting the two lines just a little bit
can make a bit difference in the point of intersection.

Inverting a matrix

In the previous problem,
we were able to find an exact inverse for the coefficient matrix.

To illustrate some of the inherent difficulties
in doing Gaussian elimination using floating point arithmetic,
we will look at the row reduction of a standard "badly behaved" matrix.

The matrix given below is called a Hilbert matrix.
It is a well-known example of a matrix that causes problems
for numerical algorithms.
To help understand the problems,
we will do an exact row reduction,
compared to a row reduction done using floating point arithmetic.
To see how the error in the approximations can be compounded,
we will use a highly simplified example,
in which the floating point arithmetic
is carried out with accuracy to only three significant digits.

In the original matrix, labeled (1),
some of the decimal entries are already inaccurate.
In matrix (3), the three digit computation that produces the
3rd entry in row 4 is this:

.837×10-1 - (.904)(.830×10-1)
= .837×10-1 - .750×10-1
= .870×10-2

Comparing this to the correct value of
1 ÷ 120 = .00833 (to 3 digits)
shows that it has only one correct digit.

In matrix (5), the last entry of row 4 is computed as follows:

.837×10-2 - (.690)(.127×10-1)
= .837×10-2 - .876×10-2
= .006×10-2

Because we have to subtract two values that are nearly equal,
the answer has even less accuracy.
The cumulative errors in reducing just 3 rows
produce a value of -.0000600 instead of
(-1) ÷ 4200 = -.000238 (to 3 digits).

The method for defining a Hilbert matrix can be extended to larger sizes,
and the 10 by 10 Hilbert matrix presents substantial problems
for even a very sophisticated numerical algorithm.
You can experiment on your calculator,
by inverting the Hilbert matrices of larger and larger sizes.

REFERENCES

Yves Nievergelt,
Numerical Linear Algebra on the HP-28 or How to Lie With Supercalculators,
American Mathematical Monthly, (1991), 539-544