As I understand, floating point has the truncation problem if there are a lot of decimal places. The sum will be different if it adds from the first element to the last element in the array and from the last one to the first one. I tried to find the sample data, unfortunately, all the data that I have used have failed. Any suggestion is greatly appreciated.

You don't need an array. Just put sum += 1.0/3; in a loop and cycle through a half a million times or so...the sum should be different than 500000/3.

yup, You can prove the floating point has a trucation issue. But I need to prove the array will have different sum if it adds the first one to the last one and vice versa. I just want to find out the data that should be contained in this array.

Well if you're trying to compare two values by going through the loop two different ways, shouldn't you be storing each of the two values in a different variable? That way you can compare them?

The issue isn't really with "truncation" per say, anyway. It's with the fact that floating point numbers, due to the way they're stored/work, can't accurately store every numeric possibility in the number of bits they're given. Perhaps you consider that truncation, but I don't. Anyway...

As I understand, floating point has the truncation problem if there are a lot of decimal places. The sum will be different if it adds from the first element to the last element in the array and from the last one to the first one.

Eric

Your understanding is incomplete.

Floating point numbers are limited in the number of significant digits that they can use to represent a number (not the number of decimal places). The following discussion is based on 32-bit floats used in Borland C, Microsoft Visual C++ and Gnu gcc. If you don't get the same results, you can investigate further.

Now, if a number has more than 7 or 8 significant digits, it will not be representable exactly as a float.

for example if you have, say

Code:

float x;
x = 1000000000000.0; /* 1.0e12 */
printf("x = %f\n", x);

you will see something like

x = 999999995904.000000

(note that this is correct to 7 significant digits,as can be seen if you do the following:

Code:

printf("x (rounded to 7 significant digits) = %.6e\n", x);

Then you see

x (rounded to 7 significant digits) = 1.000000e+12

Now what does this have to do with correctness as a consequence of the order in which numbers are added?

Well if all of the numbers and all of the sums are exactly representable in the machine, there is no difference.

I am going to give an extreme example where we have a large number and a lot of small numbers to add.

Now, we can't expect to get exact results if the total is has more significant digits than is exactly representable. The question might be, "Well, if we can't get an exact result, what's the best we can do?"

Try the following. Look at what's really happening. You will find that adding the small numbers first gives a result that is accurate to seven significant digits. Adding the small numbers last does not.
(I have included the same procedure with a double, to show the exact answer.)

The moral: If you have an array of floats, you may get better results if you sort the array and add the small numbers first.

The issue isn't really with "truncation" per say, anyway. It's with the fact that floating point numbers, due to the way they're stored/work, can't accurately store every numeric possibility in the number of bits they're given. Perhaps you consider that truncation, but I don't. Anyway...

Quzah.

But that's precisely what truncation error is, when applied to representation of a number with a finite number of digits:

truncation error: In the representation of a number, the error introduced when one or more digits are dropped.

The term "truncation error" when applied to an infinite series is used to designate the error introduced by stopping the summation after a finite number of terms (thus truncating the series).

In numerical analysis, they usually use "truncation error" to mean the error introduced by truncating an infinite series, and "roundoff error" to indicate errors introduced by the inexact representation of arithmetic values due to the limitations of a finite number of digits.