IEEE floating point arithmetic in Python

Sometimes a number is not a number. Numeric data types represent real numbers in a computer fairly well most of the time, but sometimes the abstraction leaks. The sum of two numeric types is always a numeric type, but the result might be a special bit pattern that says overflow occurred. Similarly, the ratio of two numeric types is a numeric type, but that type might be a special type that says the result is not a number.

The IEEE 754 standard dictates how floating point numbers work. I’ve talked about IEEE exceptions in C++ before. This post is the Python counterpart. Python’s floating point types are implemented in terms of C’s double type and so the C++ notes describe what’s going on at a low level. However, Python creates a higher level abstraction for floating point numbers. (Python also has arbitrary precision integers, which we will discuss at the end of this post.)

There are two kinds of exceptional floating point values: infinities and NaNs. Infinite values are represented by inf and can be positive or negative. A NaN, not a number, is represented by nan. Let x = 10200. Then x2 will overflow because 10400 is too big to fit inside a C double. (To understand just why, see Anatomy of a floating point number.) In the following code, y will contain a positive infinity.

x = 1e200; y = x*x

If you’re running Python 3.0 and you print y, you’ll see inf. If you’re running an earlier version of Python, the result may depend on your operating system. On Windows, you’ll see 1.#INF but on Linux you’ll see inf. Now keep the previous value of y and run the following code.

z = y; z /= y

Since z = y/y, you might think z should be 1. But since y was infinite, it doesn’t work that way. There’s no meaningful way to assign a numeric value to the ratio of infinite values and so z contains a NaN. (You’d have to know “how they got there” so you could take limits.) So if you print z you’d see nan or 1.#IND depending on your version of Python and your operating system.

The way you test for inf and nan values depends on your version of Python. In Python 3.0, you can use the functions math.isinf and math.isnan respectively. Earlier versions of Python do not have these functions. However, the SciPy library has corresponding functions scipy.isinf and scipy.isnan.

What if you want to deliberately create an inf or a nan? In Python 3.0, you can use float('inf') or float('nan'). In earlier versions of Python you can use scipy.inf and scipy.nan if you have SciPy installed.

The isnan function above looks odd. Why would x != x ever be true? According to the IEEE standard, NaNs don’t equal anything, even each other. (See comments on the function IsFinitehere for more explanation.) The isinf function is really a dirty hack but it works.

To wrap things up, we should talk a little about integers in Python. Although Python floating point numbers are essentially C floating point numbers, Python integers are not C integers. Python integers have arbitrary precision, and so we can sometimes avoid problems with overflow by working with integers. For example, if we had defined x as 10**200 in the example above, x would be an integer and so would y = x*x and y would not overflow; a Python integer can hold 10400 with no problem. We’re OK as long as we keep producing integer results, but we could run into trouble if we do anything that produces a non-integer result. For example,

The situation is a little bit more complex: Python integers are not actually arbitrary precision in Python 2.x (unsure about 3.x), what are arbitrary precision are longs, which sometimes result from integer operations (i.e. 2**400). You can tell the difference either by looking at what’s returned by type() or by seeing an ‘L’ at the end of its textual representation. Replacing all integer arithmetic with arbitrary precision arithmetic in software would likely make a lot of things unbearably slow. The Python ‘int’ type corresponds to C ‘long’, and Python ‘long’ is the arbitrary precision type. NumPy also contains integer types that have a specific bit-width: int8, int16, int32, int64, which correspond to the C type. This can be pretty important if you’re working with large files and don’t want 4 or 8 bytes for every element when 1 or 2 would do.

Also, a neat feature of NumPy is that, with NumPy arrays and floating point scalar types (float32, float64, float128 as well as complex64, complex128 and complex256) is that you can control the effect of hitting division by zero, overflow, underflow or invalid ops (things that produce NaN); each of these errors can either be ignored, produce a warning, raise an exception, or call a user-specified function. Have a look at the documentation for numpy.seterr for the specifics.

@David,
I think they have further blurred things in Python 3: ints are arbitrary precision, but, when possible, fixed precision calculations are done. They are trying to have a seemless transition between what were ints and what were longs in Python 2.x days.