An integer overflow, or integer wrapping, is a potential problem in a program based upon the fact that the value that can be held in a numeric datatype is limited by the data type’s size in bytes. ANSI C uses the following minimum sizes:

data type

size (bytes)

char

1

short

2

int

2

long

4

In practice, many compilers use a 4-byte int. It also should be noted that the actual ranges for the data types depend on whether or not they are signed. for instance, a signed 2-byte short may be between -32767 and 32767, while an unsigned short may be between 0 and 65535. See your [include]/limits.h file for specific numbers for your compiler.

Why should you care? If you try to put a value into a data type that is too small to hold it, the high-order bits are dropped, and only the low-order bits are stored. Another way of saying that is that modulo-arithmetic is performed on the value before storing it to make sure it fits within the datatype. Taking our unsigned short example:

Limit:

65535 or 1111 1111 1111 1111

Too big:

65536 or 1 0000 0000 0000 0000

What’s stored:

0 or 0000 0000 0000 0000

As the above makes evident, that result is because the high-order (or left-most) bit of the value that’s too big is dropped. Or you could say that what’s stored is the result of

stored = value % (limit + 1)or65536 % (65535 + 1) = 0

In signed datatypes, the result is a little different and results in some seemingly weird behaviour:

Positive limit:

32767 or 0111 1111 1111 1111

Too big:

32768 or 1000 0000 0000 0000

What’s stored:

-32768

Why’s that? It’s because of “2’s compliment,” which is how negative numbers are represented in binary. To make a long story short, the first half of the range (0 thru 0111 1111 1111 1111) is used for positive numbers in order of least to greatest. the second half of the range is then used for negative numbers in order of least to greatest. so the negative range for a signed 2-byte short is -32768 thru -1, in that order.

You’re still asking why this matters, aren’t you? Suppose memory is being allocated based on an unsigned integer data type’s value. If that value has wrapped around, it may be that far too little memory will be made available. Or if a comparison is being made between a signed integer value and some other number, assuming that the former should be less than the latter, if that value has overflown into the negative, the comparison would pass. But are things going to behave the way the programmer intended? Probably not.

Additional Sources of Information on Integer Overflows

While it is beyond the scope of this article, there are other resources which go into more detail about integer overflow bugs, their prevention, and their exploitation. Namely, there are two very interesting articles in Phrack #60 (one by Oded Horovitz, and one by blexim) on integer overflow vulnerabilities. Integer wrapping is also covered in the “professional source code auditing” presentation from the 2002 USA Black Hat Briefings (Dowd, et als).

A Tool to Experiment with Integer Overflows

The program int_wrap.c allows you to play around with this behavior by specifying on the command line whether the data type (short) should be signed or unsigned, and which value you want to use.