A completely different approach is to use compilation methods that perform
bounds-checking (see [Sitaker 1999] for a list).
In my opinion, such tools are very useful in having multiple layers of
defense, but it’s not wise to use this technique as your sole defense.
Many such tools only provide a partial defense.
More-complete defenses tend to be slower (and generally people choose
to use C/C++ because performance is important for their application).
Also, for open source programs you cannot be certain what tools
will be used to compile the program;
using the default “normal” compiler
for a given system might suddenly open security flaws.

Historically a very important tool is
“StackGuard”, a modification of the
standard GNU C compiler gcc.
StackGuard works by inserting a “guard” value (called a “canary”)
in front of the return address; if a buffer overflow
overwrites the return address, the canary’s value (hopefully) changes
and the system detects this before using it.
This is quite valuable, but note that this does not protect against
buffer overflows overwriting other values (which they may still be able
to use to attack a system).
There is work to extend StackGuard to be able to add canaries to other
data items, called “PointGuard”.
PointGuard will automatically protect certain values (e.g., function
pointers and longjump buffers).
However, protecting other variable types using PointGuard
requires specific programmer intervention (the programmer
has to identify which data values must be protected with canaries).
This can be valuable, but it’s easy to accidentally omit
protection for a data value you didn’t think needed protection -
but needs it anyway.
More information on StackGuard, PointGuard, and other alternatives
is in Cowan [1999].
StackGuard inspired the development of many other run-time mechanisms to
detect and counter attacks.

IBM
has developed a stack protection system called ProPolice
based on the ideas of StackGuard.
IBM doesn’t include the ProPolice name in its current website - it’s just called
a "GCC extension for protecting applications from stack-smashing attacks".
However, it’s hard to talk about something without using a name, so
I’ll continue to use the name ProPolice.
Like StackGuard, ProPolice
is a GCC (Gnu Compiler Collection) extension for
protecting applications from stack-smashing attacks.
Applications written in C are protected by automatically inserting
protection code into an application at compilation time.
ProPolice is slightly different than StackGuard, however, by adding
three features:
(1) reordering local variables to place buffers after pointers
(to avoid the corruption of pointers that could be used
to further corrupt arbitrary memory locations),
(2) copying pointers in function arguments to an area
preceding local variable buffers (to prevent the corruption of pointers
that could be used to further corrupt arbitrary memory locations), and
(3) omitting instrumentation code from some functions
(it basically assumes that only character arrays are dangerous; while
this isn’t strictly true, it’s mostly true, and as a result ProPolice
has better performance while retaining most of its protective capabilities).

Red Hat engineers in 2005 re-implemented buffer overflow countermeasures
in GCC based on lessons learned from ProPolice.
They implemented the GCC flags -fstack-protector flag
(which only protects some vulnerable functions), and the
-fstack-protector-all flag (which protects all functions).
In 2012, Google engineers added the -fstack-protector-strong flag
that tries to strike a better balance (it protects more functions than
-fstack-protector, but not all of them as -fstack-protector-all does).
Many Linux distributions use one of these flags, as a default or for
at least some packages, to harden application programs.

An especially strong hardening approach is "Address Sanitizer" (ASan,
also referred to as ASAN and AddressSanitizer).
ASan is available in LLVM and gcc compilers as the "-fsanitize=address" flag.
ASan counters buffer overflow (global/stack/heap),
use-after-free, and double-free based attacks.
It can also detect use-after-return and memory leaks.
It can also counters some other C/C++ memory issues,
but due to its design it cannot detect read-before-write.
Its has a measured overhead of
73% average CPU overhead (often 2x), with 2x-4x memory overhead;
this is low compared to previous approaches, but it is still significant.
Still, this is sometimes acceptable overhead for deployment,
and it is typically quite acceptable for testing including fuzz testing.
The development processes for Chromium and Firefox, for example, use ASan.
Details of how ASan works is available at
http://code.google.com/p/address-sanitizer/,
particularly in the paper
"AddressSanitizer: A Fast Address Sanity Checker" by Konstantin Serebryany, Derek Bruening, Alexander Potapenko, and Dmitry Vyukov (Google), USENIX ATC 2012
Fundamentally ASan uses "shadow bytes" to record memory addressability.
ASan tracks addressability of memory, where addressability means
if a read or write is permitted.
All memory allocations (global, stack, and heap) are
aligned to (at least) 8 bytes, and every 8 bytes of memory's
addressability is represented by a "shadow byte".
In the shadow byte, a 0 means all 8 bytes addressable, 1..7 means
only next N are addressable,
and negative (high bit) means no bytes are addressable.
All allocations are surrounded by inaccessible "red zones"
(with a default size of 128 bytes).
Every allocation/deallocation in stack and heap manipulates the shadow bytes,
and every read/write first checks the shadow bytes to see if
access is allowed.
This countermeasure is very strong, though it
can be fooled if a calculated address is in a different valid region.
That said, ASan is a remarkably strong defense for applications written
in C or C++, in cases where these overheads are acceptable.

A "non-executable segment" approach was developed by
Ingo Molnar, termed
Exec Shield.
Molnar’s exec shield limits the region that executable code can exist,
and then moves executable code below that region.
If the code is moved to an area where a zero byte must occur, then
it’s harder to exploit because many ASCII-based attacks cannot insert
a zero byte.
This isn’t foolproof, but it does prevent certain attacks.
However, many programs invoke libraries that in aggregate are so large
that their addresses can have a non-zero in them, making them much
more vulnerable.

A different approach is to limit transfer of control; this doesn’t
prevent all buffer overflow attacks (e.g., those that attack data) but
it can make other attacks harder
[Kiriansky 2002]

In short, it’s better to work first on developing a correct program
that defends itself against buffer overflows.
Then, after you’ve done this, by all means use techniques and tools
like StackGuard as an additional safety net.
If you’ve worked hard to eliminate buffer overflows in the code itself,
then StackGuard (and tools like it) are
are likely to be more effective because there will be
fewer “chinks in the armor” that StackGuard will be called on to protect.