Exceptions and Performance

Note: This page has now been superceded by a newer version
which is significantly different. As the new version is responding to some flaws pointed out
elsewhere, it seemed unfair to just change the contents of this page and make the comments
look foolish. The newer page is probably a more balanced one - this page should be considered
obsolete now.

Almost every time exceptions are mentioned in mailing lists and newsgroups,
people say they're really expensive, and should be avoided in almost all
situations. As an idea of just how expensive some people think they can be,
in one article someone asked whether the fact that his web application was
throwing about 200 exceptions an hour was likely to be harming his performance.
Various people replied saying that it would indeed be causing a problem.
Let's examine that claim, shall we?

The True Cost of Exceptions

Here's a short program which just throws exceptions and catches them, just to
see how fast it can do it:

Now, the above isn't geared towards absolute accuracy - it's using DateTime.Now to measure time,
just for convenience, but if you give it enough iterations to make the test run for a fair time (half a minute
or so) then any inaccuracies due to a low resolution timer and the JIT compiler are likely to get lost in the
noise. The main thing is to see roughly how expensive exceptions are. Here are the results on my
laptop, using .NET 1.1, running outside the debugger (see later for the reason for emphasis):

Total time taken: 00:00:42.0312500
Exceptions per millisecond: 118

Update (2006): .NET 2.0 makes exceptions significantly slower. It depends on the exact hardware involved,
but on my laptop it can "only" throw about 21 exceptions per millisecond. The rest of the figures in the
article still refer to 1.1.

Now, that doesn't involve any significant depth of stack, and indeed if you change the test to recurse until it
reaches a certain stack depth, it does become significantly slower - recursing to a depth of 20 takes the
results down to about 42 exceptions per millisecond. Also, running with .NET beta 2 gives fairly different
results - even the test above only manages to throw about 40 exceptions per millisecond. However, those
differences are only a factor of three - not enough to change the overall performance of exceptions, which
is clearly pretty good.

Let's look back at the example from the newsgroups - 200 exceptions being thrown in an hour. Even assuming
a server which was 10 times slower than my laptop (which seems unlikely) and assuming a fairly deep stack,
those 200 exceptions would still only take about 50ms. That's less than 0.002% of the hour. In other words,
those exceptions weren't significant at all when it came to performance.

Performance In The Debugger Is Unimportant (But Poor)

I have received an email in response to this article which claimed that throwing 5000 exceptions to a stack
depth of 1000 was taking nearly ten minutes. Now, the correspondent acknowledged that a stack depth of 1000
was very rare, but even so this figure seemed very high to me. I ran the short but
complete program that he'd kindly sent me and which he'd used to get his figures. (He'd also provided
the results he'd seen. This is very much the kind of thing I look for when someone disagrees with me -
the cause of the disagreement and the means of reproducing it. It's a far more productive basis for
discussion than just words.) So of course, I ran his program.

The same program that took over nine minutes on his box took about about 47 milliseconds to run on mine.
Now, while I'm very happy with my recent laptop purchase, I find it hard to believe it's over 10,000 times
faster than my correspondent's box. Looking for explanations, I ran the same program having recompiled it
under the .NET 2.0 beta 2, and it took 250 milliseconds. That's significantly slower, but not nearly
slow enough to explain the discrepancy.

I then ran it under the debugger of Visual Studio 2003. It took nearly 6 minutes. Now, that's much
closer - close enough to easily be explicable by hardware differences. So, it's reasonable to assume that
my correspondent was running his code under a debugger. Why is that important? Because the performance
when running under a debugger is unimportant.

You should usually only run code under a debugger when you're debugging - hence the name! When you're
debugging, performance shouldn't usually be important - the only case I can immediately think of where
it is important is where you have an error which crops up one in a million times, and you need
to run some code a million times before you can diagnose the problem. Well, if you're in that situation
and each of those million runs requires an exception to be thrown from a very deep stack, you
may have a problem unless you're willing to leave a machine running the test overnight. Personally,
I can't remember the last time I was in that situation.

I am utterly convinced that the cost of exceptions in the debugger is the cause of a lot of the myth
about the general cost of exceptions. People assume that a similar cost is present when running outside
the debugger when it just isn't. (It doesn't matter much whether you run a debug version of the code or
not - it's whether a debugger is attached or not that matters.) This is made worse by the way that the
first exception thrown under a debugger can take a few seconds (although I still haven't worked out
why this happens - it doesn't seem to involve hard disk or CPU activity). I've seen developers claim
that exceptions each take a second or two to be thrown - something that would render them much,
much less useful.

Note that it's not only exceptions which fall foul of this. The debugger disables lots of optimisations
the JIT can normally make, including inlining (in at least some places). If you take too much notice of
performance in the debugger without testing outside the debugger, you could easily end up making
optimisations which only make a significant difference in the debugger, but which hurt your design
and readability.

So Should I Throw Millions Of Exceptions?

My rebuttal of the claims of the cost of exceptions has sometimes been misinterpreted
as a belief that exceptions should be used anywhere and everywhere, and that you
shouldn't be worried if your application throws millions of them. This is not the case - I
merely think that in almost every situation where the performance of exceptions would
be significant, there are bigger design problems to worry about. While it really
wouldn't matter in performance terms if every so often you threw several thousand
exceptions in a second, it would be suggestive of a situation where either exceptions
shouldn't be used, or the first exception should have triggered a back-off of some
description.

As an example of the first, if you're trying to parse a huge list of strings which
were meant to be integers, knowing in advance that many of them wouldn't be valid integers to start
with, it would be wise to use some kind of validation prior to the actual parsing. (For simplicity,
the validation would probably only want to be a very simple filter which weeded out obviously bad
data - you'd still want to handle a possible exception in the real parsing.) Alternatively,
if a suitable library call were available, you could use a version which didn't throw exceptions:
in .NET 2.0, for example, each of the numeric types has a TryParse method for precisely
this kind of situation.

As an example where back-off (or indeed total failure) should be used, you shouldn't have a loop
which constantly tries to access a database, retrying the same operation again eternally if something
goes wrong. Here, using exceptions is the right way of indicating that something has failed, but you'd
want to put some kind of limited retry mechanism in, with the overall operation failing if any part
failed more than three times, or something similar. Whether that overall operation then goes into
a retry state (after a suitable back-off delay) or whether it's aborted permanently depends on the
situation - but changing the interface not to use exceptions to indicate failure wouldn't help anything,
and would just make the error handling more complicated.

This article isn't meant to be an in-depth style guide for exceptions - there are many,
many articles on that topic, and frankly I'm not sufficiently convinced that I have "the
right answer" to try to write another one. The crucial point is that although exceptions
are indeed slower than simply returning an error code, unless you are abusing exceptions
to start with, you're very, very unlikely to find that exceptions are a performance bottleneck.
When designing the contract of a method, it's worth considering whether it's reasonable to
call that method in a fairly tight loop, continuing to call it even if the previous call fails.
In that case, it may be worth having two methods, one of which uses exceptions to indicate
errors and one of which doesn't (like Parse and TryParse) but in all
other situations, using exceptions is unlikely to cause performance problems, but is likely to make
the calling code much easier to understand.