Introduction

VS.NET provides two mechanisms to aid the programmer in diagnosing and correcting programming errors. One is the Debug class, and the other is the Trace class. These two classes contain an Assert function. The Assert statement is used to verify value, and in many cases, that the value of, say a pointer, is not null after invoking some system API. With the inclusion of try-catch blocks, it seems that we are migrating to a different approach, one in which the system API throws an exception which we have to “catch”. This makes our lives more complicated because now we cannot simply test the return of an API function for a failure return; we have instead to handle “exceptions”. Finally, life as a programmer is complicated by the fact that we build two kinds of versions of our programs—the debug version and the release version.

This article attempts to discuss these issues from the point of view of one biased programmer, based on his experiences in the real world, working with hardware engineers, junior grade programmers, feisty QA staff, and “touch it and it breaks” marketing people.

Now, if you’re one of those programmers that doesn’t use asserts and traces, then you might as well stop reading here, because nothing I say is going to motivate you to start programming in a way that makes your life easier, lowers the cost of the development cycle, and gives you useful feedback when your product goes in to the test cycle.

An Example Of A Project Development That Worked

Several years ago I was the team leader for a multi-channel video surveillance system, accepting video feeds from up to 64 cameras over four RS-232 lines, connected either directly to the video sources or via a leased line modem or a dial up line. There were various complexities of the project that I’m not going to get into here.

Instrumentation: trace logs

At the very start of the project, we instrumented:

all GUI based events—menu selections, button selections, etc.;

all internal messages (we used a messaging architecture to isolate functional components);

all state changes.

By “instrumented”, I mean that we output to a debug log file informative messages regarding events, messages, and current state. Additionally, we instrumented various “mission critical” variables that fell outside of the event/message/state domain.

Instrumentation: Asserts

We also asserted the correctness for parameter inputs to all functions and asserted the return values of those functions. An assertion was also output to the debug log. When an assert occurred, the debug log was closed, a message was displayed, and the program was terminated.

Uncooperative QA Personnel

During development, the QA department tested various modules as they became functional. Initially, the relationship with the QA department was rather adversarial, and because of our instrumentation, we discovered that QA was testing precisely the functions we told them not to test (they literally wanted us to fail in management’s eyes). After correcting various perceptions, the QA people actually started producing useful work. One of the problems with any tester is that when asked “what did you do”, they will confidently say, I clicked on X, then Y, then Z, when in reality they clicked on A, B, and C. As a result of our instrumentation of all GUI events, we never had to ask what the tester did, we could figure it out from the log.

Field Testing

Because in-house use is different from actual field use, we also provided the program in a “beta” form to select customers with the instrumentation enabled. This revealed some issues, mostly with hardware differences, that we didn’t see in-house.

Regression Testing

Finally, because of the way the instrumentation hooked into the GUI’s, we were able to record and playback test scripts to automate time consuming yet useful test procedures, so we were actually able to do regression testing—the ability to test what used to work on the new code.

Development Testing

Needless to say, the instrumentation was highly useful during our own coding (especially coupled with version control, which allowed us to revert to previous versions and test the same event sequence).

Shipped Ahead Of Schedule

Management, having been burned on previous projects and having irate customers to deal with, allocated 3 months for testing (an unheard of amount of time because the coding took only 6 months, and testing was ongoing during coding also!) After only one month, the QA department was unable to find further bugs and the product shipped ahead of schedule.

Release vs. Debug

For performance reasons, because the application described above was so heavily instrumented, we turned off all trace statements but left in the assert statements in the release mode. This was also done because in the field the program would be running continuously, and the log file would get unmanageably large. Furthermore, the assert statements produced a valuable message as to where and why the assert occurred, because inevitably something will fall through the QA testing process, especially in the world of ever changing hardware.

I believe that this is a good philosophy, to ship an un-instrumented release version to the customer. If the customer is repeatedly having problems and they are willing to work with you, then you can send them a debug version with full trace capability. In addition, with a properly implemented assertion handler, you can display a graceful error message to the user. I love the error message that IE 6 comes up with—something like “we’re sorry to inconvenience you, but a problem has been detected, etc.” It even gives you the option of automatically restarting the application. From a user’s point of view, it’s more like getting bruised by an air bag instead of thrown through the windshield—a definite improvement. If the customer has Internet access, then the application can send you some information about the error, and this is where leaving the asserts in the release code comes in handy—you can get useful information beyond an execution stop address, some stack information, and a register/memory dump.

Debug vs. Trace

In C#, two classes, Debug and Trace, are provided. While both can be disabled, by default both are on in the debug build mode. When in the release build mode, the Debug functions are disabled while the Trace functions remain enabled. Now, both classes support assertions and tracing, however, given the names of these classes, one would more likely use Debug.Assert for assertions and Trace.Write for traces.

If you buy into the logic I discussed regarding asserts and traces, this is exactly the wrong thing to do. Because of performance and application lifetime considerations, the pure trace functions should be removed in the release version, and the Debug.Assert statements should be left in!

Thus, an application should use Debug.Write for traces and Trace.Assert for assertions. When built under release mode, the Debug.Write statements are inactivated but the assertions are left in.

The Trace Switch

C# has the ability to, without recompiling, modify the trace flag. This enables you to turn on tracing at different levels in a release version of your program. I consider this to be of dubious value for two reasons—one, your program still incurs the performance hit of the function call, and second, anyone with minimal programming experience can adjust the trace level and thus learn important information about your program that my reveal corporate secrets or compromises the security of your system. This is another good reason not to use the Trace class for instrumentation logs.

Exception Handling with Try and Catch Blocks

It appears that we have to live with handling errors with try-catch blocks because API’s are now shipping that throw exceptions as the “new and improved” method of reporting errors (instead of, for example, returning a result status or a value to test against indicating failure). This means that we are now left with the dilemma of how to handle the exception. In the old days, we would have handled the error either with an assert or a conditional test, assuming there was some graceful way to back out of the function. So, how do we deal with exceptions from the viewpoint of asserts, traces, debug and release builds?

The programmer has only one decision to make regarding an exception: can it be gracefully handled? For example, can the file read operation be terminated with an informative message to the user without terminating the application? If the answer is “yes”, then code the exception accordingly. If the answer in “no”, then generate an assert. In the cases where some information can be gleaned as to the type of exception, then this becomes valuable information in making addition decisions as to how to handle the exception and what information to display as part of the “this program has failed” message.

A third class of exceptions exists, which I think is much rarer, and this is the kind of exception that doesn’t impact the operation of the program. Perhaps the program is testing for a feature. For example, “is there a CD in the drive”. The programmer may have no choice other than to handle a “no” answer as an exception, with the result that a CD-ROM icon is simply not displayed in a file list.

Introducing “Warn”

Let’s assume that in the discussion above, the exception can be gracefully handled, but it indicates a failure of some kind. So, we still want to inform the user of the failure. This is accomplished with a “Warn” message. Wouldn’t it be nice if a debug class had this capability, so warnings could also be logged in our debug mode, produce a warning message in the release mode, and wouldn’t be fatal to the application?

Introducing The “Possible Problem/Reason Dictionary”

Wouldn’t it also be nice if we could display to the user some information as to the problem, and the possible reasons for this problem? This is accomplished with a problem/resolution dictionary. Each assertion or warning would reference one or more problem/reason statements that would be displayed as part of the “this program must terminate” message. Placing these messages in a dictionary means that you can easily generate this information for the documentation department, perhaps to be included in an appendix or a help system.

Note that both asserts and warnings can reference the same dictionary, making for a flexible feedback system for the user.

Behind The Scenes

I wanted to test what the actual assembly code looks like for both Debug classes, using the Assert and WriteLine methods.

To summarize: all Debug function calls are removed, and the Trace functions are changed to call the Debug class functions! This is undoubtedly all controlled by the “Conditional” attribute capability of C#, and we will use the same methodology to implement a “smarter” debug class.

Implementing A Custom Debug Class

Unfortunately, the Debug and Trace classes cannot be overridden, so we have to implement our own version. In this class:

Can hook into the unhandled exception handler to further enhance user friendliness.

Listeners And Member Functions

Interestingly, the Debug and Trace classes use the same Listeners. If you add a listener using the Debug.Listeners.Add method, this listener is also active when using the Trace class methods. This is also true of the indent functions and other functions. This also means that a class that mirrors and extends the Debug and Trace class functionality need only pass functions through to one of the two .NET classes.

Debug vs. Release

In the debug class provided in this article, the following differentiation is made:

Debug mode:

includes all Trace mode functionality, plus:
all write functions are enabled
adding a file listener is enabled
all functions having to do with indenting, flushing, and closing listeners are enabled

Trace mode:

An Unhandled exception handler can be initialized (built in to the class)
Warnings can be output (Warn method)
Asserts are enabled

The Unhandled Exception Handler

The following code instantiates an unhandled exception handler. This provides a more graceful way to terminate the application, and in debug mode, outputs the exception to the log file.

Notice that the exception handler can be initialized for both Debug and Trace conditions, meaning both Debug and Release modes.

The Warn Method

The following code can be used in exception handlers to output a warning to the user that some corrective action ought to be taken. The Warn method uses the problem dictionary to indicate the problem and look up the reasons associated with the problem. This is presented in a (hopefully) nice way to the user.

The Assert Method

The assert method, when invoked using the overloaded method Assert(bool b, DbgKey key) outputs a trace line containing the assertion and produces a user friendly message. This message, as with the Warn method above, uses the problem dictionary to show some hopefully usefull information to the user.

Concluding Remarks

In the above discussion, we have identified that in debug mode, both asserts and traces should be enabled, and in release mode, only asserts should be enabled. Secondly, we have identified that exceptions generated by try-catch blocks that the application cannot handle should assert, whereas those that are handled should possibly generate a warning to the user. Lastly, we have identified that an assertion should provide some problem/resolution information to the user. Also, in the debug version of the program, all trace and asserts are logged, whereas in the release version, there is no logging.

Also, it might be nice to extend this class to read the problem dictionary in from, say, an XML file.

Updates

Fixed a serious bug, in which the boolean condition was not being tested in the Warn and Assert functions. How
embarassing!

As suggested in the responses below, added a Verify method and added the stack trace to the exception handler. Note that the Verify method doesn't exactly correlate to the C++ VERIFY macro. This is due to the limitations of the [Conditional] attribute--you can't perform boolean logic.

I added a Fail method that can be used in conjunction with exceptions that are trapped but not recoverable.

References

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

Comments and Discussions

I've been using tracing for a long time but one thing I would really like to do is to have conditional tracing in Class library.

I got App X with option set to only write trace errors, when I write the trace I write it like Trace.WriteLineIf(logErrors, message,cat).

I've created a custom TraceListener class which writes the trace information.

Everything works fine but what I want is my custome dlls to be able to selectively write tracing as well. What I can't work out is how my class library knows if to log errors or not, or if to log warnings or not. I want to enable/disable tracing in the main applications (which I can do using App.config files) as well as any dlls(in house class libraries) at the same time using configurations.

I loved this article. I too have written many programs which log each and every UI action and various levels of tracing into functions, etc. It was very helpful in debugging/QA, as you mention. But I also find it highly valuable in production, especially if running a desktop app (or I would assume mobile also) where the OS is the same but, alas, how "same" can any OS be these days? With auto-updates, different add-on apps affecting the user experience, etc., I find that having full debug capabilities in the production app is a must.

That being said, the performance issues are real, as are the space issues.

Using the event log for this type of logging is not intuitive, nor do I find it helpful when it comes time to debug. So I tend to write out my own log files and either ignore the event log or just write exceptions there so the sysadmin who knows little or nothing about my apps has a place to look.

Using the debug and trace objects have been counter-intuitive for me in this regard, but your article may help me synthesize them into my future apps in a more central way (which is exactly what I was looking for when I searched for your article in the first place).

So, I like to opt for a configuration-based level of logging. The highest level logs every function call, and is useful in QA mode or in production support mode. But in production this would not be the default. In production, a lower level is used, with most of the logging turned off. However, when a problem arises that is not reproducible in support, instead of giving them a build version (which is always suspect that it will give the same problem anyway), just change a config parameter and reproduce the problem in place.

There is still a slight performance penalty becuase the routines are being called. But this could be a small price to pay. But I do admit that it may depend upon the platform. For example, a phone-based app may not have the luxury of having all this non-logging taking place, so a physically different build may be warranted.

Hi
I tried to use the C++ code for tracing given in MSDN. But i get compilation error that "Trace" as undeclared identifier. Could you tell me is there Trace class in C++? If there how to use it.
Below is the sample code given in MSDN.

Marc,I've been having a think about your excellent article and came up with an idea for populating the ProblemCollection from an auto generated dataset.

Let me explain:I've been looking at the MSDataSetGenerator tool, supplied with visual studio, and keep finding new and interesting ways to use it, this being one of them.If we create an XSD to define our xml (and therefore our dataset), the tool will generate the c# code for the data retrieval. All we have to do is use it to populate the sorted list.

If we make the collection a singleton class then it will only get populated once by the constructor and we could always make it write out to the xml on destruction if we've added any new problems/reasons.

From here it's only a small step to creating a TraceListener for xml files and we could have the output as xml to.

Why is the Debug.WriteLine do not print to the output window pane? Any settings do i need to configure? Where will it output to?
I use Trace.WriteLine instead cause i could view it in the DebugView utility or to the log file i set in the listener.

First of all, congratulations on an outstanding article. Apologies in advance for the long message/commentary.

Some comments...
For those of you who find this type of article useful, Steve Maguire's 'Writing Solid Code' is a must read.

I'm not sure I understand your aversion to raising exceptions. From my perspective I see many advantages to using exceptions:
- They are more code friendly in that they generally expose more information than return codes (i.e., I can query for an description string rather than decoding return values). In addition they can be consolidated to an appropriate area for exception handling (e.g., say at the transaction level).
- They are less passive -- they happen even if the consumer forgets to check for them.
- They are much easier to fire/propogate in certain circumstances (e.g., overloaded operators or deeply nested routines).

I agree that handling certain types of non-fatal exceptions in code is ugly but I believe this is indicative of poor code design (either in the calling function or callee's design) rather than a flaw with exception handling.

Exception handling does requires more thought as to the location and structure of try...catch blocks. For example, I think all UI handlers should have try...catch logic in them -- its pretty embarrassing when your app terminates because you forgot to catch a file not found exception.

I'm not sure including Asserts in a release build is a good idea. Especially if you subscribe to the 'assert everything' strategy -- its too much of a performance hit. Asserts should not be used to raise error conditions... they are really a method for identifying the original source of an error.

This is where having access to the stack information is really helpful. Sometimes an exception is generated far from where it is caught (especially when it makes it to the unhandled exception event handler) -- sometimes the most useful piece of information is how the exception was generated. How about placing a More>> button in your DbgExceptionHandler which shows the call stack (or an encrypted version of the call stack if you are concerned about exposing this info). This provides the developer with a huge amount of information as to how the exception was generated. See http://www.codeproject.com/dotnet/unhandledexceptions.asp for some discussion on this.

One last item (finally!) -- How about adding a Verify routine which amounts to an Assert in debug and does nothing in release (the difference being that the code generating the boolean is always evaluated). I find the VERIFY macro is C++ to be really useful and it would be great to have a similar C# routine.

Great comments! In addition to fixing a glaring bug (I seem to have forgotten to check the value of the boolean condition in the Warn and new Assert function-- ), I'll see if I can incorporate your suggestions. Thank you for the unhandled exception article reference.

I agree with your points about exceptions. I think my experiences are biased by working with other team members that touted this new feature when it first appeared and then didn't use it, or used it inappropriately. I think the whole try-catch-throw-finally thing needs a good article, and my next project was to look for one (before I even consider the possibility of writing one myself ).

When I suggested the Verify routine I had this nagging thought in the back of my head..."What if the optimizer is 'too smart' and optimizes out the whole call to Verify?" -- it could be a real bear to find since it would only show in the release build.

Any thoughts on this... I'm not too familiar with the optimizer yet but I know it uses inlining for small routines. Have you tried out the Verify routine and found it to be successful in the release build?

Hey guys, I'm not really sure where to post this idea, no one seems to look at the article suggestion forum. At any rate, I've been unable to find an article on exception management strategies and you all seem to have a pretty solid understanding of the issues. At least you did 9 months ago... Note that this idea wouldn't even touch on logging or debugging. There are plenty of great strategies in this area already, and this strategy should work regardless of the one chosen for that...

Basically, I'm looking for a strategy that meets the following requirements:
Goals
1. Provides an Intelligible message to the user that
a) Explains the problem
b) Provides a list of possible explanations
c) Suggests steps to remedy the issue
d) Can be told what actions are allowable in the wake of an error (continue, quit, retry)
2. Is fully configurable via XML
3. Doesn't clutter your code
4. Provides backward compatibility so if a component developed with this strategy is used by an application that does not use it, the new author does not have to handle things differently.
5. Information must be preserved as the exception is passed up the stack. (more on this later)

High level design considerations
1. My thinking is that exception handling does not need to be very fast, it is more important to be very accurate.
2. Display should be performed at the highest level (usually the display level, but not necessarily), and never anywhere lower. (Displaying an error in a utility function ties the backend code to a display technology and method, reducing it's ability to be reused. It also means that the same error may result in multiple messages to the user).

Tentative Implementation Plan
Currently I'm planning to create an exception factory, and XML files for each assembly. The user might throw an exception thusly:

The factory first would check a config file to see if it should use this new exception handling mechanism, or just the built-in .net error handling. If regular handling, then it simply returns the original exception. If enhanced error handling is used, then the factory would use reflection to determine where the exception originated (the 'me' object). Then it will hit that assembly's associated XML file to find the info on this error. In this way, the code can be used for any project w/out change. It also means that the XML file could be used as an appendix of sorts. Finally, storing this info in the XML file means that it doesn't have to be in your source code. The args may be used to pass extra information. For example, if it's a file not found error, it would be helpful to know what the path is of the file that didn't exist is. All of this info is stored in a custom exception class.

When the exception is formed and thrown, it will proceed up the stack. If there is another intermediate function, the factory method is again invoked. However the factory will check to see if it is of the same type as our custom exception. If so, information can be added to the messages, and possible actions can be reduced as necessary.

This brings me to a more detailed explanation on #5. My concern is that creating an error message way down in some utility function may not make any sense at all to a user because they may not understand why that utility function was even called. (Half the time I don't even understand!) Further, a utility function is probably so technical, that no possible error message would make sense anyway. If you're performing an autosave that fails, you want to tell the user that the autosave failed, not that the filestream could not be opened... So basically, care must be taken so that information is ADDED as the exception passes up the chain to get a message like this:

-------------------------------------------------------------------------
Page copy failed
because
the data could not be obtained
because
deserialization failed because
the file can not be found the exception in the first place

If the file: {0} exists on a network share, please check your connection to the network and try again. If the file is gone, then the data has been lost. You may either continue without this info, or quit.
<<Retry>> <<Continue>> <<Quit>>
--------------------------------------------------------------------------

What I'm looking for is:
1. What do you think of my requirements?
2. Is this the best strategy to go with?
3. Any other general suggestions?

I'd like to point out that since you posted this in my article, the only two people that are going to read it are me and the fellow to whose message you replied.

Your writeup is well worth putting into the article suggestion forum and possibly the collaboration forum. Note that I agree with you, though, that neither gets a lot of interest.

Jeff Deville wrote:What I'm looking for is:
1. What do you think of my requirements?
2. Is this the best strategy to go with?
3. Any other general suggestions?

I think your requirements are excellent.
I like your strategy, but keep in mind that everyone has a different solution to this problem.
I'd suggest submitting a project to www.tigris.org or www.sourceforge.net and see what interest you can gather. What you're describing here is more a product than an article, in my opinion.

Thanks for the feedback Mark. I did realize that I'd be limiting my audience posting here, but you were one of the main people I wanted feedback from anyway. (No one replied to me in the Article Suggestions forum earlier)

At any rate, I think that I'll take a stab at it on my own since I need to anyway, throw up an article as a strategy to generate hits/interest, and just keep things up to date.

An excellent article on exceptions is "When and How to Use Exceptions" by Herb Sutter, C/C++ Users Journal, August 2004. The article is very generic and completely applicable to the C# and Java worlds as well.