Decoding the Mysteries of .NET 2.0 Configuration

Continue delving into the mysteries of the powerful .NET 2.0 Configuration framework, and learn how to write validated, type-safe, and performant configuration code.

Introduction

One of the wonderful features of .NET has been its XML configuration features. In the days of .NET 1.x, common application settings, database connection strings, ASP.NET web server configuration, and basic custom configuration data could be stored in a .config file. Custom configuration sections could use a few basic but custom structures, allowing a small variety of information to be stored in a .config file. More complex configuration, however, was most often accomplished with custom XML structures and custom parsing code. Such code could become quite complex, though, and there are a variety of ways of differing performance to accomplish the same thing.

With .NET 2.0, the days of writing your own (probably complicated, poorly-performing, and tedious) code to manage custom XML configuration structures are over. The custom configuration capabilities of the built-in XML configuration subsystem in .NET 2.0 have been greatly revamped, boasting some extremely useful and time saving features. With relative ease, just about any XML configuration structure you might need is possible with a relatively minimal amount of work. In addition, deserialization of the XML in a .config file can always be overridden. This allows any XML structure necessary to be supported without losing the other advanced features of .NET 2.0 configuration.

The Continuing Mystery

In the first installment of the Mysteries of .NET 2.0 Configuration, I covered some of the basics of the configuration framework. Creation of custom configuration sections, section groups, elements, and collections encompass the core features of the configuration framework, but are only the beginning. In this article, we will begin decoding some of the more interesting pieces of the .NET 2.0 configuration puzzle such as type conversion and validation. This article will also cover some of the performance and usability concerns of the .NET 2.0 Configuration framework, and how to improve or overcome them.

This article is a continuation of the Mysteries of .NET 2.0 Configuration series. The first article, "Unraveling the Mysteries of .NET 2.0 Configuration", can be found here. If you are unfamiliar with what the .NET 2.0 Configuration framework has to offer, and do not know how to write and use custom ConfigurationSection classes, it is recommended that you read the first installment of this series before continuing.

As we learned in "Unraveling the Mysteries", writing custom configuration sections is quite simple and straightforward. The benefits of writing a .NET 2.0 ConfigurationSection are efficient, globally accessible, type-safe, and validated configuration. Type safety and validation are two topics that were briefly discussed in the original article, but which were never elaborated on. These two aspects of configuration, while not always critical, can be very important to an application's health and correctness when you expect specific ranges and types of data.

Validating the correctness of data in your custom configuration is very straightforward. The .NET 2.0 Configuration framework actually includes several premade validators that should suffice for most validation needs. In the event that one of the premade validators is not sufficient, there are two additional methods of validating your configuration data. It is important to note that only one validator at a time can be applied to a configuration property. Sometimes you may find yourself needing to use two validators, and the simplest solution to this is to create your own wrapper validator, using the two premade solutions within your custom validator class.

Using premade validators in your custom configuration classes is a simple matter of applying them to the properties you wish to be validated. There are two ways of applying validators, one imperative and one declarative. If you choose to explicitly create your configuration properties and override the Properties collection of your ConfigurationSection and ConfigurationElements, you must use the imperative method. Otherwise, you may declaratively apply a validator to each property using attributes. As you will see from the list below, all configuration validator classes have a matching attribute class:

For the most part, these validators are self-explanatory. Validating ints, longs, and TimeSpans is a fairly trivial matter that requires little in the way of examples, but they do make for a simple introduction. There are some additional features of these validators that also warrant discussion, too. Let's assume we have a configuration section like the one below:

We can write a validated configuration section that ensures the data typed into each of the three properties is indeed valid TimeSpan, int, and long data, and that it falls within the range we desire. In the event that the data is not correct, an ArgumentException will be thrown during serialization or deserialization. (Note: Without a validator applied to a property, the exception thrown could be one of a few, including ConfigurationErrorsException, NullReferenceException, or InvalidCastException. By using a validator, we know what exception to look for in the event some configuration data is invalid.) Take special note that we can restrict the range of values with our validators. Our configuration section should look something like this:

In the event that any of the configuration data was outside of the specified ranges (i.e., myInt was -12), you could capture an ArgumentException when ConfigurationManager.GetSection() is called, and take the appropriate action. In addition to being able to check a value within a specified range, you can also specify that the range is exclusive. In this case, any values outside of the specified range are valid, and anything within the specified range is invalid, causing the exception to be thrown. Also, if it was ever desired, these validators can check for a specific single value if the resolution parameter of the full constructor is set. The validator new TimeSpanValidator(TimeSpan.MinValue, TimeSpan.MaxValue, false, 15) would require that the time span equals exactly 15 seconds.

The other premade validators may not be quite as straightforward as the ones just discussed. The StringValidator, as it turns out, does not validate a particular string against what was configured. Rather, it allows validation of the length of the string, ensuring it is between a minimum and maximum value. The StringValidator also allows a string to be specified that determines invalid characters, and throws an exception if the configured string contains any one of those characters. If you do wish to validate that a configured string meets certain formatting requirements, the RegexStringValidator is actually what you need. With this validator, you can specify any standard regular expression which will be matched against the actual configured string. There is no premade validator that will validate both the length of a string and match it against a regular expression.

The last premade validator we will discuss in this section is the SubclassTypeValidator. This handy little validator allows you to verify whether the type of object for a particular configuration property is one that derives from the specified type. This can be very useful when using a custom type converter to serialize and deserialize data from a custom class hierarchy to your configuration files, amongst other things. A more detailed overview of this validator will take place in the Writing Your Own Converter section.

Most often, when writing any piece of code, a simple solution is all that's initially needed. In the event you find yourself writing a validated configuration section and one of the premade validators is not sufficient, but your needs are not particularly complex, the CallbackValidator will usually suffice. This handy little validator takes a ValidatorCallback delegate as a parameter, which should point to a method that takes a single object as a parameter. You can perform whatever kind of validation that may be necessary in this callback function.

Let's assume we have a configuration section that expects an integer value to be the modulus of 10, between -100 and 100. The IntegerValidator and LongValidator do not support this kind of validation, so another solution is needed. If you only need to perform this kind of validation once in a single configuration class, the simplest solution is to use the CallbackValidator.

It is possible to reuse a callback for multiple properties in a ConfigurationSection or ConfigurationElement. If you need to use callbacks across multiple configuration classes, you could create a container class and group all of your callbacks into a single location. However, in the event that you need to reuse a validator for multiple properties or configuration classes, the better solution is to write a custom configuration validator class and its corresponding attribute. This will allow you to more properly encapsulate and share code in a less confusing manner.

Validators require no special constructors, so you are free to create as many as you like with whatever parameters are necessary. It is interesting to note that configuration validators can be created and used in your own code, and do not necessarily need to be applied to a ConfigurationProperty to be used. This is convenient, since it allows you to wrap several validators into a single validator class. Since a ConfigurationProperty can only have a single validator applied to it, the wrapper validator pattern can be very useful in quickly applying multiple validators.

Below is an example of a custom validator. This validator is also a wrapper for both the StringValidator and the RegexStringValidator. When writing your own validators, you must make certain that you override the virtual method CanValidate. Even though it is not abstract, its default return value is false, which will always cause your validator to fail when invoked during deserialization.

As you can see, writing a custom validator is extremely simple. However, we are not done quite yet. We still need to write a custom attribute so we can declaratively apply our validator to a ConfigurationProperty. This step is only useful if you prefer to program declaratively, and is unnecessary if you follow the imperative method proposed in the first article.

A reigning beauty of the .NET platform is its strict type safety. This feature helps in writing safe, secure code. The most vulnerable components of any application are those that access external resources, such as configuration. Thankfully, the .NET 2.0 Configuration framework includes some facilities to help you ensure your configuration data is type safe. Initially, all configuration data is read as a string, and must be converted to the proper type during deserialization to be used. The .NET 2.0 Configuration framework does this automatically for the most part, but there are times when the default conversion is insufficient. Configuration type converters can be used to handle custom conversion, and when used in conjunction with validators, can ensure your configuration data is type safe and secure.

When writing a custom configuration section, type conversion is generally something that is done automatically by the .NET 2.0 Configuration framework. There are some special instances where automatic type conversion is not sufficient, such as when an integer value needs to be any intor infinite, or when a comma-delimited string needs to be converted to a collection. Numerous premade converters exist, and they can be applied declaratively or imperatively to your custom configuration classes:

The Converter Types

GenericEnumConverter - Converts between a string and an enumerated type

InfiniteIntConverter - Converts between a string and the standard infinite or integer value

InfiniteTimeSpanConverter - Converts between a string and the standard infinite TimeSpan value

TimeSpanMinutesConverter - Converts to and from a time span expressed in minutes

TimeSpanMinutesOrInfiniteConverter - Converts to and from a time span expressed in minutes, or infinite

TimeSpanSecondsConverter - Converts to and from a time span expressed in seconds

TimeSpanSecondsOrInfiniteConverter - Converts to and from a time span expressed in seconds, or infinite

TypeNameConverter - Converts between a Type object and the string representation of that type

WhiteSpaceTrimStringConverter - Converts a string to its canonical format (white space trimmed from front and back)

Again, these are all self-explanatory and need little explanation, but for posterity's sake, I'll provide an example. One thing to note is that enumerations are handled automatically and quite well by the framework through the use of the GenericEnumConverter, so manually applying it does not seem to have any value. When applying a configuration type converter to a custom configuration class, you use the standard TypeConverterAttribute.

The true value of type converters shows when you run into a scenario where there is no direct correlation between an XML/text representation of data and the .NET classes and structures that store that data. Consider this simple structure:

Using such a structure in a configuration file is not as straightforward as using an int, TimeSpan, or string. A common, human-editable way of representing this structure is needed so our .config file can be modified by hand (vs. with a configuration program). Let's assume we have chosen to format our string representation like this:

L: 20, W: 30, H: 10, X: 1.5, Y: 1.0, Z: 7.3

Let's also assume we don't care what order each pair is in, nor what capitalization or spacing there is between each component. Let's also assume that all six components are required, and that none of them can be left out. We can now write a type converter to convert between a string representation and our MyStruct structure. Note that for the ConvertTo method, we simply convert straight to a string without verifying that the type parameter is requesting a string. We also assume that the ConvertFrom method will always be receiving a string value for the data parameter. Since this is a configuration converter, it is safe to always assume these cases.

Slow performance is probably one of the top killers of any application, and often the hardest aspect of an application to tune. What runs like a dream in a developer's environment can quickly become overwhelmed in high-throughput, high-use production environments. Optimization is often a long running and ongoing task for many applications. Thankfully, optimizing your custom configuration classes is a very simple matter, and requires hardly any additional work over not optimizing them. Throughout this article and its predecessor, I've written example configuration classes using two methods: declaratively through attributes, and explicitly using static variables and a static constructor. The explicit method is the optimized method, but why it is optimized is not apparent unless you dig into the source code that drives the .NET 2.0 Configuration framework.

The quickest and simplest way of writing a configuration class is to use attributes, applying ConfigurationPropertyAttribute, TypeConverterAttribute, and the various validator attributes to public properties of a simple class. This provides very rapid development, but in the long run, performance will be poor. When such a configuration class is loaded by the .NET 2.0 Configuration framework, a large amount of reflection and object construction must ensue to create the necessary ConfigurationProperty objects and their associated validators and type converters. This is done when the Properties collection of a custom configuration class is accessed, as can be seen in the code snippet below:

This is the default ConfigurationElement.Properties property. Whenever it is accessed, the static PropertiesFromType(Type, out ConfigurationPropertyCollection) function is called, which starts a process that discovers and builds any ConfigurationProperty objects that wrap the specified type in your custom configuration class. Note that it builds allConfigurationProperty objects for the specified type, not just the one you requested. Once a property collection for a specified type is created, it is cached, improving performance on subsequent calls.

It is also important to note that the PropertiesFromType function is static, and as per Microsoft's guidelines, all static methods must be thread-safe. Multiple requests to any property on a configuration class will be serialized by a lock, halting progress on all competing threads until all the ConfigurationProperty objects for the given type have been created and cached. In the event that the ConfigurationPropertyCollection has not been created, the ConfigurationProperty objects are discovered and built with the call to static CreatePropertyBagFromType(Type). This function starts a fairly lengthy process that incurs multiple reflection hits and property table lookups to create each ConfigurationProperty object.

Once your ConfigurationProperty objects have been created and placed in their corresponding ConfigurationPropertyCollection, they are cached for subsequent calls. However, if the underlying .config file changes, the properties must be reloaded, incurring the cost of creation again. It is possible to completely bypass this process by explicitly defining and creating your ConfigurationProperty objects using the static constructor method shown throughout these articles. The trick to understanding why this improves performance is in understanding what overriding the Properties property does. Compare the code below to the original Properties code:

In the overridden property, we are completely bypassing the base code and returning our prepared properties collection. Our properties collection incurs no reflection costs due to the fact that we explicitly defined each and every ConfigurationProperty, as well as the corresponding validators and type converters. The difference in effort to create a custom configuration class that is explicitly implemented, vs. declaratively implemented, is so minimal that there is little reason not to. Writing configuration classes explicitly also helps during a configuration reload, as configuration is available to the application much quicker. (Note: During a configuration reload, it is possible to trigger a subsequent reload while the first is still in progress. When this happens, other code that relies on that configuration can crash or behave oddly. Shortening the reload time helps minimize this effect, and reduce the need to code special cases to react to it.)

Making the best use of the .NET 2.0 Configuration framework generally means following the same best practices you would when writing any piece of code, or when using XML for any other purpose. Best practices are best practices, so don't skimp out just because its configuration you're dealing with. Some of the best practices I use when writing configuration for my .NET apps follow.

Never use more than you need. If your requirements are extremely simple, use the <appSettings> section. If you need something more type-safe but don't need a complete custom configuration section, try using the project settings feature of VS2005. Don't write a custom configuration section to store a single value, unless there is a strong reason to do so.

Don't try to keep it too "simple". A project may start out with simple requirements, but over time, requirements often increase in complexity. Managing and maintaining hundreds or thousands of <appSettings> entries is always going to be a tedious task. Consolidating your configuration into structured configuration sections will allow you to encapsulate, modularize, and simplify your configuration. The time required to put together a set of custom configuration sections will be quickly earned back in reduced configuration maintenance costs.

Code for reuse. Try to keep in the back of your mind, "Reuse, reuse, reuse." The same piece of configuration is often needed for many applications. When designing and implementing your configuration sections, try to keep them as generic as possible, and whenever possible to maximize their reuse in the future. Don't code a custom configuration collection if one of the premade ones can be reused. There are several, listed in the first article in this series.

Always write performant configuration code. There are a few simple patterns that can be applied to custom configuration sections to maximize performance, as described in the Concering Performance section. The performance gains from these patterns are significant, and the added complexity is very minimal over not implementing them. Never say "I'll fix performance later", by then it's too late. ;)

Share

About the Author

Jon Rista has been programming since the age of 8 (first Pascal program), and has been a programmer since the age of 10 (first practical program). In the last 21 years, he has learned to love C++, embrace object orientation, and finally enjoy the freedom of C#. He knows over 10 programming languages, and vows that his most important skill in programming is creativity, even more so than logic. Jon works on large-scale enterprise systems design and implementation, and employs Design Patterns, C#, .NET, and SQL Server in his daily doings.

Comments and Discussions

This was the only useful documentation of the .NET configuration framework I found so far in the internet. Extremely helpful! It's a shame that Microsoft doesn't offer something similar in the MSDN documentation... :(

There are times when writing a custom configuration provider is an overkill, but working with AppSettings is not as elegant.

I have written a simple tool to generate a wrapper class for app settings from Web.config or App.config. The tool is called ClassFromConfig. I'm planning to add automatic type recongnition based on settings values (currently all properties will be of type string).

This is a doozy - maybe someone here will know the answer? Haven't had any luck with Google yet.

I've got a basic custom ConfigurationElement that has a DateTime configuration property. If I modify the element instance in memory then save back to the configuration file, the format of the serialized DateTime property is screwed up, ie it's mm/dd/yyyy when the user is entering and expecting dd/mm/yyyy.

I'm assuming that CultureInfo.CurrentCulture or CurrentCulture.InvariantCulture is being used under the hood to format the DateTime value.

Is there a way to force a DateTime to be serialized with a specific format?

Certainly. You can write your own TypeConverter to convert a DateTime to a string and back, and assign it to the ConfigurationProperty that specifies your date. In your type converter, you have the freedom to serialize the data any way you want. Check the secton in this article on type converters to see how to write one.

So in my configuration project, I have several classes that inherit from ConfigurationElementCollection (BasicMap), and per your first article I override the Properties property in addition to the other methods you suggest (except for the new instance methods used to handle the collection, since my configuration collection is going to be read-only anyway).

When I unit test my project and turn on code coverage, I notice that my Properties property never gets hit for the ConfigurationElementCollection overrides, only for the ConfigurationElement overrides. I thought that was rather odd. I can put a breakpoint in the property, too, and it never gets hit.

In your research, do you know of anywhere that the ConfigurationElementCollection uses its Properties property?

The Properties property shouldn't ever get hit. The purpose of overriding it, in the event that someone DOES use it for some reason, is to prevent a relatively heavy process from ensuing. If you do not override Properties, the base ConfigurationElement Properties property is used, which does some lengthy xml parsing and processing to discover and populate any properties that may exist.

For a pure collection, you don't want that to happen. By overriding the property and returning and empty collection, you reduce the amount of possible processing that occurrs by a fairly significant amount.

That said, it is actually possible to have custom configuration properties on your ConfigurationElementCollection, if you so desired. It is, after all, a derivative class of ConfigurationElement. If you ever needed to implement some custom attributes on your collection, you would override the Properties property and return your collection of ConfigurationProperty objects. (This is somewhat advanced, and may have other repercussions I hope to cover in the third or possibly a fourth article in this series).

Hi John, A very nice series indeed. I am wondering if you have any magic bullets for this problem. Global application settings that are not read only as far as the framework is concerned. Yes, I get the app settings vs user settings spiel, but I don't buy it. Why should the "Administrator" of the system have to hand edit the configuration settings? Hum, who do you trust more to get it right, the developer of an application (programmatic) or someone (manual) who most likely has never see the file before? Awhile back Microsoft made a big deal about splitting the settings into user and app level using the "All users" profile sections to store the app level data. Do you know why they dropped the "All user" for app settings? It would make more sense to me if MS provided three levels of config data App (RO) / Global (RW) / User (RW). I though of rolling my own, but it would be nice to be able to capitalize on MS's work (IDE integration, data binding, versioning, etc.) I know this is something of a rant, but I thought I might at least find a kindred spirit...

From the .NET 2.0 documentation: "Applications use a global configuration that applies to all users, separate configurations that apply to individual users, and configurations that apply to roaming users."

The configuration framework is build around a multilevel "configuration user", and you can access App, User, and even Machine level settings with the ConfigurationManager class. The Machine level configuration is global, system and application wide, affecting everything. Its not recommended that you ever load the Machine.config file or edit it directly in your app, but its possible. If you ever need to do so, you can like this:

ConfigurationManager.OpenMachineConfiguration();

Every application can have both "app" settings and "user" settings. By default, you start with just application settings. Your normal App.config or Web.config files are application level settings. Whenever you make the following call, your opening application settings:

If you have a need of creating, loading, and saving user level configuration, you have that option. To open existing user level configuration, you have two options, depending on whether you want the users local or roaming profile:

ConfigurationMAnager.OpenExeConfiguration(ConfigurationUserLevel.PerUserRoaming); // Roaming profile only
ConfigurationMAnager.OpenExeConfiguration(ConfigurationUserLevel.PerUserRoamingAndLocal); // Local (and roaming, if there is no specific roaming profile)

When accessing user level configuration, it may not exist, in which case you will need to create it. By default, all configuration files may only be written if they are application level. You will be able to open user level configuration, but when you attempt to save, the save will fail. To save user level configuration you must first change the configuration "allow" definition. This can be done in code or in the .config file:

I hope this helps explain how to access different levels of configuration with the .NET 2.0 configuration framework. I'll be including a more detailed explanation of user levels and how to create/access the neccesary settings in the final article, which I am once again working on.

HI Jon, Thanks, for the reply. I wasn't aware of some of those API calls. I will look into them so I can add them to my quiver, so to speak. Great news about you working on the final installment, I look forward to getting schooled...

Nope, not quite.The Type should be set to typeof(YourSection) and the method name should stay as just "MethodName", that is w/o specifying fully qualified name and assembly.However, the callback method is called then TWICE. For the first time the value is 0, and for the second time it is equal to the one specified in the config file.

The same happened when I used the declarative way. If the config contains "6", setting [IntegerValidator(MinValue=3, MaxValue=10)] will result in an exception. But when I set MinValue to 0, it validates.

All your efforts are very much appreciated. The painstaking efforts you have taken to write this is quite evident in your both articles. Admittedly I had never looked into configuration this seriously before. Your article explained it in such a detail and yet keeping it so simple was just marvelous.

I will sure keep looking for your next article and of course put all this knowledge to some good use.

Hi Frank. I have an idea on how you could accomplish what you need for your own custom configuration sections. This little trick, which is something I'll be discussing in detail in the third article, will only work for custom configuration sections, and can not affect existing sections.

For any custom configuration class (meaning anything that ultimately derives from ConfigurationElement, including ConfigurationSection and ConfigurationElementCollection), you can override the serialization and deserialization process and inject your own. You just need to override the void DeserializeElement(XmlReader reader, bool serializeCollectionKey) method, and perform your own deserialization. The best practice would be to define a reserved attribute for your custom configuration sections, say "dbConfigSource". The value of this attribute would be a formatted string containing database connection string and whatever information you need to find the config information in the right table/row. In your override of DeserializeElement, you check if this attribute exists, and whether the database server and data exist. If they do, you download the config information and populate your configuration section.

More best practices would be to call the base DeserializeElement first, before downloading your external configuration, to load the local copy. Once you have downloaded and parsed your external configuration, save the configuration section to your local .config file. It would be best to keep a timestamp of the last time you updated configuration in the config file, as well as one in the database to identify when configuration changed. That way, you can load your config locally if there have been no changes, and not incurr the hit to download.

I will be going into much more detail about how to override DeserializeElement, along with its counterpart, SerializeElement, in the third article. So stay tuned.

Thanks a lot for this article series, i lot of work behind and you should get some reward Anyway still I'm feeling a lit bit unsure regarding .net configuration capabilities. I wrote section handlers in 1.x - i would kill "designer" who designed this in 1.x, then they came with this new stuff in 2.0 and still not sure if it's worth using as still there this is complex and still some issues aren't solved.Currently I think that the best way is to use custom own config framework that will not change with version 3.x of .net

Configuration works the same way in .NET 3.0 as it does in .NET 2.0. Its unlikely it will ever change, as the old .NET 1.x configuration stuff also still works in .NET 2.0 and 3.0.

Also, I'd like to know what issues there are that arn't solved by .NET configuration sections. The two articles I've written so far only cover the common uses of the configuration framework, but there is much more that can be done with them.

In regards to complexity, I could write a fourth article that shows the difference between writing your own custom configuration system that has the same capabilities as the .NET Configuration framework, and using the .NET Configuration framework itself. I think you would see then, when push comes to shove, it is still much simpler and faster to write a custom configuration section than to write your own custom configuration system.

1.1: section handlers, something very awful and hard to implement => .net2.0 configuration => fine, some improvements but i think it can much better, some ideas i like are implemented in Spring/Obix project (extendable object wrappers, provider pattern could involved more intensively etc.), simply configuration is similar to OR-DB concepts.

Regarding custom implementation - I still see this as better solution based on some open source projects as those core framework parts are under general control and will not change in time so frequently and you can't guarantee that it will not change it in future. Simply I want to control core parts by myself.

I've not spoken about 1.1 not working with other versions. It's fine but it's deprecated as this has happen millions of time in past with other technologies That is why i like to have core components under my control.But this is only my opinion and i think there is no bullet-prove advice on this and even my and your choices are corrent depending on our requirements

Hello everyone. I've finally managed to finish part two of the Mysteries of .NET 2.0 Configuration. I apologize for the delay. I had to restructure the article a bit, move some things out to the third article that were too complicated, and move some other things in. I think this article will provide enough for most of you to go on until I am able to finish the third article, which will explain in detail the more complex intricacies and capabilities of .NET 2.0 Configuration (and explain how to write custom configuration sections that can be stored in external files in alternate locations).