Cracking the Mysteries of .NET 2.0 Configuration

Maximize your understanding of the .NET 2.0 configuration framework, avoid common pitfalls, and gain insight into the details of how configuration works in various scenarios and environments.

Introduction

One of the wonderful features of .NET has been its XML configuration features. In the days of .NET 1.x, common application settings, database connection strings, ASP.NET web server configuration and basic custom configuration data could be stored in a *.config file. Custom configuration sections could use a few basic but custom structures, allowing a small variety of information to be stored in a *.config file. More complex configuration, however, was most often accomplished with custom XML structures and custom parsing code. Such code could become quite complex, though and there are a variety of ways of differing performance to accomplish the same thing.

With .NET 2.0, the days of writing your own (probably complicated, poorly-performing and tedious) code to manage custom XML configuration structures are over. The custom configuration capabilities of the built-in XML configuration subsystem in .NET 2.0 have been greatly revamped, boasting some extremely useful and time saving features. With relative ease, just about any XML configuration structure you might need is possible with a relatively minimal amount of work. In addition, deserialization of the XML in a *.config file can always be overridden. This allows any XML structure necessary to be supported without losing the other advanced features of .NET 2.0 configuration.

Exposing Details

This article continues a series on .NET 2.0 configuration. This article aims to expose all of the details and inner workings of the .NET 2.0 Configuration framework to allow developers to better utilize the extensive capabilities provided. If you are new to .NET 2.0 configuration, or have not yet mastered concepts such as type validation and conversion, you should first read the previous articles, which can be found at the following links:

Please note that the code examples provided in this article are only meant for clarification of points made in the article. Compilable and downloadable code examples are provided in the first two articles of this series. The goal of this article, which is intentionally more detailed and advanced than the previous two, is not to provide compilable, runnable code examples. Rather, the goal is to expose the core underlying theory and important details of the .NET 2.0 Configuration framework. The desired result is that anyone interested in utilizing the .NET 2.0 configuration to it's fullest extent may do so after reading the information contained in this series of articles.

Topics of Configuration

Welcome to the third installment of the Mysteries of .NET 2.0 Configuration series. This article will cover the .NET 2.0 Configuration Framework in complete detail. I will be exposing little-known features, capabilities, quirks and internal workings of all aspects of the framework. Several architectural diagrams will also be presented to provide a visual anchor for the details to be discussed.

Many forms of configuration are available for modern .NET applications. Binary data, INI files, databases, XML, even arbitrary text structures. Depending on the environment, type of application, usage factors, etc. your configuration storage needs may change. For the most part, however, most .NET applications (and the programmers who write them) just need the ability to store configuration... doesn't really matter how. Most configuration tends to be hierarchical and manual editing is usually desirable. This makes XML an ideal platform upon which to build a configuration storage framework. This also makes the .NET 2.0 configuration framework very appealing to the .NET developer.

"Hierarchical" is the name of the game when it comes to the .NET 2.0 Configuration framework and in more ways than one. Aside from being an application of XML and providing a hierarchical medium within which to store actual configuration settings, the .NET 2.0 configuration framework is hierarchical in it's own right. Depending on the application context, a natural hierarchical order of multiple configuration files exists and are merged together when configuration is requested by code. This hierarchical structure to configuration files allows settings to be applied at various levels, from the entire machine, to the application and even down to the individual user or resource being requested.

The hierarchical structure of the .NET 2.0 Configuration framework can be seen in the diagram below. A few important points to note about this diagram are the order in which configuration files are merged and the contexts within which configuration is available. Depending on the context, the nature of what can be configured and how those settings are merged can differ in important ways.

In the .NET world, there are two primary types of applications: Windows Applications and Web Applications. Windows applications, or executables, have a relatively simple 4-layer configuration structure and merge process. Web applications, on the other hand, have a more complex structure for applying and merging configuration for specific locations. These two primary types of applications each create a configuration context with the rules that govern how configuration is applied.

Independent of any context is machine-level configuration. Machine level configuration applies to any application that runs within the CLR, as well as the basic setup for all other configuration options that the .NET framework itself uses. It may come as a surprise to many, but the .NET framework uses the same configuration framework discussed in this series of articles. It may further surprise many that none of the configuration capabilities observed within the .NET framework are special or exclusive... you can do all of the same things.

A quick examination of the machine.config file (found at %SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\CONFIG) will show that this is where all of the "preconfigured" or "default" ConfigurationSectionGroup and ConfigurationSection elements are defined. Things like appSettings, connectionStrings, system.web, etc. Many default settings, such as section encryption providers, default ASP.NET membership, profile and role providers, etc. are applied here. In the same directory, you should also notice commented versions of this file, a default version, as well as several default web.config files for various security levels. While editing the machine.config file can be risky, it is also the only way to apply settings globally to every application. Below the machine level, configuration splits into the two available contexts... Exe and Web.

The Exe context is available to any executable application. This includes Windows Forms, Windows Services and Console applications. Within this context, the core configuration infrastructure understands a total of four levels of configuration: Machine, Application, Roaming User and User. Each level is subsequently more specific in "context." Therefore each subsequent level can override settings defined higher up. It is important to note that Roaming User configuration is less specific than User configuration. This allows user-specific settings to reside on both a desktop and a laptop independent of each other, while roaming settings are shared between the two and available on both.

The Web context is available only to applications that run within an ASP.NET host (does not necessarily have to be IIS... any ASP.NET host will do, including the VS2005 developer server or a custom web server that utilizes System.Web.Hosting.ApplicationHost). Unlike the Exe context, which is process- and user-dependant, the Web context is location dependant. Configuration may target specific website locations either explicitly as configured in a web.config file, or implicitly as multiple web.config files are merged. Configuration on a per-user basis is generally unavailable in the Web configuration context, even if a user is properly authenticated.

The hierarchical nature of .NET 2.0 configuration provides a great level of flexibility, allowing specific users or locations to have their own configuration settings. However, those configuration settings are not isolated and duplicate settings made at a more specific level have the ability to override settings made at a less specific level. As can be seen in Figure 1, the most specific configuration files are merged into the less specific, with the most specific settings overriding the least specific. In the Exe context, User (or to be more precise, Local User) settings are most specific, followed by Roaming User (shared between two or more machines), Application and, finally, Machine.

In the Web context, merging is a little more complex. In addition to multiple *.config files merging from subsequently more specific locations (i.e. \wwwroot\siteA\web.config is more specific than \wwwroot\web.config when merged), location-specific configuration can be defined directly within a single web.config file for an application. The nuances of web.config and location merging will be discussed later in this article and in greater detail.

The .NET 2.0 Configuration framework, while providing a quick, simple means of implementing custom configuration, is fairly complex in and of itself. The configuration framework provides not only the means to implement custom configuration, but also provides the means to implement custom data validation and conversion, custom providers, exposes hooks for custom serialization and deserialization and even allows configuration to be encrypted. Metadata about each configuration element is also exposed, providing details about the whats, wheres and hows of the data that was loaded. The architectural structure of the .NET 2.0 Configuration framework is displayed in Figure 2. The specifics of each element in the diagram will be the primary topic of this article as you read further.

Hidden within this complicated looking diagram is an elegant, logical and efficient system for managing configuration. I have tried to organize the individual components of this architecture into logical groups, which form the core sections of this article:

Configuration Management:

Retrieving, storing and finding or mapping configuration files.

Configuration Representation:

Storage structures used to represent configuration sections, elements and attributes.

Configuration Metadata:

Information about where configuration came from, what context its available in and various other details.

The first concept to be discussed in detail is also, logically, the first place you start when working with configuration in your projects. Configuration management, in the scope of this article, refers to finding configuration (either directly, or by mapping to specific configuration files), retrieving configuration sections and the storage of those sections during use. Configuration files may be loaded directly, providing additional capabilities and allowing specific *.config files (vs. those mapped by default) to be loaded. Mapping to specific configuration files differs depending on the context your operating under, but allows any configuration file to be loaded provided the necessary permissions to read from the resource are granted.

The ConfigurationManager and subsequently the WebConfigurationManager, are the primary starting points for accessing .NET 2.0 configuration. The ConfigurationManager provides all of the core services required to retrieve configuration sections and the settings contained within them and also provides methods to allow configuration files to be explicitly loaded in the Exe context. The WebConfigurationManager provides additional methods to explicitly load configuration in the Web context, while also acting as a proxy for common ConfigurationManager methods. These two static classes are diagramed below for reference.

The primary functionality provided by the ConfigurationManager class, GetSection(string sectionName), has been described in detail in the previous articles in this series and will not be reiterated here. By default, the ConfigurationManager class provides implicit read-only access to configuration. Often, configuration needs extend beyond simply reading involatile settings and saving is required. The ConfigurationManager exposes several methods to allow configuration files to be opened in a more explicit context. The first way of opening configuration is by using the ConfigurationManager.OpenExeConfiguration() methods. These provide read/write access to configuration file(s) by way of the Configuration class.

There are two overloads of the OpenExeConfiguration() method. One overload takes a string representing the path of the currently running executable and the other overload takes a ConfigurationUserLevel enumeration value. The first method will append ".config" to the filename you provide and load that configuration file. It's important to note that OpenExeConfiguration(string exePath) is a very misleading method, as the filename does not have to be the filename of the .exe that is running. One of the holy grails (unattainable, as of yet... based on my internet research) of configuration addicts is available through this method, provided it's used properly. Consider the following scenario:

Requirements

Define configuration settings at the application level.

Consume code from a class library called ConfiguredClassLibrary.

Allow the class library to have its own *.dll.config file.

Problems

ConfigurationManager.GetSection() only returns sections from the main Application.exe.config file.

ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None) only opens the main Application.exe.config file.

The solution to the above problem, which is one of the problems developers most frequently ask about, is OpenExeConfiguration(string exePath). By providing a filename other than the EXE filename, an alternate *.config file can be opened. Consider this solution to the previous scenario:

Solution

Copy ConfiguredClassLibrary.dll.config to same directory as ConfiguredClassLibrary.dll.

The above scenario, which appears to be frequent enough, allows multiple configuration files (specific to any assembly, not just the primary EXE) to be used at the same time. Despite the fact that OpenExeConfiguration(string exePath) was called on a DLL file, the configuration file accessed by ConfigurationManager does not change. Any other code that must still access settings stored in Application.dll.config may continue to do so without change or conflict. A simple proof of this concept can be demonstrated by the following code:

Despite its misleading signature and name, OpenExeConfiguration(string exePath) is an extremely powerful method capable of solving one of the more frequent configuration problems. This method is also capable of loading configuration from HTTP paths as well, which is a possible scenario with ClickOnce smart client deployments. Using OpenExeConfiguration("http://someserver.com/clickonce/someapp.exe"), the *.config file for a ClickOnce deployed application may be identified and loaded from its proper versioned folder (which is automatically generated during installation and updates).

The second method, OpenExeConfiguration(ConfigurationUserLevel level) will load the appropriate configuration file for the specified configuration level. Configuration levels, available in the Exe context, allow you to specify whether you want exe, roaming user, or local user configuration. The ConfigurationUserLevel enumeration is also a little misleading in how its values were named. This can lead to a misunderstanding of what the method does and what kind of configuration to expect as a result of calling it. The real meanings behind each value are as follows:

None

Specifies there is no "user level".

No user level defaults to the primary exe configuration file.

The internal configuration path for this level is MACHINE/EXE.

Stored in \[AppPath]\[AppName].exe.config

PerUserRoaming

Specifies that the roaming user configuration should be loaded.

The roaming user configuration is a shared configuration, replicated between any roaming users systems.

The internal configuration path for this level is MACHINE/EXE/ROAMING_USER.

Stored in [ApplicationDataPath]\[AppName]\[CodedPath]\[Version]\User.config.

PerUserRoamingAndLocal

Specifies that the local user configuration should be loaded.

The internal configuration path for this level is MACHINE/EXE/ROAMING_USER/LOCAL_USER.

Stored in [LocalApplicationDataPath]\[AppName]\[CodedPath]\[Version]\User.config.

Remember that configuration is hierarchical and merged. When requesting roaming or local user configuration, that level up through machine.config are merged, resulting in the complete configuration accessible by your application for the given user level. An interesting consequence of this merging allows you to request roaming or local user configuration even when the User.config file does not exist. The Configuration object returned will contain a FilePath that does not exist and the HasFile property will be false. Any sections defined in higher configuration levels will still be accessible though and with the proper allowances, changes to those sections can be saved. Saving changes at a level that previously did not exist will create the appropriate User.config file.

Roaming and local user configuration settings are an interesting beast. Some of the more subtle behaviors of .NET 2.0 configuration take shape (or rear their ugly heads, depending on how you look at it). In spite of their ugly heads, these subtle behaviors are another great reason to choose to use .NET 2.0 configuration over a custom solution. The .NET configuration framework provides some extensive security features that allow settings, sections and even section groups to be locked down (absolutely, or depending on what level they are being accessed from). Consider the following code:

This example should open the local User.config file and get (or create, if it does not exist) some CustomSection. A value on this custom section is edited and the changes saved. The ultimate goal is to load any existing settings, or create the local User.config file with the new section and setting if it does not yet exist. This may seem simple, but depending on whether "customSection" was previously defined at the roaming, exe, or machine level, adding the section, or editing it if it already exists, may not be possible. This scenario can happen fairly often and can become quite complex when a high volume of configuration is used. Details of the causes and solutions to editing/saving configuration at user configuration levels will be discussed in the Configuration Metadata section of this article.

In addition to those just discussed, several other methods exist to open configuration files. Unlike the OpenExeConfiguration() methods, which make several assumptions about where your configuration files reside, OpenMappedExeConfiguration() and OpenMappedMachineConfiguration() allow you to explicitly specify where your *.config files reside on disk. Using these methods, you can load an alternate machine.config, load User.config files from the locations of your own choosing (vs. letting the .NET framework decide on some convaluted path), etc. When accessing machine.config, it a custom version is not required, OpenMachineConfiguration() should be used instead. More details on how to use these methods and the corresponding ConfigurationFileMap classes, are available below in the ConfigurationFileMap class subsection.

The last method exposed by the ConfigurationManager class, RefreshSection(string sectionName), is another answer to one of the questions I more frequently get asked. The numerous OpenConfiguration methods described above allow configuration to be opened read/write and changes saved. There are times, however, when saved changes to configuration to not get picked up by the ConfigurationManager class (especially in web environments). There are many ways to solve such problems, but the simplest way is to call RefreshSection with the appropriate section name immediately after saving. This should force (in most cases...in a very few instances I've had reports that calling it had no effect) the ConfigurationManager to load from disk and re-parse the specified configuration section the next time GetSection() is called.

The ConfigurationManager class, while the first step to fully accessing your Exe context configuration, is woefully inadequate within the Web context. Unlike executable applications, web applications do not have a well identified local user for which User.config files may be created. In fact, there can be no user specific configuration at all in a web application. Despite that, web configuration can be an even more complex beast when location-specific configuration is considered. Location specific configuration and how to use the WebConfigurationManager to take full advantage of it is nearly an article in and of itself. A complete discussion of the WebConfigurationManager and ConfigurationLocations, will be discussed in later sections of this article.

The ConfigurationFileMap is an essential component of the OpenMappedExeConfiguration and OpenMappedMachineConfiguration methods of ConfigurationManager. These classes allow specific paths to *.config files to be specified and the OpenMapped methods will perform all of the appropriate merging when a Configuration object is created. The ConfigurationFileMap class represents a machine configuration file mapping and is required to call OpenMappedMachineConfiguration. Additional file mapping classes, ExeConfigurationFileMap for the Exe context and WebConfigurationFileMap for the Web context, are required to load configuration beyond the machine level.

The ExeConfigurationFileMap allows you to specifically configure the exact pathnames to machine, exe, roaming and local configuration files, all together, or piecemeal, when calling OpenMappedExeConfiguration(). You are not required to specify all files, but all files will be identified and merged when the Configuration object is created. When using OpenMappedExeConfiguration, it is important to understand that all levels of configuration up through the level you request will always be merged. If you specify a custom exe and local configuration file, but do not specify a machine and roaming file, the default machine and roaming files will be found and merged with the specified exe and user files. This can have unexpected consequences if the specified files have not been kept properly in sync with default files.

The WebConfigurationFileMap allows you to configure virtual paths to configuration files, much like the ExeConfigurationFileMap allows you to configure *.config files for each configuration level. This plays into the location-dependant configuration available in the Web context and will be covered in detail in the final section of this article.

If the ConfigurationManager class is the first step along the Yellow Brick Road to the Emerald City of Configuration, then the Configuration class is definitely the second step. Any call to one of the OpenConfiguration methods exposed by the Configuratio nManager class will return a Configuration object. A Configuration object represents the merged configuration for whatever configuration user level, location, or file mappings you requested. Unlike the ConfigurationManager, the Configuration class exposes the full details of configuration sections in read/write mode. Sections and section groups may be created, removed and their security settings adjusted when accessed through the Configuration class.

After examining the diagram in Figure 5, you should see that the Configuration class exposes much more information and functionality than the ConfigurationManager does. Some of the information is the same, including AppSettings, ConnectionStrings and the GetSection() method. In addition to the GetSection() method, the Configuration class also exposes GetSectionGroup(string sectionGroupName) which allows you to load ConfigurationSectionGroup classes defined in a configuration file. The Configuration class also exposes collections of all defined ConfigurationSections and ConfigurationSectionGroups. The Configuration class also exposes overloaded Save and SaveAs methods, allowing modifications to be saved back to an existing configuration file, or to create a new one. You should already be familiar with loading and saving sections and section groups from reading the previous articles.

Some features of the Configuration class that have not yet been discussed and which are not readily apparent, are made possible by the section and section group collections. In addition to allowing you to load and save existing configuration sections and section groups, you can also add and remove section groups. This is a powerful feature that allows a configuration file to be programmatically created using code. This can be very useful when working with roaming or local user configuration that requires settings not necessary for base application functionality. An important factor to note when creating sections this way. By default, sections may only be defined at the machine and exe levels. If you need to add a new configuration section, even if the section will only be used in a roaming or local user *.config file, you must add the section at the exe level first, then modify the section settings at the user level. Consider the following code:

The definition allowances (AllowDefinition and AllowExeDefinition) are an important factor when working with multiple levels of configuration. They are two of those "subtle behaviors" mentioned previously that can complicate working with .NET 2.0 configuration. Detailed explanations of definition allowances, as well as other settings that can have similar subtle effects, will be discussed in the Configuration Metadata section of this article.

An important property of the Configuration class is the EvaluationContext property. This property exposes an instance of the ContextInformation class, which provides access to the context object and a flag indicating whether the Configuration object represents machine.config, or an application or user level *.config file. The context object exposes basic but helpful information that can be used to simplify tasks that may otherwise require more complicated logic. A detailed explanation of the available context classes will be discussed in the Configuration Metadata section of this article.

Configuration management, while the first step in accessing configuration, is far from being the most important component of the process. The ultimate goal of configuration is to represent setting structure and setting data in a simple, logical manner. The ultimate core of the .NET 2.0 configuration framework is the ConfigurationElement class. The ConfigurationElement class is the code counterpart to an actual XML element in a *.config file. It provides all the functionality required to represent a configuration settings element, its attributes, as well as a variety of metadata. The ConfigurationElement can facilitate most, if not all, of the complex needs developers require to store adequate configuration.

In previous articles in this series, we discussed how to create custom ConfigurationSections and explained how attributes and elements work in the scope of an object-model configuration system. Those articles provided a very basic overview of what configuration is and the role that ConfigurationElement plays. The ConfigurationElement is more than just a means of accessing the settings you define in an XML file, though. In this section, I'll provide a lower-level view of the ConfigurationElement class, its derivative classes and the capabilities it offers you as a configuration creator and consumer.

The primary purpose of the ConfigurationElement class is to provide access to strongly typed, validated configuration settings. Those settings may be simplistic, stored in attributes of the current element, or they may be complex, stored in child ConfigurationElements. Setting representation is only half of what the ConfigurationElement provides access to, however. Several types of configuration metadata are also accessible through the ConfigurationElement, including the current configuration context, element and attribute locking information, XML source information and a list of possible parsing errors. The ConfigurationElement class also provides several hooks into the serialization and deserialization process. Metadata and serialization will be discussed in detail in the two sections following this.

The .NET 2.0 configuration framework allows a configuration "specification" to be defined in one of two ways: declaratively and imperatively. The declarative method is accomplished by placing attributes on class properties that describe how that property corresponds to an element in an XML file. The imperative method is accomplished by predefining the properties programmatically in a static constructor. In the end, both methods accomplish the same thing, but how they accomplish it is quite different. The declarative method is generally considered the simpler of the two and requires less work on the programmers part. However, this simplicity of implementation comes at the cost of higher processing whenever a configuration element is refreshed.

The features of the ConfigurationElement class can generally be split into four groups: parsing (serialization and deserialization), configuration infrastructure, settings data and metadata. If you glance at Figure 6, you might also notice that the vast bulk of this class is non-public. Most of the methods and properties of the ConfigurationElement are only accessible to derivative classes, leaving only locking information, indexers and the IsReadOnly method public. The majority of the protected members are also marked as virtual, allowing a great deal of extensibility beyond what has already been discussed in previous articles.

Sections dedicated to discussing parsing and configuration metadata can be found later in this article and defining & accessing configuration settings data has been discussed in previous articles and will not be reiterated here. While the final aspect of the ConfigurationElement class, infrastructure, is seldom needed, yet it can be the solution to those many small problems. The infrastructure methods of the ConfigurationElement we will be discussing are as follows:

Despite their woefully inadequate documentation, the Init() and InitializeDefault() methods have some specific and often necessary uses. As part of the infrastructure of the configuration subsystem, these methods, while exposing no apparent value, are called internally during serialization and other processes. Since data in a ConfigurationElement is cached in memory, possibly for long durations, initialization may occur more than once due to saves or resets. Basic initialization in a constructor is sometimes insufficient to ensure that the internal state of a ConfigurationElement is properly maintained throughout its lifetime. To ensure that your ConfigurationElement's internal state is maintained and updated at the proper times, initialization should be performed in one or both of these methods.

The Init() method is the most commonly called initialization method and is generally where custom initialization should reside. When implementing the Init method in your own classes, care must be taken to properly track whether the ConfigurationElement has been initialized or not. Re-initialization of a single instance of ConfigurationElement is rare, however Unmerge operations (see discussion below) and resets often duplicate ConfigurationElements and repopulate them from the original. When the initial state is important to subsequent operations, implementing the Init() method is essential. Knowing when this method is called is important in understanding how to properly implement it and the primary reasons it may be called are listed below:

During section deserialization

During preliminary element creation, before values are set

Will always happen during serialization of a ConfigurationSection, before an Unmerge operation

May happen when a ConfigurationElement is first created by a parent element property, before being populated with data.

When new elements are internally created and added to a ConfigurationElementCollection

Can happen during deserialization, an element reset, or through an Unmerge operation

When elements are manually added to a ConfigurationElementCollection

While a theoretical discussion of how and when the Init() method is called is important to filling in holes and answering questions, it doesn't answer every question. For those who need a more complete understanding of the methods capabilities and possible usage, I recommend using Reflector to examine the KeyValueConfigurationElement and KeyValueConfigurationCollection classes in System.Configuration. These two classes provide the clearest example of when initialization may happen at times other than object construction and one instance of how Init was implemented in the .NET 2.0 framework.

Unlike the Init method, which may be called at any time in response to a variety of triggers, the InitializeDefault() method is only called directly in one instance. Whenever a method is Reset, which usually happens in response to an Unmerge operation, InitializeDefault is called of the ConfigurationElement is the root element (or sole element) of a ConfigurationElement hierarchy (generally when it's a ConfigurationSection). Reset is another one of those overridable infrastructure methods, however and you may wish to call InitializeDefault on your own in response to a reset.

Modified and ReadOnly States

The modified and read-only states are a fairly self-explanatory and common object state. In regards to the configuration framework, however, it helps to understand when modified and read-only states may be set and reset. The modified state is set true only by the SetPropertyValue() method and set false only by the ResetModified() method. The ResetModified method is usually called when a configuration section is saved.

The SetReadOnly() method is called during initial configuration section setup and is almost always called. This method is the reason why one of the most common issues regarding saving configuration, read-only exceptions, arise. Except for a few situations, the only way to access a configuration section and its elements in read-write mode is to directly open a configuration file through a call to one of the Open*Configuration() methods on the ConfigurationManager or WebConfigurationManager classes. Another instance where this method is called is through overridden implementations of object ConfigurationSection.GetRuntimeObject(). The only two known uses of this method are on the AppSettingsSection and ConnectionStringsSection classes, for exposing read-only versions of the AppSettings and ConnectionStrings collections through the ConfigurationManager class (all configuration data accessed through ConfigurationManager is always read-only).

Listing Configuration Errors

The ListErrors() method has only one use and a simple one at that. Internally in the configuration framework, it is used to supply a list of errors (ConfigurationException instances, to be exact) to an exception that is later thrown when something goes wrong. You may override this method to add your own list of errors to the collection. This list is populated only in two instances: during section deserialization (parsing errors are generally not thrown individually, rather collected and thrown wrapped up in a ConfigurationErrorsException), or through a call to the ElementInformation.Errors property. The most common time errors may be added to this list is during custom deserialization, which will be discussed in detail later on.

Setting Property Values

The SetPropertyValue() method also has limited custom use, but a very specific one that can solve some problems. Generally, this method is used internally whenever a configuration property that you have defined in a custom configuration class is set. During common usage, the third parameter is set to false, ensuring that any locks applied to a configuration element are taken into effect before the property value is changed. Some situations, however, may require a property to be set regardless of locks that may be applied (such as during a reset). The best example of a custom use for SetPropertyValue is in the AppSettingsSection.Reset override, for those interested in a practical example.

Resets

An element reset is a fundamental process that restores lock information to defaults and reapplies inherited locking information and sets property values to those previously loaded from the configuration file. A reset call to a ConfigurationSection will cascade the call to all child elements, etc. recursively, forcing all elements that make up the configuration section to be reset. For the most part, understanding how a Reset works in inconsequential. Configuration is generally used read-only, or when configuration must be read/write, most needs are basic. However, intercepting a Reset call and modifying it is often necessary when data stored in configuration properties is cached in a different form (i.e. an encoded string is decoded and cached upon request, requiring the cached copy to be dropped when the section is reset). The base workings of a configuration element reset should generally not need to be modified, but as in the case of dropping manually cached data, sometimes it must be augmented to ensure an element is fully and properly reset. The Reset method is called in only two instances... during an Unmerge operation, or during the various methods of creation of a configuration element.

The Unmerge Operation

The Unmerge operation is another fundamental process that is performed when a configuration section is saved. You should remember from earlier in the article that configuration files are hierarchical, with more specific files merging with less specific files. This merging provides a single unified view of all configuration settings from the machine level all the way through a specific users configuration (or in the case of a web environment, through a specific web site or web application). Configuration sections defined in a lower level may have their settings changed in a higher level. Collections of configuration elements may have their children removed, changed, or even cleared by a higher level. When working with configuration in an application, this merged view makes consuming configuration a very simple task and removed a lot of the need for a complete understanding of where configuration settings come from. However, when the time comes to save configuration settings that have been modified during runtime, this merged view must be properly unmerged.

Considering that, depending on which level of configuration your working with and saving, you could be adding, removing, clearing and changing settings that may or may not yet exist at the level your working at (yet should be saved back to the current level), the unmerge operation is fairly complex. The specific details of how an unmerge operation will not be discussed here (as I am not entirely clear on the precise workings myself), but suffice it to say, this process handles the separation of inherited settings from lower levels from the current settings of the working level. What is more important is the understanding that many of the methods described above are called during an unmerge, as current elements are often cloned, reset or initialized and repopulated with only the appropriate data that should be saved. What that data is depends on the ConfigurationSaveMode chosen when Configuration.Save() was called.

ConfigurationSection

The ConfigurationSection class is a derivative of ConfigurationElement, meaning it may behave the same, containing its own attributes and child elements. In addition to inheriting all of the functionality and core behavior of a ConfigurationElement, ConfigurationSection adds its own features specific to a root node of a primary part of a configuration file. The only public feature that ConfigurationSection adds is the SectionInformation metadata property. This property, which will be discussed in detail later, provides some detailed information about a configuration section, including access to its raw XML. The ConfigurationSection adds a few new protected methods, including DeserializeSection and SerializeSection.

The DeserializeSection method simply calls the base ConfigurationElement.DeserializeElement() method. In derivative classes, this method may be overridden to provide custom deserialization processes. The AppSettingsSection overrides this method to provide a more refined version of external configuration files through the use of the file="" attribute. A detailed discussion of custom deserialization will take place later in this article. The SerializeSection method is a bit more interesting. This is the method that is called when Configuration.Save() is called and performs validation of data then an unmerge, after which the element containing the unmerged version of the configuration section is serialized. This method may be overridden in a derivative class to provide advanced multi-file configuration, which will also be discussed later in this article.

ConfigurationElementCollection

The ConfigurationElementCollection, like ConfigurationSection, is also a derivative of ConfigurationElement. The ConfigurationElementCollection has been discussed in previous articles, so a detailed discussion of it will not be covered here. An examination of the class diagram should provide a clear view of the features this class provides to derivative collections, most notably the Base* methods. One method that should be noted is the OnDeserializeUnrecognizedElement() override. This method is called when the ConfigurationElement's deserialization code encounters an element name that does not directly correspond to a ConfigurationProperty. Since the ConfigurationElementCollection class allows you to customize the add, remove and clear element names, an override of this method is necessary to properly handle such elements. I recommend those interested in how to handle unrecognized elements, for whatever reason, to review this methods code using Reflector.

In addition to the classes that directly correspond to configuration elements in a *.config file, there are also some basic organization classes that loosely correspond to section groups. Unlike a ConfigurationSection, a ConfigurationSectionGroup is not directly mapped to a configuration element after it has been loaded and is not directly saveable. A section group and the related child collections, are populated directly from a configuration record, which is a class that is part of a hidden framework of factories, records and various support types used to process raw XML data and generate configuration sections. Two other classes, ConfigurationSectionCollection and ConfigurationSectionGroupCollection, are processed in the same way as ConfigurationSectionGroup. Considering that the bulk of the code that processes the XML for these elements and populates them is hidden, a detailed discussion of them is probably not required. They simply provide an organizational structure, allowing configuration files to be cleaner and easier to maintain.

Configuration, if it requires a system such as that provided with .NET 2.0, is usually fairly complex. Such configuration is usually hierarchical, relational and fundamentally critical to supporting a flexible, reusable, easily maintainable architecture for one or more applications. Often, in such complex application architectures, more information is required about configuration besides just the configuration settings themselves. This additional information is either required to support proper implementation of the configuration itself, or possibly to support use of that configuration by the code that consumes it. Thankfully, the .NET 2.0 configuration framework is riddled with metadata, exposing every last conceivable detail about the configuration your working with.

Early in this article, I covered the basic hierarchical structures of configuration in two possible environments: configuration for an executable and configuration for a web application. These two configuration environments define the context within which configuration is loaded, merged, saved and unmerged. The context is more than just a context, however and specific details about the current configuration context can be accessed through the ContextInformation object. The ContextInformation object is exposed in only two locations... by the Configuration object and by the ConfigurationElement object (and all its derivatives). The ContextInformation class is simple, providing access to a more detailed context information object (HostingContext), as well as a flag indicating whether the current configuration object is the machine-level configuration. Note that Machine-level configuration is inherently context-less, existing and behaving the same in both the Exe context and the Web context.

The ExeContext class is a very simple class, providing only information about the path of the current executable application and the current UserLevel for which the configuration object was loaded. This information will usually be known by the time a Configuration object was loaded, but if the context is accessed through a ConfigurationElement object, the ConfigurationUserLevel may not necessarily be known without accessing this object.

The WebContext class provides a bit more detail than the ExeContext, as the configuration hierarchy in a web environment can be more complex. Through the Web context, you have access to the site name, virtual path, application path and location sub path that the configuration object or element was loaded for. In addition, you have access to the WebApplicationLevel that the configuration file itself resides at. Depending on whether the configuration is being accessed by a root page, a page in a child directory, or from a child web application, the actual *.config file may reside at a higher or lower directory level than the code using that configuration.

Of all metadata classes in the .NET 2.0 configuration framework, the ConfigurationProperty should be the most familiar to those who read the first two articles in this series. This class provides the means through which we define the actual settings that can be configured within a given configuration section and its child elements. Despite the fact that it is familiar, it should be clarified that the ConfigurationProperty is indeed metadata about a configuration element's settings and does not directly represent the XML stored in a *.config file. Aside from use in custom configuration classes to define what settings should be available, the ConfigurationProperty class is only used internally by the configuration framework.

An interesting note about the ConfigurationProperty class is that it provides access to the only two non-metadata, non-configuration support types used in the .NET 2.0 configuration framework. Each configuration property may be associated with one type converter and validator, to facilitate conversion between natively typed data and a string and to validate the property value once it is in a native type. These two support types provide a great deal of flexibility in the configuration framework, enabling specific string structures to be stored in an XML file and later converted to native, possibly complex, .NET types upon deserialization (and back to strings upon serialization).

Of all the metadata classes provided in the .NET 2.0 configuration framework, the ConfigurationElementProperty has to be the oddest. This class exposes only a single property, Validator, which one would think could have been exposed directly. Regardless of its oddity, the ConfigurationElementProprety provides a ConfigurationElement with direct access to the validator assigned to ensure that elements data is correct. During both deserialization and serialization, an elements validator is called to ensure that invalid data is not read or written at any time. The ConfigurationElementProperty is generally assigned during construction, when the internal configuration management framework reads a *.config file and generated ConfigurationElements from the defined properties.

Unlike the ConfigurationElementProperty, ElementInformation provides some very useful metadata about a configuration element. To access the ElementInformation object you must reference the ConfigurationElement.ElementInformation property, as this is the only location it is exposed by the framework.

This object provides some basic flags indicating whether the element is a collection element (IsCollection), whether it is locked (IsLocked) (can only be read and not modified in any way) and whether the element is present (IsPresent) in the file. The last property, IsPresent, may seem a bit odd at first. For the majority of the time, configuration will only be read and that will almost always mean the configuration element was read from the configuration file. However, when you are modifying a configuration file, it is possible to programmatically add elements, collections, even define and add whole configuration sections. When a section has been added but is not yet saved to the *.config file, IsPresent will be false. The ElementInformation object also provides two other useful little tidbits of information: Source and LineNumber. The Source property returns the path and filename of the *.config file on disk that the element was read from. When using the configSource attribute, or the file attribute of <appSettings />, the Source property will reflect the external configuration file. The LineNumber property will return the starting line number in the Source file that the element was read from.

In addition to some basic flags and file information, the ElementInformation object provides two collections: Properties and Errors. The Properties collection exposes a list of metadata about each property defined for the element. This PropertyInformation class is similar to the ElementInformation class, providing some basic flags and source file information about each individual property, as well as some additional details specific to a configuration property. It is possible to check the what the default value of a property is and compare it to its current value, a useful little feature. Most of the information exposed by PropertyInformation should be well-known to anyone who has worked with the ConfigurationPropety class before and to anyone who has read the previous articles in this series.

The last collection exposed by ElementInformation, Errors, is another useful little gem. Generally, during deserialization, a hard exception is not immediately thrown when a parsing error occurrs. Instead, a ConfigurationException is created and added to a collection, which is then later exposed by a ConfigurationErrorsException that contains all of the errors that occurred during deserialization. The Errors collection of ElementInformation provides access to that same list of errors. Depending on the exact nature of the errors that occurred during deserialization (or serialization, or possibly other activities performed with a configuration element), the list may not be complete and additional issues may exist. However, it can be useful in determining what happened with configuration at runtime, allowing automatic resolution or notification of a configuration issue.

The SectionInformation is another very useful metadata object. This object provides a considerable amount of allowance and locking information, as well as providing some interesting encryption and loading mechanisms. For the most part, the allowance and locking properties of this class can be set in XML using various attributes. Configuration section allowances and locking is an important concept that is used fairly extensively in machine.config and which may also be the root cause of many odd issues one may run into when changing inherited settings or programmatically editing them.

Credits

Before I finally close this series (or at the very least, these three articles...there may be more), I feel I must offer credit where it's due. I've researched .NET 2.0 configuration for at least a hear and a half and it's been great fun learning and writing these articles. None of this would have been possible, however, if it weren't for a little utility called Reflector, written by one Lutz Roeder. As someone who has to know all the details about everything, I would be completely lost in the world of .NET software development without this perfect little gem. It has offered me with a clearer, deeper understanding of the .NET framework than any amount of documentation could ever provide. Not to mention the fact that I could very well still be ignorant of the .NET 2.0 configuration framework to this day if I hadn't been poking around the .NET source code.

Second, I would like to thank everyone who has posted questions to previous articles, or sent me questions through email. Many of your questions I was able to answer strait off from prior knowledge, but many others provided interesting challenges that forced me to dig deeper and deeper into the .NET 2.0 configuration source code to figure out the answer. Without the continual challenge to provide an answer, I would have never found them. With hope, this article will answer any future questions that may arise, but for those adventurous types who like to live on the fringe, I always welcome the company.

I would also like to thank the CodeProject Staff for helping me maintain these articles and for their hard work editing them. I'm a very verbose individual and I can only imagine how much work it is to keep my ramblings up to snuff. Work on these articles is sure to continue on into the future and I have to thank the CodeProject staff for being available to post future updates as well. I would also like to thank CodeProject editors for choosing both the first and second articles as Editors Choice when they were first posted. Their confidence in me as an author has helped inspire me to continue working on this series and there have been times when I wasn't sure I had the energy to continue.

Finally, I must thank Microsoft for providing the .NET 2.0 configuration framework in the first place. After years of toiling in futile attempts to create my own reusable, simple, flexible, type-safe configuration frameworks, (and often restarting from scratch when a brick wall suddenly appears out of the mist) I no longer have to bother. The configuration framework provided by Microsoft with .NET 2.0 is wonderful and provides everything I could hope for in terms of configuring one application after another. The more I learn of the .NET 2.0 configuration framework, the more it shines as one of the most polished aspects of .NET 2.0 and definitely one of my favorite tools.

Article Revision History

This article is currently undergoing heavy, active editing. Be sure to check back often to see new sections and updates. During this active period, it is possible some content will be changed or sections rearranged. I apologize if that poses an inconvenience, but many people have been requesting this article. Considering the scope and detail of this article, I thought it best to post it in pieces as I refined and finalized sections.

1.1 [08.09.2007 11:31 PM] - First update to fix some grammatical and spelling errors and to add the Configuration Representation section. Added a credits section to thank those who have helped me along.

1.0 [07.21.2007 12:04 AM] - Original article, glorious errors, omissions and all! After many months of research, other months without a chance to research, additional research, revising, editing, more revising and revising, yes revising and more research, yes, yes research *pant* *pant* ...Hope you enjoy and benefit! Stay tuned for more...lots, lots, lots more!!

Share

About the Author

Jon Rista has been programming since the age of 8 (first Pascal program), and has been a programmer since the age of 10 (first practical program). In the last 21 years, he has learned to love C++, embrace object orientation, and finally enjoy the freedom of C#. He knows over 10 programming languages, and vows that his most important skill in programming is creativity, even more so than logic. Jon works on large-scale enterprise systems design and implementation, and employs Design Patterns, C#, .NET, and SQL Server in his daily doings.

Comments and Discussions

This was the only useful documentation of the .NET configuration framework I found so far in the internet. Extremely helpful! It's a shame that Microsoft doesn't offer something similar in the MSDN documentation... :(

I would like to leverage the configuration framework but I have to load particular values, like webservice url and certain site altering settings based on a value on the URL. Is this still a good use of the framework or should I roll my own?

I stumbled uppon your articles while trying to solve our deployment problems - they offered some help, but it seems I'm still missing some things.
I have a rather complex solution which consists of some services and a couple of seperate web application.
My naive idea was to create a deployment structure like this:
Solution folder
-Service1 folder
-Service2 folder
-Service3 folder
-Web1 application folder
-Web2 application folder
-Customer settings xml file

Customer settings xml file could then include sections:
- Solution settings section: parameters common for all services/web applications
- BackEnd settings section: parameters common for backend services (like ConnectionStrings, mail settings etc)
- Service settings section: each service has it's own specific settings section
- FrontEnd setting: parameters common for all front end applications (like backend location)
- Web application settings: separate sections each web application.
The general idea is, that all of the setting are in the same file and on the separate location

I don't know how to solve following problems:
- single file (configSource can not be in folder above the deployed application)
- how to include web sections into a single file (like SessionState setting and Forms setting)

Anyone has any ideas how to solve above mentioned issues? Or maybe reccomend a better solution?

These series of articles is amazing. Helped me a lot to construct a few custom configuration sections to my company applications. I have a few common properties and also collections of complex objects in my configuration, and we are currently using only read only access.

We are facing an issue that happens with no specific reason that stops the application from reading the config section, throwing an NullReferenceException. After we recycle the IIS process or restart it, the problem is solved. I've already reviewed the code many times and compared to the examples in the article, and it's implemented exactly the same.

Here's a sample exception we are logging:
Exception : Object reference not set to an instance of an object. at System.Configuration.ConfigurationPropertyCollection.get_Item(String name) at System.Configuration.ConfigurationElement.get_Item(String propertyName) at SearchEngine.Configuracao.Navegador.get_Campo() in C:\ProjetosVSTS\Site\SearchEngine\Configuracao\Navegador.cs:line 131 at SearchEngine.NavigatorHelper.ObterConfiguracaoNavegadores(Vertical vertical) in C:\ProjetosVSTS\Site\SearchEngine\NavigatorHelper.cs:line 29 at SearchEngine.NavigatorHelper.ObterNavegadoresQuery(Vertical vertical, List`1 nomesNavegadores) in C:\ProjetosVSTS\Site\SearchEngine\NavigatorHelper.cs:line 210 at SearchEngine.MotorBusca.ObterIQuery(ParametrosBusca parametros, Vertical vertical) in C:\ProjetosVSTS\Site\SearchEngine\MotorBusca.cs:line 474 at SearchEngine.MotorBusca.ObterResultadosQueryCarros(ParametrosBusca parametros) in C:\ProjetosVSTS\Site\SearchEngine\MotorBusca.cs:line 492 at SearchEngine.MotorBusca.ExecutaBuscaCarro(ParametrosBusca parametros) in C:\ProjetosVSTS\Site\SearchEngine\MotorBusca.cs:line 96 at carros_home_autos.ExibeNavegadores() at carros_home_autos.Page_Load(Object sender, EventArgs e) at System.Web.UI.Control.OnLoad(EventArgs e) at System.Web.UI.Control.LoadRecursive() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

The code is just trying to get a property from one of the config objects (Campo property of Navegador object) but the exception is thrown.

I tried to check if this is related to any code that is not thread-safe but found nothing to update and correct the issue.

That would have caused the problem. If you did override the Properties property, then I can't say. You linked a thread about a race condition. Does this error only occur during high load, or does it occur consistently under any load? If it occurs under any load, then I would doubt it is a race condition.

I would need to see your configuration file as well...there may be some invalid or missing data in there that is causing the problem.

This issue does not happens consistently. We are still trying to figure out with our IT department if the server was on a heavy load when the exception ocurred, but it's something that occurs now and then.

On the other hand, I think that the config file is well formed, I spllited a piece of it so you can take a look, I did not posted the whole file, because it's a bit long.

So, I am a little confused here. Is this configuration section properly registered in your app.config file, and defined in the same file? I see the xml header at the top of your example. The .NET configuration system is designed to pull all configuration from your executable's (or web sites) configuration file. In the case of an executable, you would need to put this configuration in the [ExecutableName].exe.config file. If it is a web site, you would need to put it in the web.config file.

Ok. Well, it may be that race condition. It seems that is a relatively recently discovered issue, and Microsoft probably has a fix for it by now. I would contact Microsoft and reference that thread, and see if they can send you a hotfix. Given all the info you have provided, I would expect it all to work. Everything seems to be in order.

Ok Jon. I've tried to research deeper into the subject but everything seems right with the app. We will try to set up monitoring on the webserver to check the status of the server when this issue occurs. Also, will try to reach MS Support and check if there is a fix for this issue too.
Thanks for your help and patience on this issue.

This series of articles are very well written and have been very helpful to clarify this topic. It is truly remarkable that this stuff is so poorly documented from Microsoft and that it takes someone to sleuth out how best to use these classes as if it were a piece of found alien technology without the benefit of a user manual. Your work has saved me a great deal of time and frustration and served as an excellent example of how to model my own configuration objects. I just wanted to thank you and tell you how much I appreciate you having taken the time and effort to prepare this information and share it with everyone.

Hi Jon. Great article, helped me a lot!
I'm using the openExeConfiguration to let my DLL read its own configuration file.
That works perfect, but now I must implement a service reference into my DLL, which also should read this config file (but it searches default for app.config..). Do you have a solution for that?

Sorry for the late reply. I am hoping you figured out a solution to this by now, but if not... Could you provide me with a little bit more context? Why does something that is not part of this DLL need to access its configuration? If the configuration is shared beyond the scope of the DLL, is it really best to give the DLL its own configuration, rather than putting it in the global configuration file (app.config)?

It looks like you did some great work here - I spent the last couple of days digging up just about the same information, and my stumbling upon the "WebConfigurationFileMap" helped me find your article. I'm looking to have a solid solution for being able to control the configuration settings dependent on our web environment (development, staging, and production) without changing the web.config but still controlling custom modules that access the ConfigurationManager and WebConfigurationManager directly. (This will be especially useful when changing providers for third-party CMS's without having to modify our web.config.) My department is planning to use a VCS-based deployment system, so we can't really use web deployment projects (which, last I checked, don't help with custom configuration sections anyway), and your planned section on "Web Configuration and Locations" looked like it would fit our requirements exactly. I'm wondering whether you found the solution but never got time to post it, or discovered that the framework wasn't flexible enough to allow for a custom file map, or if maybe you got distracted and couldn't continue.

I had actually researched every section that I had planned to write in this article, I just ran out of time to write it. I haven't been able to find time to finish it yet, either.

As for using WebConfigurationFileMap to map configuration, from what I could discern, as long as the paths are part of the "virtual" file hierarchy that make up a given web site, it should be possible. I don't think it is possible to map outside of the physical root of the virtual tree. Mapping configuration files to alternative, "parent-pathed" locations is pretty much the only weak spot I have found with .NET configuration. While 95% of the time the default locations for configuration files is adequate, there is that 5% (or around there) of the time that you need to store configuration in an alternate place. Currently, the .NET configuration framework does not provide any direct support for that.

You could add it using a custom configuration section that implements its own custom serialization/deserialization overrides, which pull the actual configuration from some file you specify in an attribute (similar to the AppSettings sections fileSource attribute.) To move all of your configuration out to some shared location, however, you would have to do away with as much built-in .NET configuration as you could (i.e. ConnectionStrings)...the rest you would have to leave in a default location.

Thank you for writing such a superb series of articles: if only the entire framework were document in such a way.

One question to which I didn't spot the answer: is it possible to add ConfigurationSections (or Groups or ...) dynamically to the config file? My app has addIns and I'm trying to work out how to enable them (the addins) to save and retrieve settings. I couldn't define all of the ConfigurationSections up front in the config file because I won't know what they are?

As far as I can see, the only way is to update the config file everytime a new addin is added and write a mechanism to parse in the old data. Am i right?

I've done this before, but unfortunately, my code is off in a subversion repository I don't have access to right at this moment.

I know the following steps were followed:
1. Open the configuration file using ConfigurationManager.Open... so that you have a Configuration object.
2. Access Config.Sections and add your new ConfigurationSection to the list.
3. Save out your Configuration object.
4. Call ConfigurationManager.RefreshSection with the path to your configuration section as the parameter so that it loads in properly.
5. Access it from ConfigurationManager like you're used to.

Of course, you can use WebConfigurationManager just like you would ConfigurationManager if you're in a web context.

Great article. I have one issue though. When I update some configuration and save it, the configuration file is updated with the correct data but also the layout of the configuration data is modified. Some of the comments will also disappear. Is that a behavior which can be changed, meaning is there a way to only change the actual data content when saving without having the whole file reformatted ?

I am not sure if this behavior can be changed or not. My guess, given what I do know about the framework, would be no. It might depend on exactly how you save, however...as you can save only changed sections, or force saving of the entire config file. My guess, however, is that any part that is saved will loose any custom formatting and comments. One way to get around this might be to externalize your configuration using the configSource attribute. Only save external files, and leave the root file alone...and any customization in the root should (cross fingers) be kept in tact.

I have tried (and failed) to replace the predefined handler ConnectionStringSection with a custom class (that extends the connection string elements with a 'host' attribute).

Doing so is not straight forward, since ConnectionStringsSection is sealed, so you must start with deriving from ConfigSection. After all I was possible to create a custom handler that works via ConfigurationManager.GetSection().

It would be much better to call it via ConfigurationManager.ConnectionStrings[], so that all ASP.Net Providers and DataSource Controls etc. keeps working; but there are some additional hurdles:

First it seems you cannot change the handler for a section if already defined in the config hierarchy. So the replacement must either be done in machine.config or the default section handler for the connectionStrings section must be removed from there (and inserted in every web.config... instead). At least this was my observations.

After I came to the point, when the section handler was finally getting loaded, the following exception was thrown (translated from German):
ConfigurationErrorsException - "Configuration Section \"connectionStrings\" contains an unexpected declaration."

Do you happen to know, whether it is possible at all to replace the ConnectionStringsSection or any other predfined section handler class? And maybe you even have tried that already before?

Sorry to say, it is not possible to replace the predefined ConnectionStringsSection with your own implementation. That is an integral part of the .NET framework, and it is hard-coded into the ConfigurationManager. I wouldn't really recommend trying to replace an integral configuration component like that, either. If you need to add additional configuration information for connection strings, you should just create another config section that is similar to ConnectionStrings, and simply map by similar name. If you have a connection string named "Test", use the same name in your section so additional information can be accessed by a common key.

The configuration framework is a low level system that is pervasive accross the .NET platform. The consequence of it being so low level is that by necessity it is tightly coupled to the underlying configuration files. Its best to work within the constraints of the framework than to try to bypass them, in my experience. You'll be much happier in the end, not having to fight the framework to accomplish something fancy, when something simple would suffice.

The settings file in the project properties offers another way of managing configuration data, which seems completely separate from the framework you're discussing here even though the data is in the same files. Which method is recommended? I'm inclined to use the project settings, as it uses a wizard to generate type-safe wrappers. However, if I want to do something like change the file mappings as you describe above, this method doesn't seem as flexible. Do you know why there are two different ways of managing configuration data, and what are the best ways of reconciling them to get the most benefit?

Sorry for the extremely late reply. To answer your question with the best answer I can think of. Visual Studio contains a lot of easy to use tools for novice programmers, which I think are largely geared towards the VB.NET crowd, but also for new C# developers. The Settings and Resources editors are some of those tools, and provide a nice, convenient, simple entry into basic configuration and settings. However, as you noted, they are relatively inflexible, and have a generally fixed schema.

If your needs are basic, I would use settings. Settings build on top of the existing configuration framework, and are just a custom configuration in themselves, albeit a relatively simple one. However, any time you need to have more control over your configuration hierarchy, particularly when you have nested levels and collections, I would use a custom configuration section.

I would like to write a program (controller) which edits app.config of another programI also would like to use the knowledge got from you articlesBut I have the problem. Suppose the app.config is like this

I haven't tried the following, but perhaps if the configuration class is in it's own DLL that both EXE's share you can then load the configuration data. Give that a whirl... I'm curious too on this and plan to try it out soon.

Thank you for reply.
Yes, if a class is in shared DLL this approach will work an there is no problem.
I think there must be some mapping between class name written in app.config and object in the application.

I came up with the idea of each element having a HandlerType which will manage the custom configuration for each type.

And on MyConfigElement class handle the unrecognized attributes and elements thru this handler. This is the basic idea, however I realized that the reading of the config file is done at application start so probably the handler would have to cache the unrecognized attrib's and elements and then my class would have to read it afterwards.

I've also read there is a IgnoreSection class that can be overriden to have the Configuration framework not throw exceptions. But I'm not so sure I want this because I still want the xml structure to be validated somehow.

Thank you for the great series of articles, they really helped me A LOT!

Now comes the "however"!

I see that the configSource option does not work for the <runtime> section of a configuration file. The handler for <runtime> is declared in machine.config as System.Configuration.IgnoreSection - which, if I understand correctly, means that this section is not handled by the System.Configuration types.

I would like to store the contents of the <runtime> section in a separate file - like as if I had configSource there. I cannot do this, because this section is not read via the System.Configuration mechanisms, and I don't know where and how this reading is performed. Probably, somewhere it is read using an XmlDocument parser, or parsed "manually"...

Do you know what and when reads the contents of the &lt;runtime&gt; section? I would appreciate any information.

First, I just want to say what an amazing set of articles you have put together here. I hope you are writing your book on this so I can buy it in a heartbeat.

As for a couple questions I had...
1. I am a little confused over this new subsystem that surrounds the applicationSettings/userSettings/ClientSettingsSection/integration with VS 2005. There appears to be some information related to SettingsProviders and the hint of creating your own. I see that there is a LocalFileSettingsProvider documented that writes configuration to config files. I was wondering how all this plays into what you wrote. It seems like it is something that non-web site projects (Win Forms etc.) use to propagate settings. Is it just a special enhancement Microsoft added to ease some of the configuration with windows apps or can the web use this (and should it)?

2. I am currently working for a client that has a web site that someone wrote a section of configuration using pure xml files. He reads the raw xml and populates a strongly typed class using XmlReader classes etc. I would like to change this to utilize the .NET configuration logic in these articles (mostly for the type validation and conversion and because it is very easy. I can really chop away some existing code.) My issue is that the client wants us to set up a second administrative web site that will load these settings and allow the user to modify the settings and save them back while the main web site "uses" (reads them only). I was thinking of trying to keep the settings in a database and have both apps pull from that, but I can only find information about creating a ProtectedConfigurationProvider to override the Encrypt/Decrypt methods to achieve this. I was wondering if there was a better way (maybe create a SettingsProvider, but I don't see where that plugs in and it seems to be aligned for applicationSettings over general configuration). Do you have any ideas on this? Basically the two challeneges are storing configuration outside config files (another storage source) and if using an alternate storage medium, how would the caching work (i.e. changes to the config stored in a seperate medium and how those settings get propagated to the application).

As to question #2: I have a similar requirement to allow an Admin user the ability to modify configuration settings of a Windows Service application via another WinForm controller application. I struggled using Application-Scoped Settings features of .NET to fullfill this requirement and in the end it seems I needed to replace the LocalFileSettingsProvider with a custom provider.

The .NET Settings features seem to use the principle that the end user shouldn't have privileges to modify anything in the program installation directory including Application-Scoped settings - hence User Scoped settings. However, I have the requirement to modify Application-Scoped settings in the installation directory (occasionally by an Admin) from a user interface. Some have recommended storing these types of settings in Isolated Storage... after reading Jon's articles (3 times )I'm moving towards using ConfigurationSections instead.

Jon: this appears to be a common theme and would be a nice example in your proposed book of how to allow Admin users the ability to change ConfigurationProperty settings for a target application, but end-users can only view them.

This is not the first article you write and that I am interested in.
I respect your work and the knowledge that fits in your head.
I admire the idea of sharing this knowledge over the web and the fact you find time to write such articles.
But honestly, in the real life situation, I mean when you have a project, a planning, deadlines, and people shouting behind your shoulder, it is not realistic to implement solutions as complicate as the standard .Net configuration sections.
More than that, when you look at all the codes major points (at least) should be considered:
1) If you choose this configuration solution, it is written in the stone
2) You have a lot of stuff (attributes, base classes preventing you to have your own hierarchy) around your actual configuration data
3) You have many hardcoded strings and we should all know, now in 2007, that hardcoded values are not acceptable.
I know many people will shout at me, but in real life, at the moment I need configuration settings, it must be up and running in one hour for the first approach and must evolve allowing data validation (written later if needed), relocation, reviewing and so on.
You will then ask me a workaround?
At first glance:
1) A simple configuration string in a simple App.config containing an Xml representation of my configuration data.
2) Once you have Xml, you may de-serialize it as you want.
3) If you configuration is located else where, simply load the Xml another way (other file, other location, from a database, from a service…)
I repeat I respect your knowledge, but losing your time on a small piece, you loose it for its usage, the APPLICATION.

I'm not sure exactly where your comming from, but you seem to have missed the point of configuration and the point of my articles. First and foremost, these articles are informational, and their goal is to provide interested parties with as much information as possible about the .NET configuration framework and how to put it to good use. I try not to dictate usage scenarios, and its always possible that someone may need some alternative solution. However, for .NET programming, I think its possible to eliminate the need to spend time, money, and effort designing, developing, and maintaining a custom configuration framework when one is already available that meets the vast majority of needs.

Second, .NET configuration is only as complicated as you need it to be. Using attributed development (the declarative method) you can create a fully functional configuration section in minutes...its very, very easy, and that should be apparent to anyone who really truely reads my articles (skimming examples or introductions doesn't cut it). Configuration sections for the majority of needs are extremely easy to use, and don't need to be complicated. (In most of my examples I use the imperative method, which is more strict, secure, performs better, and offers a few additional capabilities not available when using attributes. These are a little bit more complicated than those written with the declarative method, but not much.) In the event that you NEED something more advanced, the .NET configuration framework provides you with a core system that does most of the work, allowing you to focus on the details of your specific implementation. In contrast, if you were to write your own configuration framework that met complicated needs, you not only need to maintain configuration sections themselves, but you also need to maintain the configuration framework. When your on a tight deadline, a custom configuration framework could only exacerbate your problem.

In response to the points you made:

1) Written in stone? Absolutely NOT. The whole point of these articles is to show you that the .NET configuration framework is extremely flexible, providing dozens of hooks where you can customize the framework to your specific needs. The framework is a base, providing core code that meeds 90% of needs, but still allowing custom processes (such as serialization or deserialization of alternate XML structure) when they are needed. No, the .NET configuration framework is not set in stone at all.

2) True, your configuration classes require predetermined base classes (although the attributes don't pose any problem at all...they are attributes, something external tacked onto the code itself, but not changing it in any way). Best practice regarding configuration is that it should generally be abstracted away from the objects and/or code that uses them. IMO, merging configuration and functional code can lead to confusing code. Objects end up having multiple purposes...one operational, one configurational. Sure, serializing an object to XML is super easy, and in simple applications, that might be acceptable. In larger projects, however, isolation of purpose (abstraction) brings many benefits, as can be seen in n-tier architectures.

That said, because .NET configuration XML elements are backed by .NET classes, you can use configuration classes as objects if you so wish. Quite often these days, I put custom methods and properties in my configuration classes to encapsulate my configuration settings and operations performed on those settings, much like I would encapsulate a database table with a business entity. The concept is largely the same...you have a data store and you have business logic, and you want to abstract consumers of that data (i.e. presentation code) from the data store. By using .NET configuration, you can achieve the same abstraction of your consumers from your configuration settings.

3) I'm unsure what your stating here. The whole entire point of configuration is to eliminate hard-coded values. The whole point of implementing a .NET configuration section is to allow you to provide dynamic (not hard coded) settings, or refactor hard-coded settings (which are often not though of as configurable the first time around) outside of your code into a more dynamic location.

Another point that I think should be made is in regards to your assumptions about when configuration should (or most often is) implemented. Use of configuration, regardless of whether your using .NET configuration or some other framework, does not neccesarily happen in the first iteration of coding on a project. Quite often (in my experience), configuration is an afterthought that is added during subsequent refactorings of a piece of code. When under a tight deadline that can't be met due to current workload, the first response is to cut something. I've learned that making things configurable is a first-choice feature to cut when you have a deadline to meet, as configuration can be easily added later on, replacing configurable values with appropriate hard-coded values for a short term.

Written in the stone means your configuration choices are so complicate it becomes a hard job to change it into another solution.
A data (configuration item or whatever) should not be aware the way it is stored or retrieved (or as less as possible).
It is also important for it to be designed in an application –oriented way, if you HAVE TO inherit from some base class; it is written in the stone.

Yes you have hardcoded values, when you have:
s_propString = new ConfigurationProperty("stringValue", ...
and later
[ConfigurationProperty("stringValue", IsRequired=true)]
and again
get { return (string)base["stringValue"]; }
you have 3 times the same hardcoded and uncontrollable value "stringValue", it IS an hardcoded value.

You don’t seem to believe I have any clue what I’m talking about, so here is a 10 minutes configuration for a first iteration.

After adding a XmlDocument to my settings, I have:

Just a test

My config looks like:

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml.Serialization;

No hardcoded value, no dependency on any base class, no complex concept, open to changes such as validation, relocation, use of the standard way .Net handles configuration and I have now time to write the actual application.
If my config is moved to a database or else where I don’t have to change anything to the config definition, just the way it is retrieved.
If I need to validate the xml before deserialization, I can do it with just a few changes.
Also mention the MyConfigData just contains MY DATA and a few attributes.

You have overcomplicated your configuration. You do not have to use application settings, which are a settings feature with VS2005 UI integration, mostly a convenience for those who don't even want to spend....yes, just a mere 10 minutes, writing a config section. If you wrote a config section properly (which would only take 10 minutes or less), your xml would be more like this:

I still don't think you understand .NET configuration. You think its not flexible, that it can't serialize custom XML structures as values, or read from a database, etc. It can, its very flexible, its simple, its easy to understand (its xml with an object model behind it) and writing a fairly normal config section doesn't even take 10 minutes.

It really depends on your needs. If your needs are complex, the .NET Configurations greater complexity can be a huge boon. However, that doesn't preclude its simplicty. I have provided an extremely detailed view into the heart of .NET configuration so that IF you needed more than just some basic settings, you could use the framework effectively. If all you need is something simple, you can keep it simple:

The above example shows how simple .NET configuration can be if all you need is a few typed settings, and if you can't understand the above example, then you need to hit the C# books again, because it doesn't get much simpler than that.

I agree it's easy to understand once you've understood it -- but the mechanics of it are quite complex, and without understanding what's going on under the hood, it's often difficult to know what exactly the best way is to go about doing certain things.

For instance the necessity to add sections to app.config, the necessity to create sections in app.config dynamically in case app.config isn't there, and the necessity to set allowExeDefinition before you can write to roaming/local users. The amount of work you need to do there isn't so much that I'd complain, but unless you understand how it all works, you're left to feel like you're writing boilerplate code for no apparent reason.

I wouldn't say you have to write any boilerplate code. The framework is the boilerplate. You simply have to write confituration implementations on top of the framework. In regards to being able to understand it...if your needs are more complex than something like the example I posted previously, I would hope that this article series clears up all the rest, and gives you that additional understanding you need.