Introduction

This article provides code that will take a text-based template and perform substitutions of variables identified in the template with data values extracted from a data source. Currently supported data sources are .NET class object instances, and IDataRecord and IDataReader objects (e.g., SqlDataReader).

Background

A lot of the programs that I write for my company involve sending emails to various people for one reason or another. Quite often, these emails are data-based - that is, I pull some data from a database and then have to format that data into the email before I send it. In the past, I have achieved this by reading a text template from the DLL resources, then using string.Replace to perform substitutions for predeterminedvariable names. Here's what I used to do:

{AccountManagerName}:<br/>
Sales Order <b>#{SalesOrderNumber}</b> has been fulfilled. It contains the following items:
<table><tr><th>Item #</th><th>Part #</th><th>Description</th><th>Price</th></tr>
{SalesOrderItems}
</table>

and the SalesOrderItems would have been built up in essentially the same way from a different template,
and I would have generated many lines of items and subsequently joined them with string.Join(string.Empty, SalesOrderLinesList). The template for the SalesOrderItems might have looked like this:

After building up this email from multiple different templates, I would use the .NET libraries to send the mail off to the right people. Despite how well this has served me over the years, there are several problems with this method:

If I want to add new fields to the template, I have to add code to the .Replace series of method calls

I have to have separate code to build the SalesOrderItems that gets replaced in the main email

The template must be split into multiple pieces (one for the main email, one for the repeated items)

Therefore, I recently decided to re-code how I was handling this need, and the code provided below is the result of that effort. There are still several improvements that can be made to this code, but it handles the first few scenarios that I required.

Advantages of the new code

The new version of my template-filling code has the following benefits and advantages over the old method:

Easier variable names - Instead of using the {Property} syntax described above, which I found hard to type and hard to read, I now use @Property syntax, similar to SQL variables.

Not data-structure specific - It is fully self-contained, and does not require any coding specific to the data structure. That is, you can pass it an object of type A or an object of type B, where A and B have totally different fields / properties. The code automatically locates variable names in the template and makes the appropriate replacements (it uses .NET reflection to obtain the data value).

Multi-level data - The code is capable of traversing into data structures using the @Property.SubProperty syntax. Thus, if A has a property called Data that is a class of type B, and class B contains a property called Name, you can use @Data.Name. It can go as deep as you need.

Data formatting - It supports formatting data by using @Variable{Width}{Format}. The width and format strings are the same as that used in .NET string.Format, and fully supports all of the built-in type formats. For example, if object A contains a property called DateProcessed that is a DateTime, you can use @DateProcessed{12}{yyyy-MMM-dd} to format the output to your liking. Both the width and format specifiers are optional - you can use both, either, or neither, but the width must come first.

In-line list handling - It supports in-line repeated replacements to any realistic level. Therefore, there is no longer a need to have separate template files for each portion of the template that is built up from lists. The only requirement is that the referenced object property (or object, if using @$) that needs to be repeated implements IEnumerable or IDataReader.

In-line conditionals - Simple numeric and string conditionals are supported to output either "this" or "that".

Using the code

For each of the code samples below, I shall assume the following object structure:

Output: Hello, your order #173123 has been fulfilled. The total price is 360.

Example 2: Multi-level data and data formatting

This example demonstrates the ability to dive into the object structure. Note how the code references the Customer.Name variable. It also demonstrates formatting a number - in this case the TotalPrice - with a format specifier "C" to cause the decimal property to display as a currency.

As might be obvious from the code above, in order to process lists of data with a "repeated template", you should use the syntax:

@PropertyName{Width}[[C ....repeated template... C]]

where C is any character (in the example above, I use the # character). The character chosen for C serves as part of the closing tag that identifies the end of the repeated portion of the template. By choosing different characters for C, you can even have repeated templates inside other repeated templates! For example, you might use something like:

Notice how the SimilarParts is generated by using a repeated template with [[% ... %]]
inside the Items repeated template which itself uses [[# ... #]]. As long as you don't use
a template delimiter character sequence inside another repeated template that uses the same delimiter character, you can essentially nest these repeated templates as far as you need to. (To be clear, the character itself can be used inside the template - it only has significance when adjacent to the ]] template termination characters.)

As with standard variables, the {Width} specifier is optional in this case, and if used will be used to format the string that is returned by the repeated template subexpression. It should only be used if you are formatting data on a single line - trying to use a {Width} for multi-line constructs such as the item list probably doesn't have much value.

Example 4: Conditionals

Conditionals are implemented with a syntax similar to that of repeated data:

Note that the conditional must use @? as the prefix, and have at least two subsequent clauses, one for the actual conditional, and one for the expression to output if the conditional is true. It can also be followed by an optional clause to be output if the conditional is false.

Currently, only very simple conditionals are implemented. It must be either a numeric (decimal) or a string comparison using the operators >, >=, <, <=, ==, or !=. The conditional is first evaluated as a numeric comparison, and if that fails (that is, if the expressions on both sides of the operator cannot be converted into decimal numbers) then the comparison continues as a case-sensitive string comparison. No compound expressions (using && (AND) or || (OR), for example) are currently allowed. White-space around the expressions in the conditional is ignored.

Minor Tidbits

There are a few other things of note:

The code will attempt to locate both properties and fields. The variable name is probably case-sensitive.

There is a special variable called @$ which will return the object itself instead of trying to find a field or property. This might be handy in a few circumstances, for example if you have an override on .ToString() that you want to use as the output, or if you want to use a List as the data source (since the list itself doesn't have a property or field to access the items.) Then, to access the list use a construct similar to: Users: \r\n@$[[#@FirstName @LastName -- @Email\r\n#]]

The code is designed so that it can look at both instance and static properties and fields, but I haven't tested it with static properties or fields. Similarly, it is coded so that it can access both public and non-public properties and fields, but I haven't tested non-public properties or fields. I recommend sticking to public instance fields or public instance properties.

The code is able to handle both object-based data sources (as primarily discussed in this article)
but it can also handle IDataRecord objects and IDataReader objects such as SqlDataReader. (Note that SqlDataReader is in fact both an IDataRecord and an IDataReader!)

If the object passed is only an IDataRecord, it will be treated as a single instance of a class would. However, when using an IDataRecord, multi-level data, repeated templates (single and nested) are not available, since the data structure provided by IDataRecord simply doesn't implement those concepts.

If an IDataReader is passed, then it will iterate over all of the records passed, so repeated templates are available. Nested repeated templates are not available, since each iteration is treated as an IDataRecord. If you use a repeated template, you must use @$ as the main variable name, similar to the syntax described in point #2.

The implementation recursively evaluates all detected repeater elements first, then conditionals. By doing this recursively, there shouldn't be any conflicts with "conditionals inside repeaters" nor "repeaters inside conditionals" as long as you honor the nesting character issue (that is, don't use [[C C]] inside another another one that uses the same delimiting C character, even if one is a repeater and one is a conditional.) The demo project has a more complicated example where several levels of nesting are used. (Specifically, it demonstrates a conditional that contains a repeater that contains a conditional.)

Points of Interest

The code uses a single .NET Regular Expression to locate variables in the template that it needs to evaluate and replace, and a second one to locate conditionals. The evaluator regular expression is a bit wild, but not too difficult to understand:

This part locates a variable name consisting of either the $ special variable, or a repeated set of Property.SubProperty.SubProperty... Note in particular that the '.' characters must be contained within (surrounded by) alphanumeric characters - a period at the end of a variable name won't be matched (and it shouldn't). This allows you to put a variable name at the end of a sentence, as in Example #1 above.

({(?<Width>-?\d+)})?

This part finds an optional .NET style width specifier. It must be an integer (although 0 has no valid meaning,
and the .NET formatting mechanism might even reject it.)

For the last part, I will split this portion apart slightly so that it's easier to see what's happening... (In other words, ignore the line-breaks and other white-space!)

This part finds either a format string contained inside curly braces, OR ELSE it finds a matching-delimited subexpression indicated by the [[C ... C]] syntax discussed in the "Using the Code" example #3. Note that this implies that it is invalid to have both a format specifier and a subexpression. This restriction is by design because there aren't any valid format specifiers for strings in .NET (although you can specify width).

The conditional regular expression is similar and I won't discuss it in detail here.

Known Limitations and Possible Future Enhancements

If the member found is a Property (as opposed to a Field) and the Property is an indexed property, only the default property will be used

As demonstrated in Example #3 part 2 (nested repeated templates) there is no way currently to trim what we would desire to be strictly intervening characters. This is why the list of "similar parts" in the output not only has commas between the part numbers, but also at the end of the list.

There is currently no code to account for the case where you would want an @ symbol next to characters/numbers in the template file without performing a replacement. If you need an @ symbol,
just put a space after it and deal with it.

Code Notes

I have provided sample code that provides the source code for the FillTemplate method and demonstrates most of the features described above. The only thing that isn't fully covered is passing a SqlDataReader, although I do provide a template for use. (This is because I don't know what SQL Data sources you might have access to, and you'll need to fill in a bit of SQL code to see this feature in action. See the code for details.)

Because I could. Because it was fun. Because I like to build things for myself. Because I like to figure things out. Because I'm picky about having intimate understanding of the code I use. Because I'd never heard of DotLiquid or Razor. Because I wanted something I could write a CodeProject article about.

Because I wasn't aware of any libraries that did this already, and I didn't bother to search. I use this feature so often that had I taken a moment I could have guessed that others would have already done it. But I didn't. The end.

Hmm possibly, yes - it looks similar. From what I could see on the site, though, there's very little in the way of documentation. It wasn't clear to me that they support conditionals or repeated templates, but I didn't look into the details of the code at all.

They seem to have a more capable interpreter - that is they have ability for language elements such as @foreach and lambda expressions and such in the template. That's a pretty nice capability.

So at this point, I think I have to admit that my version probably has no technical advantages over Razor, and it probably never will. As I mentioned in the article, I wrote it to satisfy a need, which it did.

It would be nice to be able to pass in a culture string (en, en-US), an LCID (9, 1033) or even an actual instance of a cultureinfo. That way the formatting can be localized.

Another nice feature to have would be to allow for substitutions to come from resource files (RESX or WPF Resources). This would be for dynamic substitutions of strings, not the entire text. The template should be in the appropriate language before formatting

Otherwise it looks interesting... When I have a chance to play with it a little more, I may have some additional suggestions... I had need of something like this in an older project, but I used something like your .Replace method.

It sounds like you are actually proposing a question and solution... That is, why not just set your template text based on the desired culture? That way, the formatting could be the same or completely different. The data would need to be in the appropriate language too. Maybe I'm just not understanding your point / question. Maybe a more specific example would help me.

I've cleaned up the code a little, so you might want to take a look. I'd actually got a bit further, like:
-Breaking out the two major different sections into separate private static methods (IDataReader vs other), as you can optimize the IDataReader a bit more. You could try saving off the indices...
-Avoiding creating additional StringBuilders and sub-strings when you could pass the StringBuilder around internally to private static methods.

Hey Andrew, thanks for the suggestions. I did consider splitting into two functions for IDataReader vs object, and I may still do that, but the code didn't seem complicated enough to really need it, and there was enough common code that it was easier to just execute the single if.

Also, since you posted this comment, I had been making some more changes, and I've posted a new version of the code. You won't find much has changed in regards to your comments, but I've added the capability to handle conditionals. It did make the code a bit longer, so I took the code listing out of the article. Note that I did rename the function to FillTemplate.

Hmmm you know I just updated this article - perhaps you caught it somewhere in the middle of updating... I can see the stuff in the ZIP file when I go to "Browse Code" so I know it's there somewhere, but I can't actually see where one might download the whole file. I'll try to update it again.

Oh I see... I just forgot to click the link to "Add selected files to article"... My bad - I haven't published very often so I'm not totally familiar with the process. It should be up there shortly, as soon as new version gets approved.