So, I finally have CacheGen to where I can probably integrate it into this website. I did some rough concurrency testing (spawning 60 threads accessing the cache with random clearing). It's a rough test, but it does show that there isn't anything obviously wrong with it at least.

So, the code it generates is brilliantly simple as well. Some good use cases for this:

Keep all your cache settings in one place

Statically typed and named! No more remembering manual casts or magic strings

Make your caching logic testable! It generates code against an easily mockable interface

Switch out your caching layer with ease.

Now, I'm only going to elaborate on the last point. "Why would I ever want to change out my caching layer!?"

Here's why. You built Bookface 1.0 and a few dozen users are on it. People start talking though and suddenly you have a few thousand(or more). You page response times have crept up into the seconds range. Something must be done. After upgrading servers, and expanding some of the hardware, you find the bottleneck. Your web server's caches are being cleared too often. There isn't anything you can do though, the memory is maxed out as it is. So, obvious choice: Use something like memcached for distributed caching on a dedicated server or two.

What's makes using memcached or something so hard? It requires code changes! Luckily for you, you used CacheGen though. Why? All of your caching is in one place, and your interface to the caching method(CacheMechanism) is in one single simple class. It's trivial to implement a two-level cache between ASP.Net and memcached at this point and all of your code relying on your cache will just magically work without being changed.

This is what I think makes CacheGen especially awesome. It manages your caching settings, makes everything statically typed, AND lets you have an almost unreal amount of flexibility.

It's not quite ready for primetime yet. I've proved that it should work, the thing now to do is clean up the API some and add some more unit testing to see if I can catch more bugs.

Anyway, I don't expect this process to take too long. I plan to tag an alpha release for this relatively soon (within the month)

Basically, it's a T4 template that generates really awesome helpers for caching. It's geared at ASP.Net, but my goal is to make it so it can be used in other ways as well. So, how does it work? You give it a list of specifications for something you want to cache. Let's say you have a string that's really expensive to generate, but doesn't belong in a database. To go on my theme of markdown being slow, let's call it MarkdownTranslation

Much much easier. But, there is a flaw in this. It's possible that we could consume double the server resources required for this. This wouldn't be a problem in this case, but imagine a very heavy SQL query or something. This isn't quite what we want when things are really expensive. So, let's use some other fancy syntax

What this will do instead is raise a flag so that requests for MarkdownTransform will block and during this time, it will execute the expensive code and other threads will sleep. This way, it only gets executed once. And, when it finally gets the results back, the cache will be loaded with it and the other threads will be able to access it. So, instead of computing the expensive thing multiple times, instead we just hold out other requests for a little while so the expensive thing is done only once.

Sure, you can do this same thing with ASP.Net's cache, but how much code would it require for each time you did this? Hence why it's T4. Also, it's completely statically typed. No casting!

Now, what if you wanted to cache the markdown for multiple posts? Or just want to cache entire post objects? Well, I thought of this too:

These are the ideas anyway. None of it is actually implemented yet, and a lot more research must be done to ensure that this is a sane way to go about it. But, you can expect to get something similar to this.

So, you're like me and have a greater than 500 line T4 template that is a steaming pile of... code. And of course, no syntax highlighting without addons, no intellisense, generally horrible Visual Studio support, and near impossible to unit test.

Well, my friends, I have just the thing for you! After beating my head against a wall for several days, I've found salvation!

To use my method requires some "neat" code, and it requires a few assumptions:

You're using T4 to generate C# code

You're then taking this C# code and including it in your project

You're ready to do some major refactoring for an awesome experience in the end (and your T4 template will be so much cleaner afterwards!)

You can have at least 2 tiles tied to T4 (one T4 template and one include file)

The corner stone of making T4 testable is to separate out the "logic" and the "content"; where the content is the generated C# code, and the logic is what you do to generate it.

To do this and enforce this separation cleanly, you must have two files. One of these files is the T4 "view", and another file is the logic, which is capable of being normally compiled outside of T4.

Example Untestable T4 Code

Lets start with a simple example. You have a simple T4 template which takes a file like so:

Run an integration test manually comparing the code (very very brittle)

I can't think of anything else worth mentioning

The Great Refactor

Now, What I suggest:

Let's refactor this and make it so we can eliminate some logic out of our "view". In this case, the view should worry about getting the code into the generated file and that's it. The logic on the other hand should work at a level of abstraction.

Here's what we're going to do:

Eliminate most of this code from the view

Create a new file for the logic

Use a very clever trick so that our logic file will compile outside of T4 and work when included in T4

Make it so that instead of just outputting text, we're building an object model that happens to be easily translatable to text

Because I came prepared, I already have the object model abstraction built. you can catch it at the bottom of the last code snippet. (It's easy to rip out and put elsewhere)

Compiling Code In and Out of T4

So, we create a new T4 logic file and name it something like "GenerateClassLogic.tt.cs". Now, you may be thinking "but you can't use the same code in T4 and a regular project!" *WRONG!

I came across a very nifty trick. Behold:

//<#+
/*That line above is very carefully constructed to be awesome and make it so this works!*/
#if NOT_IN_T4
//Apparently T4 places classes into another class, making namespaces impossible
namespace MyNamespace.Foo.Bar
{
using System;
using System.Linq;
using System.Text;
using System.Collections.Generic;
using System.IO;
#endif
//regular ol' C# classes and code...
#if NOT_IN_T4
} //end the namespace
#endif
//#>

The key parts are the first and last lines. These begin with a comment so that the C# compiler will ignore them outside of T4, but inside of T4 these instruct the transformer to include this as a "class feature" (which ends up being a nested class)

Note here also that you must define a compile symbol. This is super easy to do. If you add it to your project, then it won't carry over to the T4 template though, making this an easy way to add in a few key things that can't be done without knowing if we're executing within T4 or not.

So, now we have a T4 logic file that will compile inside and outside of T4. Perfect for unit testing! All we need now is some logic!

Conclusion and Remarks

So, in conclusion, we've learned that T4 is actually capable of taming. (when I first learned T4, I wouldn't have thought it possible either) The main thing to do is maintain a separation of "content" and logic. Now of course, there are some gotchas to watch for:

This only works when your T4 generator's target is to generate C# code

When you add a reference to something, you must add it to both the logic file and the view

It's still very difficult to reference external assemblies(particularly project assemblies). This doesn't solve that problem at all

You must define a compiler symbol for your project if you wish to run unit tests against it.

When other people use your T4 template outside of your project(and thus don't need to test it), they must ensure that the logic file is not compiled outside of the T4

Unit testing that a piece of code is generated is very difficult and brittle(hence the need for abstractions where possible to make this easier)

My abstraction mechanisms for building objects aren't really that good. They're good enough for me, but please someone improve them! (if you do it I'll link to you from here)