2004 (28)

Some ASP.NET compiler black magic

In the work we’ve been doing with Rob on the Kona commerce app, our quest for extreme pluggability has led us to look at quite a few interesting features of ASP.NET compilation. Features I didn’t know about before Dmitry and David pointed them out for me. I thought I’d share…

They both enable you to reference code that is in a different file in the site. With both of them, you get full IntelliSense on the referenced code, but they don’t reference the same kinds of files. @Reference is meant to reference a specific class whereas @Assembly brings in an arbitrary code file. As a consequence, @Reference needs a file where there is a well-defined default class, such as a Page or a User Control. @Assembly on the other hand will enable you to reference an arbitrary code file with as many classes as you wish and get the compiler to dynamically build a neat assembly out of it. The difference really amounts to whether or not the build provider implements GetGeneratedType, which the aspx and ascx build provider implements, and which generic code file providers typically don’t. In other words, if you built your own Build Provider and implemented that method, the files it compiles could be referenced using Reference. Without it, ASP.NET wouldn’t know which type to pick as the referenced one. But @Assembly will still work.

Building an assembly from a virtual path was really important for us because it enables a file anywhere in the web site to be dynamically compiled and used, which is exactly how we wanted plug-ins to work: we could have put the plug-ins into App_code (which does dynamic compilation automatically) but the name is not exactly intuitive, and it limits your ability to mix languages within the same folder. Please note that there *is* a way to mix languages in App_code:

This config setting instructs the compiler to compile the cs and js folders in App_code into separate assemblies, which is all we need to allow for multiple languages. But we can do better than that.

It so happens that the compiler feature that enables the Assembly directive is also available as a public API (that works in Medium Trust):

BuildManager.GetCompiledAssembly

Seriously, this is now officially my favorite API in the whole .NET framework. This quite remarkable API takes a code file within the site and compiles it into an assembly (if that hasn’t already been done by a previous call to that API or by an @Assembly directive). What you get back from it is an Assembly object, which you can reflect on (using public reflection, which works in Medium Trust) and use any way you want.

This will enable us to dynamically compile the files in a Plugins top-level directory of the app. Doing so, we are getting multi-language support without config settings or special folders, nicer folder name and bonus points for not shutting down the app domain every effing time any file is touched. Bye bye App_code!

Well, of course that’s now one assembly per code file, which could be a problem if you have 800 plug-ins in your app, but then again if that’s the case maybe it’s time you moved all those into a nice pre-compiled assembly. Or do some clean-up. Anyway, this will work just fine for the type of scale we have in mind.

A question you may ask at this point is what exactly happens when one of those files is modified. Well, the BuildManager will generate a new assembly and “forget” about the old one. And when I say forget, I don’t mean it’s getting unloaded, just that it won’t get used anymore (unless you have code that held on to a reference). That means that potentially, there could be some assembly rot after a while if your files change often. To mitigate that, BuildManager has a set of rules that it uses to determine that it needs to restart the app domain after a while. Just like App_code, just a lot less often. Basically, the rules are pretty much the same as for aspx or ascx files.

All right, so this is all quite useful (I know I’m going to use that stuff a lot, at least). How about a useless hack now? (if you don’t like a fun hack, feel free to skip the rest of this post)

So think about all the neat stuff we could do by combining this with Virtual Path Providers... Except that in ASP.NET up to and including 3.5 SP1, Virtual Path Providers don’t work in Medium Trust. Neither do Build Providers, which would also be quite neat to play with. Bummer.

But wait, people in the team thought about that and wondered what harm exactly you could do with VPPs and BPs that you couldn’t already do by simply writing code and well, the answer is pretty much nothing. So the good news is that VPPs and BPs will work in Medium Trust in ASP.NET 4.0. Hurray!

So just for the sake of it, here’s the real black magic part of this post. Please note that there is more than one better way to do what I’m about to do and pretty much all of them would be simpler. I’m just hacking here and it’s going to be relatively convoluted.

And as Adam Savage says, “don’t try any of what you’re about to see at home. Ever!”. Seriously, this is dangerous code that has too many holes to count and that I wouldn’t run on anything but Visual Studio’s built-in web server (which is limited to requests from the same machine). Again, just hacking for fun here.

The thing I’ve done is build a VPP that takes an operation embedded in the file name and builds a function that executes that operation. So for example, if you ask it for “A+B.cs”, it will get you a cs file that contains a class that has a method that takes two arguments and returns their sum:

@Mike: Applications such as WordPress enable you to create plug-ins with just notepad. We don't want our users to have to install Visual Studio in order to be able to start working and tweaking the application.
This model enables plug-ins to be deployed as simple code files, and it also enables the administrator to create and modify plug-ins directly online, from the site.

Ok, I understand. But do you think that is actualy what users want? I have written some simple plugins (for Graffiti CMS for instance) using Visual Studio. I can't imagine doing it without visual studio. I have to reference assemblies, I write C#, it needs to compile before it can run, etc. It would be painful to write that in a textarea and catching compile errors in the browser...

Also, I watched the Kona screencast, notice how the web code editor has syntax highlighting? Where are we going with this really? We don't want to use Visual Studio, but we do feel the need for code highlighting in our browser editor? How about intellisense?

@Mike: yes, that is actually what users want, but in fact you have a choice. You can write a Kona plug-in as a simple code file (in Notepad or in VS), but you can also build it into a dll and deploy that into bin. Kona will be able to discover it just as well, it just won't be able to modify it in the online editor.
The code highlighting that is currently in Kona is a fast, existing component we included to demonstrate the direction we are going into but this is not what we are going to use in the end. We are exploring including a more elaborate editor that includes IntelliSense.

I'm building a plugin based on this code for the Umbraco CMS - basically as an XML-powered CMS, people want to write XSLT extension methods, but don't necessarily want to have to write a full-blown compiled library. *But* doesn't work in Medium Trust. Solution? Abstract that into a .cs file and use this code to compile it for accessing through XSLT code. Works in Medium Trust a treat.
This code is brilliant, thanks for sharing it!