App Domains and dynamic loading (the lost columns)

App Domains and dynamic loading (the lost columns)

As promised, I'm going to start republishing some of my columns that were eaten by MSDN.

I spent some time reading this one and deciding whether I would re-write it so that it was, like, correct. But it became clear that I didn't have a lot of enthusiasm towards that, so I've decided to post it as is (literally, as-is, with some ugly formatting because of how I used to do them in MSDN).

I also am not posting the source, though I might be tempted to put it someplace if there is a big desire for it.

So, on to the caveats...

The big caveat is that my understanding of how the probing rules work was incorrect. To get things to work the way I have them architected, you need to put them somewhere in the directory tree underneath where the exe lives, and if they aren't in the same directory, you need to add the directory where they live to the private bin path. I may also have the shadow directory stuff messed up.

So, without further ado, here's something that I wrote five years ago and has not been supplanted by more timely and more correct docs. AFAIK. If you know better references, *please* add them to the comments, and also comment on anything else that's wrong.

App Domains and dynamic loading

Eric GunnersonMicrosoft Corporation

May 17, 2002

Download the ???.exe sample file. ???MSDNSamples\C#

Download or browse the ???.exe in the MSDNOnlineCodeCenter!href(/code/default.asp?URL=/code/sample.asp?url=/msdn-files/026/002/???/msdncompositedoc.xml).

This month, I’m sitting in a departure lounge at Palm Springs airport, waiting to fly back to Seattle after an ASP.NET conference.

My original plan for this month – to the extent that I have a plan – was to do some work on the expression parsing part of the SuperGraph application. In the past few weeks, however, I’ve received several emails asking when I was going to get the loading and unloading of assemblies in app domains part done, so I’ve decided to focus on that instead.

Application Architecture

Before I get into code, I’d like to talk a bit about what I’m trying to do. As you probably remember, SuperGraph lets you choose from a list of functions. I’d like to be able to put “add-in” assemblies in a specific directory, have SuperGraph detect them, load them, and find any functions contained in them.

Doing that by itself doesn’t require a separate AppDomain; Assembly.Load() usually works fine. The problem is when you want to provide a way for the user to update those assemblies when the program is running, which is really desirable if you’re writing something that runs on a server, and you don’t want to stop and start the server.

To do this, we’ll load all add-in assemblies in a separate AppDomain. When a file is added or modified, we’ll unload that AppDomain, create a new one, and load the current files into it. Then things will be great.

To make this a little clearer, I’ve created a diagram of a typical scenario:

In this diagram, the Loader class creates a new AppDomain named Functions. Once the AppDomain is created, Loader creates an instance of RemoteLoader inside that new AppDomain.

To load an assembly, a load function is called on the RemoteLoader. It opens up the new assembly, finds all the functions in it, packages them up into a FunctionList object, and then returns that object to the Loader. The Function objects in this FunctionList can then be used from the Graph function.

Creating an AppDomain

The first task is to create an AppDomain. To create it in the proper manner, we’ll need to pass it an AppDomainSetup object. The docs on this are useful enough once you understand how everything works, but aren’t much help if you’re trying to understand how things work. When a Google search on the subject returned up last month’s column as one of the higher matches, I suspected I might be in for a bit of trouble.

The basic problem has to do with how assemblies are loaded in the runtime. By default, the runtime will look either in the global assembly cache or in the currently application directory tree. We’d like to load our add-in applications from a totally different directory.

When you look at the docs for AppDomainSetup, you’ll find that you can set the ApplicationBase property to the directory to search for assemblies. Unfortunately, we also need to reference the original program directory, because that’s where the RemoteLoader class lives.

The AppDomain writers understood this, so they’ve provided an additional location in which they’ll search for assemblies. We’ll use ApplicationBase to refer to our add-in directory, and then set PrivateBinPath to point to the main application directory.

Here’s the code from the Loader class that does this:

AppDomainSetup setup = new AppDomainSetup();

setup.ApplicationBase = functionDirectory;

setup.PrivateBinPath = AppDomain.CurrentDomain.BaseDirectory;

setup.ApplicationName = "Graph";

appDomain = AppDomain.CreateDomain("Functions", null, setup);

remoteLoader = (RemoteLoader)

appDomain.CreateInstanceFromAndUnwrap("SuperGraph.exe",

"SuperGraphInterface.RemoteLoader");

After the AppDomain is created, the CreateInstanceFromAndUnwrap() function is used to create an instance of the RemoteLoader class in the new app domain. Note that the filename of the assembly the class is in is required, and the full name of the class.

When this call is executed, we get back an instance that looks just like a RemoteLoader. In fact, it’s actually a small proxy class that will forward any calls to the RemoteLoader instance in the other AppDomain. This is the same infrastructure that .NET remoting uses.

Assembly Binding Log Viewer

When you write code to do this, you’re going to make mistakes. The documentation provides little advice on how to debug your app, but if you know who to ask, they’ll tell you about the Assembly Binding Log Viewer (named fuslogvw.exe, because the loading subsystem is known as “fusion”). When you run the viewer, you can tell it to log failures, and then when you run your app and it has problems loading an assembly, you can refresh the viewer and get details on what’s going on.

This is hugely useful to find out, for example, that Assembly.Load() doesn’t require “.dll” on the end of the filename. You can tell this in the log because it will tell you it tried to load “f.dll.dll”.

Dynamically Loading Assemblies

So, now that we’ve gotten the application domain created, it’s time to figure out how to load an assembly, and extract the functions from it. This requires code in two separate areas. The first finds the files in a directory, and loads each of them:

void LoadUserAssemblies()

{

availableFunctions = new FunctionList();

LoadBuiltInFunctions();

DirectoryInfo d = new DirectoryInfo(functionAssemblyDirectory);

foreach (FileInfo file in d.GetFiles("*.dll"))

{

string filename = file.Name.Replace(file.Extension, "");

FunctionList functionList = loader.LoadAssembly(filename);

availableFunctions.Merge(functionList);

}

}

This function in the Graph class finds all dll files in the add-in directory, removes the extension from them, and then tells the loader to load them. The returned list of functions is merged into the current list of functions.

The second bit of code is in the RemoteLoader class, to actually load the assembly and find the functions:

public FunctionList LoadAssembly(string filename)

{

FunctionList functionList = new FunctionList();

Assembly assembly = AppDomain.CurrentDomain.Load(filename);

foreach (Type t in assembly.GetTypes())

{

functionList.AddAllFromType(t);

}

return functionList;

}

This code simple calls Assembly.Load() on the filename (assembly name, really) passed in, and then loads all the useful functions into a FunctionList instance to return to the caller.

At this point, the application can start up, load in the add-in assemblies, and the user can refer to them.

Reloading Assemblies

The next task is to be able to reload these assemblies on demand. Eventually, we’ll want to be able to do this automatically, but for testing purposes, I added a Reload button to the form that will cause the assemblies to be reloaded. The handler for this button simply calls Graph.Reload(), which needs to perform the following actions:

1.Unload the app domain

2.Create a new app domain

3.Reload the assemblies in the new app domain

4.Hook up the graph lines to the newly created app domain

Step 4 is needed because the GraphLine objects contain Function objects that came from the old app domain. After that app domain is unloaded, the function objects can’t be used any longer.

To fix this, HookupFunctions() modifies the GraphLine objects so that they point to the correct functions from the current app domain.

Here’s the code:

loader.Unload();

loader = new Loader(functionAssemblyDirectory);

LoadUserAssemblies();

HookupFunctions();

reloadCount++;

if (this.ReloadCountChanged != null)

ReloadCountChanged(this, new ReloadEventArgs(reloadCount));

The last two lines fire an event whenever a reload operation is performed. This is used to update a reload counter on the form.

Detecting new assemblies

The next step is to be able detect new or modified assemblies that show up in the add-in directory. The frameworks provide the FileSystemWatcher class to do this. Here’s the code I added to the Graph class constructor:

watcher = new FileSystemWatcher(functionAssemblyDirectory, "*.dll");

watcher.EnableRaisingEvents = true;

watcher.Changed += new FileSystemEventHandler(FunctionFileChanged);

watcher.Created += new FileSystemEventHandler(FunctionFileChanged);

watcher.Deleted += new FileSystemEventHandler(FunctionFileChanged);

When the FileSystemWatcher class is created, we tell it what directory to look in and what files to track. The EnableRaisingEvents property says whether we want it to send events when it detects changes, and the last 3 lines hook up the events to a function in our class. The function merely calls Reload() to reload the assemblies.

There is some inefficiency in this approach. When an assembly is updated, we have to unload the assembly to be able to load a new version, but that isn’t required when a file is added or deleted. In this case, the overhead of doing this for all changes isn’t very high, and it makes the code simpler.

After this code is built, we run the application, and then try copying a new assembly to the add-in directory. Just as we had hoped, we get a file changed event, and when the reload is done, the new functions are now available.

Unfortunately, when we try to update an existing assembly, we run into a problem. The runtime has locked the file, which means we can’t copy the new assembly into the add-in directory, and we get an error.

The designers of the AppDomain class knew this was a problem, so they provided a nice way to deal with it. When the ShadowCopyFiles property is set to “true” (the string “true”, not the boolean constant true. Don’t ask me why…), the runtime will copy the assembly to a cache directory, and then open that one. That leaves the original file unlocked, and gives us the ability to update an assembly that’s in use. ASP.NET uses this facility.

To enable this feature, I added the following line to the constructor for the Loader class:

setup.ShadowCopyFiles = "true";

I then rebuilt the application, and got the same error. I looked at the docs for the ShadowCopyDirectories property, which clearly state that all directories specified by PrivateBinPath, including the directory specified by ApplicationBase, are shadow copied if this property isn’t set. Remember how I said the docs weren’t very good in this area…

The docs for this property are just plain wrong. I haven’t verified what the exact behavior is, but I can tell you that the files in the ApplicationBase directory are not shadow copied by default. Explicitly specifying the directory fixes the problem:

setup.ShadowCopyDirectories = functionDirectory;

Figuring that out took me at least half an hour.

We can now update an existing file and have it correctly loaded in. Once I got this working, I ran into one more tiny problem. When we ran the reload function from the button on the form, the reload always happened on the same thread as the drawing, which means we were never trying to draw a line during the reload process.

Now that we’ve switched to file change events, it’s now possible for the draw to happen after the app domain has been unloaded and before we’ve loaded the new one. If this happens, we’ll get an exception.

This is a traditional multi-threaded programming issue, and is easily handled using the C# lock statement. I added a in the drawing function and in the reload function, and this ensures that they can’t both happen at the same time. This fixed the problem, and adding an updated version of an assembly will cause the program to automatically switch to a new version of the function. That’s pretty cool.

There’s one other weird bit of behavior. It turns out that the Win32 functions that detect file changes are quite generous in the number of changes they send, so doing a single update of a file leads to five change events being sent, and the assemblies being reloaded five times. The fix is to make a smarter FileSystemWatcher that can group these together, but it’s not in this version.

Drag and Drop

Having to copy files to a directly wasn’t terribly convenient, so I decided to add drag and drop functionality to the app. The first step in doing this is setting the AllowDrop property of the form to true, which turns on the drag and drop support. Next, I hooked a routine to the DragEnter event. This is called when the cursor moves in an object on a drag and drop operation, and determines whether the current object is acceptable for drag and drop.

privatevoid Form1_DragEnter(

object sender, System.Windows.Forms.DragEventArgs e)

{

object o = e.Data.GetData(DataFormats.FileDrop);

if (o != null)

{

e.Effect = DragDropEffects.Copy;

}

string[] formats = e.Data.GetFormats();

}

In this handler, I check to see if there is FileDrop data available (ie a file is being dragged into the window). If this is true, I set the effect to Copy, which sets the cursor appropriately and causes the DragDrop event to be sent if the user releases the mouse button. The last line in the function is there purely for debugging, to see what information is available in the operation.

The next task is to write the handler for the DragDrop event:

privatevoid Form1_DragDrop(

object sender, System.Windows.Forms.DragEventArgs e)

{

string[] filenames = (string[]) e.Data.GetData(DataFormats.FileDrop);

graph.CopyFiles(filenames);

}

This routine gets the data associated with this operation – an array of filenames – and passes it off to a graph function, which copies the files to the add-in directory, which will then cause the file change events to reload them.

Status

At this point, you can run the app, drag new assemblies onto it, and it will load them on the fly, and keep running. It’s pretty cool.

Other Stuff

C# Community Site

I’ve set up a Visual C# Community Newsletter, so that the C# product team has a better way to communicate with our users. I’m going to use it to announce when there’s new content on our community site at http://www.gotdotnet.com/team/csharp!href(http://www.gotdotnet.com/team/csharp), and also to let you know if we’re going to be at a conference or user group meeting.

You can sign up for it at the site listed above.

C# Summer Camp

This coming August, we're teaming up with developmentor to host C# Summer Camp. This is a chance to receive excellent C# training from developmentor instructors and to spend some time with the C# Product Team. There's more information at the developmentor site!href(http://www.developmentor.com/conferences/csharpsummer/csharpsummer.aspx).

Next Month

If I do more work on SuperGraph, I’ll probably work on a version of FileSystemWatcher that doesn’t send lots of extraneous events, and possibly on the expression evaluation. I also have another small sample that I may talk about instead.

<HR NOSHADE SIZE=1>

Eric Gunnerson is a Program Manager on the Visual C# team, a member of the C# design team, and the author of A Programmer's Introduction to C#!href(http://www1.fatbrain.com/asp/bookinfo/bookinfo.asp?theisbn=1893115860&vm=c). He's been programming for long enough that he knows what 8-inch diskettes are and could once mount tapes with one hand.