C# 4.0 New Features Part 1.1 – dynamic keyword second look

by Justin Etheredge on October 28, 2008

This post was migrated from Justin’s personal blog, 'Codethinked.com.' Views, opinions, and colorful expressions should be taken in context, and do not necessarily represent those of Simple Thread (and were written under the influence of dangerous levels of caffeination).

I was originally going to just update my original post with some performance data regarding the dynamic keyword versus MethodInfo.Invoke, but I started looking a bit deeper at the performance implications of the dynamic keyword. In my previous post I posted these performance numbers:

But these numbers don’t really tell the whole picture. The reality is that in the above test I was factoring in the overhead of initializing the callsites used by the dynamic invoke. Since I ran the test over and over from the start, this was included every time. So, I decided to update my testing script to look like this:

What we are doing here is calling the two methods once before we start testing them in order to get the overhead out of the way. Then we run a loop 5 times and then run our 1000000 loops through calling both methods. So, we are still doing 2,000,000 calls, but we are now doing them five times in a row. So, 10,000,000 total calls per method. I also tested the above code using the “dynamic” keyword and MethodInfo.Invoke.

The numbers that I got were pretty startling. The first method didn’t change, there is no startup time for compile time binding, so we don’t really see any difference testing it this way.

The second method (the dynamic keyword) is where the real changes started to show up. The start up time for the dynamic keyword is pretty high right now (I was recording about 3.5 seconds to make the first two calls). But once the first calls were out of the way, and our CallSites were created, they are assigned as static methods on a compiler generated class and so they are only created once when first called. The performance here was dramatically better. The calls to warm it up just looked like this:

Third I tested MethodInfo.Invoke where we also started them up and invoked the calls once. The difference here is that each time the method is called, we have significant overhead to make the call, even though we are not retrieving out MethodInfo objects each time.

So, what could we do in C# 3.0 that would be similar to what is happening with the C# 4.0 dynamic keyword? Well, if you have never played around with it, lets look at the DynamicMethod class. The DynamicMethod class is a helper for creating a method at runtime that we can then invoke. It is doing something similar to what we have above, only in order to use DynamicMethod we have to emit the IL ourselves. This is no simple task, but the easiest way to do it is to compile a handwritten method yourself that does something similar, then reflect it and look at the IL. Then you will at least have a good starting point.

With DynamicMethod we have to write quite a bit of code to get this all wired up, but here it is:

Now that we have our delegates that we can all, we can run the same loops against this method. Again, we call each method once but in this instance all of the work is being done when we emit the IL.

Our four methods are planned out and now we can lay out what you really came here for. Here are the averages:

Compile Time Bound: 6 ms

Dynamically Bound with dynamic keyword: 45ms

Dynamically Bound with MethodInfo.Invoke – 10943ms

Dynamically Bound with DynamicMethod – 8ms

Compile time still wins out, but the dynamic keyword is a much much better looking option now. The overhead of the first call is pretty high, in my tests almost 3 seconds. I’m sure that they will work on the speed of this before the final release. Every successive call is actually really fast though. The next method is MethodInfo.Invoke which is quite slow compared to the rest, but in terms of 2 million calls is still pretty fast. Finally the last method is DynamicMethod, which is practically as fast as the compile bound method because essentially we are just manually compiling it to IL.

There you go, the dynamic keyword has some overhead, but you only have to deal with it once for each dynamic call. In a long running application this isn’t really a concern. I hope you enjoyed this post and I hope that I have answered most of the performance questions that you may have for the dynamic keyword.

In my mind, a speed test isn’t a very good way to characterize the "dynamic" keyword; the wonderfullness I see is with versioning issues. i.e. with the dynamic keyword, you get to load new assemblies into old code and it still works. in the past, I created wrapper classes and used reflection to initialize delegates; with the dynamic keyword, I can skip the wrapper classes since the performance without any further optimizations is sufficient.

I suppose if I was smart, I’d create a custom attribute and create proxy classes on the fly…. using the various methods discussed in these postings…