Ian Carmichael: The History and Future of the CLR

Description

Ian Carmichael has been working on the CLR team since before Microsoft came up with the .NET branding for our managed platform and the virtual machine that powers it all, the Common Language Runtime. Well, we're getting close to the third major release
of the CLR, CLR 4 (V3 was really a service or minor release, but who's counting?).

Back in the good old days prior to V1, Ian was an engineer and a peer of Chris Brumme, Vance Morrison and other top of the line engineers working through the architecture, design and implementation of the CLR. Now, Ian is the GM of CLR and he's at the helm
plotting out the course for CLR's future. Necessarily, we had to sit down with him to pick his brain about CLR then, now and tomorrow. Tune in. Get a glimpse into the past and future of the CLR from somebody who's been deeply involved with the advent and evolution
of Microsoft's now ubiquitous managed runtime.

The Discussion

I'm not convinced by Ian's claim that IL is necessary for a common language interop. You can create an ABI that works between languages - indeed you can link C/C++/Fortran and other languages on many platforms, even though they are compiled to binary in
one step.

There are however many other good reasons for CIL (and similar bytecode systems):-

Allows a single assembly to be compiled to different native machine architectures. This is important even for Windows since you have 32 and 64 bit runtimes now.

Provides a nice 'forking point' for the common infrastructure used by multiple languages. You realistically wouldn't write a complete source-to-binary compiler for most languages, you re-use a lot of stuff. So you have to have an interface at some level
of the stack. Taking the GNU Compiler Collection as an example, they have multiple language front-ends that generate an intermediate form called RTL which is them turned into machine code for different platforms by a common backend. CIL exposes that level
of interface more widely to the programmer. Which is greate because it's not just rocket scientist compiler guys that have a need to generate code.

It's makes code verification very much easier. In theory you can verify machine code for safety if the compiler obeys some very specific rules, but it's a lot harder than verifying something like CIL

I think the original argument was that you would just go from source to native cpu instructions.. because he (the v8 guy) was talking in context of javascript.. to me its an obsurd misunderstanding of .net if you expect to write pure source and jit straight
to native in one swoop.. it might be do'able but why the hell would you (at least, why would you want to move everything to that, there was a good argument that dynamic langs could benefit from skipping the IL state (seems problematic))?

For JavaScript (or any singular language framework), syntax to cpu instructions makes perfect sense if your aim is to eek out as much performance as you can. It's because of byte codes, for example, that the CLR and JVM support multiple languages. The goal
of the CLR has always been to enable n number of languages that get compiled down to a single form that is readily reasonable at runtime. This has benefits for multiple language project, debugging and runtime optimization. There's more the JIT could
do in the future. I think some of the naysayers just don't like MSIL, specifically. The folks I've talked to about it believe that MSIL is too closely related to C# and is not as generalized as we claim. That's not an area where I can chime in since I have
nothing to do with MSIL design and do not have the knowledge to present Microsoft's formal opinion on the matter... Still, I like to hear what others think. Opens up the box.

Debating is a very important part of learning and assimilating information, distilling it into working knowledge. I want more debate, not less. I wish I could get Lars Bak together with Vance Morrison and let the games begin! Lars is a brilliant guy.
It's unfortunate I didn't capture his stance in the completeness it deserves. Still, I am a big fan of IL/bytecodes. There's just so much possibility for reasoning and adaptation in runtime context. So much so, that we've barely even scratched the surface
of what's possible, especially as processing power becomes astronomical...

The future is bright, but you have to look into the light, not away from it.

Heh charles I'm certainly not set in my ways.. but what lars basically said was he thought people should just write c#, vb, f#, ironpython -whatever-... and what .net should be is an array of runtime compilers (like v8) for EACH language.. that just completely
ignores the reasons why .net has IL in the first place... like Ian says.. there are
places in .net where being able to go from source to native without IL would be useful.. and that would be for scripting languages, where compilation is done (or at least mostly so) at runtime.. in which case theres no reason to go to IL..

But here's the thing.. whilst .NET has two compilation steps:

Source -> IL -> Native

And perceiveably, javascript has one step:

Source -> Native

Thats not the reality... Source -> IL happens at compile time so the runtime then comes down to:

IL -> Native

vs.

Source -> Native

I'm no language designer or compiler guru, but I would expect that IL (which is built by a machine).. is much faster to compile to native than source (written by a human, complicated syntax to make it friendly for the human to write).. and therefore his
argument really stuck out to me as ignorance..

MSIL is source. That's the point. MSIL is a human-readable form of the higher level syntax used to construct it. But, yes. The JIT is fine tuned to blaze through IL. It comes down to the usual suspect: relative philosophy. Lars is a machine guy.
He doesn't like the notion of intermediate steps on the path to x86. That's his perogative. Classifying this as ignorance on his part is probably out of scope. His position is based on several years of writing virtual machines. However, you're right in that
he has little experience with .NET.

Another problem with MSIL is that it has no abstractions for vector instructions in CPU's (and modern GPGPU's) that could be useful for exploiting data parallelism in large-scale (and not so large-scale) scientific computing problems. The Mono team has
a Mono.SIMD assembly in the works for this. I'm hoping the CLR/CIL folks are working on something similar.

I didn't get the impression from watching Charles and Eric's interview with Lars here on C9 and the videos of his Google I/O talks on YouTube that Lars was espousing a naive source-to-assembly conversion. I suggest watching his talks again to get a clear
understanding how V8 actually works.

I am hopeing that is what was implied when Ian stated the math/science area. There are a lot of areas where performance and be explicitly improved with SSE and GPGPU support. I don't believe there would be much involved to add that support, the JIT engine
would have to be smart enough to understand the underlying hardware that is available to choose level of sse support for example or go the safe way and choose the lowest common denominator between amd and intel.

GPGPU gets more interesting, dx11 introduces compute shaders, but to be honest, it is a subset of what is available in the GPGPU area on both nvidia and amd, and then you have Intel embedded video which throws an interesting aspect to that area. (or the
other video card oem's) Open CL imo is a more robust platform on the outset but I don't see dx resting on its morals going forward either.

The question then becomes how to support on downlevel machines?? Talking cells phones etc.

i agree with stevo_ on this one, but it seems that everyone is pretty much in agreement here but dont realize it IL is good for a runtime that supports lots of languages, source to native is good for a language that only supports one runtime. Lars is correct in the context of V8 but perhaps not so much in the context of .net [of witch he presumably has far less experience]
nothing wrong with that.

stevo does make a great point though, IL is faster to compile than source. and charles, while it is true that IL is source, you must agree that its optimized for jitters more than humans by the same token, assembly code is source too i mean, technically
it is, but not really in practice for most people. human readable is a relative term i suppose

il is really bare bones as far as lexing and parsing goes, its bacically just higher order instructions right? c#, vb js and pretty much all "normal" languages are more optimized for humans so they wont compile as fast i think.

also, consider tooling, if one is to write a tool for .net one only really have to understand IL, not a bunch of native code for various platforms.. thats a huge advantage

Lars had pretty good experience in VMs, and he had worked on JVMs which are quite similar to CLR in general, I'm sure he's knows enough about bytecodes/IL. I wonder if Charles read my reply in the other thread (CLR4 debugging and profiling API). I was trying
to say, when Charles started the topic on IL, Lars was probably still in the context of V8/JavaScript, so he said what he said, and it made perfect sense. Having bytecodes in V8 wouldn't have helped much in keeping the VM fast and simple. And that you'll have
to "check twice" if you had bytecodes as a wire format for JavaScript. Charles was in the context of CLR or a more general VMs, the two guys really weren't arguing about the same thing...

Going from source to bytecodes is much easier than getting straight to native code. A lot of JavaScript VMs are imlemented as interpreters, because interpreters are easier to build. So they would interpret bytecodes, and that's the only execution mode they
have. V8 also has only one execution mode, and that's compiling straight to native code.

But there are VMs, like TraceMonkey, that use adaptive compilation system, and use bytecode for startup and profiling/tracing, and hot spots get compiled/optimized into native code. That's having two execution modes (or more) in the same system, which leads
to complications in design and implementation. Lars had worked on such adaptive compilation systems before, so he knows why it's not suitable for the goal of V8 -- build a fast and simple JavaScript VM in short amount of time.

In my eyes IL isn't similar to C#. It's true that you can execute methods and that everything is organized in classes, namespaces, methods, etc. - but that's more a property of OOP. IL is organized as a stack machine (you clearly see that if you do something
with it) and C# is not exposing that at all.

Actually, I read your post and you're asumption is incorrect. As stated, the argument was focused on the necessity of IL,
generally (and this includes Java bytecodes). The debate continued for at least 40 minutes after the camera was turned off. It was me versus Erik and Lars. As you can imagine, I had my work cut out for me... Erik isn't a big fan of MSIL, specifically.
Lars thinks intermediate steps add unnecessary abstraction, always. That's all I'm going say on this since I will not speak for other people. If you want to know the details of their thinking, then you will have to ask them yourselves.

Regardless, this is a useful debate.

There are pros and cons all around us, all of the time. Understanding what they are
exactly is key to becoming knowledable and readily capable of critically thinking through various contexts in which some set of pros and cons may take on a more dynamic characteristic moving closer to or father from one another.

Well...Walter Bright of Digital Mars doesn't seem to like the idea of having a VM/IL, either. And he designed the D language to have GC without a VM. The interview on CLR4 security changes mentioned you can do unsafe things with native code anyway, so VMs
doesn't really buy you that last bit of security; that's OS' job.

My take is that IL is just like what would have been an intermediate representation in between front-ends and back-ends of a conventional compiler; in other words, you're just splitting a compiler into two parts, with some of the machine independent parts
packed into a "compiler from source to IL", and machine independent parts with machine dependent parts packed into the VM. Then of course you can interpret IL, but in systems like the CLR, the JIT compiler is what would have been a back end of a conventional
compiler, with time-consumption taken into account -- time-costly optimizations can't be performed because it'd hurt startup perf.

So deploying a managed app sounds like...distributing the sources (MSIL), and making sure a compiler (CLR) for the platform is installed. And when the user runs the managed app, CLR compiles the app and runs it. So for M source languages with N target platforms,
you'd only have to write M compiler that target MSIL, and N CLR implementations for each platform, that's M+N instead of M*N in a traditional compiler architecture without a standard intermediate representation. JITting has this advantage of being able to
dynamically linking libraries, and generating calls into the libraries with less indirection, because the JITter has knowledge of all methods' entry points; NGEN can't do that, so it's throughput is a bit worse than JITted code. That's why they had the troublesome
hardbinding in NGEN before CLR4.

And generally bytecodes are easier to verify than native code. That buys you verifiablity, for JVM and CLR, that's important. But bytecodes are actually "harder to verify" than source code, because source languages tend to have more restrictions than their
corresponding bytecodes.

Ah, and speaking of compilers...NGEN uses the JIT compilers as CLR does, but used in ahead-of-time compilation mode. NGEN is slow because it compiles everything, while JITting just compiles what's been invoked, so the compilation time is amortized and affordable.
If you add up the total time to JIT everything used in a managed program, it'll probably take the same amount of time as NGEN does...

"back to fundamentals, how do we make interop better? (around the 30 minute marker)"

In my opinion you could make interop better if debugging com objects were first class citizens in Visual Studio. I have know idea where the problem(s) lye so this might not be a framework issue. All I can say is the debugging features I got use to with
.Net object went out the door with com objects.

For example, if I have an object Blah with a property of BlahProp when I click the plus symbol in the watch window of an object whose type is Blah I will see BlahProp if it is a .Net object. But if it is a com object I will not see BlahProp.