Verve: A Type Safe Operating System

Description

The Singularity project (an OS written in managed code used for research purposes) has provided several very useful research results and opened new avenues for exploration in operating system design. Recently, MSR released a paper covering an operating system research project that takes a new approach to building an OS stack with verifiable and type safe managed code. This project employs a novel use of Typed Assembly Language, which is what you think it is: Assembly with types (implemented as annotations and verified statically using the verification technology Boogie and the theorem prover Z3(Boogie generates verification conditions that are then statically proven by Z3. Boogie is also a language used to build program verifiers for other languages)). As with Singularity, the C# Bartok compiler is used, but this time it generates TAL. The entire OS stack is verifiably type safe (the Nucleus is essentially the Verve HAL) and all objects are garbage collected. It does not employ the SIP model of process isolation (like Singularity). In this case, again, the entire operating system is type safe and statically proven as such using world-class theorem provers.

Here's the basic idea (from the introduction of the paper):

Typed assembly language (TAL) and Hoare logic can verify the absence of many kinds of errors in low-level code. We use TAL and Hoare logic to achieve highly automated, static verification of the safety of a new operating system called Verve. Our techniques and tools mechanically verify the safety of every assembly language instruction in the operating system, run-time system, drivers, and applications (in fact, every part of the system software except the boot loader). Verve consists of a “Nucleus” that provides primitive access to hardware and memory, a kernel that builds services on top of the Nucleus, and applications that run on top of the kernel.

Here, Microsoft research scientist and operating system expert (he worked on the Singularity project) Chris Hawblitzel sits down with me to discuss the rationale behind the Verve project, the architecture and design of Verve and the Nucleus, Typed Assembly Language (TAL), potential for Verve in the real world, and much more. This is a conversational piece (no demos, no whiteboarding), but if you are into operating research and strategies for building type safe systems at the lowest levels, then this is for you. If you are interested, perhaps we could get Chris into our studio for a lecture or two on OS design.

I recommend that you read the paper and then watch this. The gory details are in the scientific literature. The goal of this conversation was to explore the thinking behind this research project. Of course, as usual, this is a conversation, not a presentation. My random questions aren't too random (I read the paper before we chatted). I like how this turned out. A little long, yes. But there are many good nuggets in this one. Hunt for them, Niners! Use your minds.

erm didnt you say in the video that Verve was available on codeplex or did i miss-heard ?

btw: is the codeplex "The Singularity project" dead ? have been no activity for a long time

You heard correctly. The Singularity Project is more than a single thing (so saying "it's" dead is somewhat meaningless...) I mean, there would be no Verve project without the Singularity research findings. In some sense, Verve is a natural, though with different specific focus (e.g., fully verifiable operating system, typed AL, managed HAL (Nucleus), etc), evolution of the Singularity project. The important point is that OS research is alive and very well both inside and outside of MS. Verve, like Singularity, is about exploring fundamentals and rethinking the OS stack. Also, as is the case with Singlularity the OS, there is no product trajectory for Verve. It's basic science, not product development...

Indeed. Chris and company have been working on TAL for a while now. It is certainly a very related project in this respect. For Verve, all of the code for the OS is statically verified when the system is built. The only unverified code in the system is the boot loader, which is fine in some sense because it only runs once and then is no longer executed... Even the GC is verified (I asked this question in the interview and he said it was or will be soon. At least, that's what I remember).

It would be useful to learn more about TAL specifically. Perhaps we'll get Chris on 9 again ( E2E ) to go over the TAL stuff with a language expert..

I know I am asking redundent question. But, how likely and how soon in estimation would be seeing this in the market? I really really really hope to see this or Singularity in 5 years, but, I am also affraid MS will not take the step and waiting for competitors. I understand their marketing decisions, but, would I be able to see MS releasing Type Safed OS first in the name of practical science?

the interesting thing about this, to me, isn't even that it's an OS, its the use of Z3/boogie as a prover...this speaks directly to turings halting problem. these guys have the coolest job on the planet.