PLEASE NOTE: I HAVE PERMANENTLY MOVED MY BLOG TO http://www.rationalsurvivability.com/blog

April 13, 2008

An Open Letter to Joanna Rutkowska

Dear Joanna:

I attended your session at the RSA conference last week titled "Security Challenges in Virtualized Environments" and was compelled to write you given our debate which you ended somewhat abruptly at the conclusion of your presentation.

Before I start in on the meat of the topic, I'm going to do what you seem to continue NOT to do. Specifically, I am going to make clear certain disclosures and frame the context of this note in a way that I hope everyone can understand.

Sadly, there will not be an accompanying eight-slide melange of virtual machine state transitions, mention of TLB misses, GIF0 emulation or ASID conflicts...

Back to your presentation.

As the room filled to over capacity before your talk began, you were upset and couldn't seem to understand why the conference organizers would not let people spill over from seats and sit on the floor and in the aisles to hear you speak. The fact that fire and safety codes prohibit packing a room beyond capacity was something you attributed to people being "...crazy in America." Go figure.

So let me further raise your ire by introducing you to another crazy American rule of law that is somewhat related: we don't think it's a good idea to yell "fire!" in a crowded theater, either.

What does this have to do with your presentation? It's quite simple actually. I think that the way in which you are presenting your research is intentionally designed to be sensational first and concise and accurately portrayed a distant last.

During your presentation at RSA and throughout other presentations,
you have illustrated how your research featuring Blue Pill technology affects hardware-based type-2 hosted virtualized environments rather than type-1 bare-metal installs.

In many cases, given the depth and complexity of your presentations,
less experienced audience members and members of the press have
completely confused or overlooked this distinction and left your presentation thinking that your research and your testing applies directly and unequivocally to both environments, despite the fact that you continue to highlight Microsoft's Vista desktop operating system as your test case.

When I spoke to you at the end of your presentation and made sure
that I understood correctly that you were referring specifically to type-2 hosted virtualization on specific Intel and AMD chipsets, you conceded that this was the case.

When I attempted to suggest that while really interesting and intriguing, your presentation was not only confusing to many people but also excluded somewhere north of 80% of how most adopters have deployed virtualization (type-1 "bare-metal" vs. type-2 hosted) as well as excluding the market-leading virtualization platform, your response (and I quote directly) was:

"I don't give a shit, I'm a researcher."

So my problem with that answer is three-fold Joanna:

As a researcher who is also actively courting publicity for commercial gain and speaking at
conferences like RSA which are less technical and more "executive" in
nature, you have a responsibility to clarify and not obfuscate
(intentionally or otherwise) the facts surrounding your research.
Allowing the continued sensationalized coverage of your research
without clarification is not allowing concerned people to make clearly
informed decisions regarding risk.

No less than five times during your presentation, you highlighted marketing material in the form of graphics from Phoenix, positioned their upcoming products and announced/credited both Phoenix and AMD as funding your research.

Further, there have been announcements suggesting that Phoenix is looking to commercialize Blue Pill not as a rootkit but as an "ultra-thin" hypervisor. This makes it hard to decide where the breakpoint between your "research" versus their "commercial" begins.

Continuing to openly and negatively disparage those who seek to challenge your assertions is unprofessional. Certainly you can disagree with them, but regardless of their approach or attitude, the continued pejorative nature of your rebuttals is getting stale.

I think it's only fair to point out that given your performance, you're not only an "independent researcher" but more so an "independent contractor." Using the "I'm a researcher" excuse doesn't cut it.

I know it's subtle and lots of folks are funded by third parties, but they also do a much better job of drawing the line than you do.

Despite your position on the matter and unlike you, I do give a shit, Joanna. I care very much that your research as presented to the press and at conferences like RSA isn't only built to be understood by highly skilled technicians or researchers because the continued thrashing that they generate without recourse is doing more harm than good, quite frankly.

Now, I know you can't control the press or what they print, but you certainly don't seem to invest much in terms of ensuring accuracy or clarifying the corner cases you're talking about. Here's an example from a Forbes article based upon your RSA presentation:

At the security industry's big annual confab, the RSA Conference, going on this week in San Francisco, security researcher Joanna Rutkowska described a new type of virtualization-based malware that could be used to take control of a machine running virtualization software. Because virtualization allows companies to store many virtualized software "images" of computers on a single physical machine, an attack like the one Rutkowska envisions would allow a hacker not only to control a single machine but to siphon data from any virtual machine it contains.

Rutkowska, the founder of security research firm Invisible Things Lab, in Warsaw, Poland, isn't the first to target virtualization as a weak point in the emerging IT landscape. In the past few months, security researchers have revealed bugs in practically every piece of virtualization software, including products from virtualization heavyweights VMware (nyse: VMW - news - people ) and Microsoft (nasdaq: MSFT - news - people ).

Exploiting those bugs, attackers can use what researchers call "virtual machine escape," or "hyperjacking." By taking control of the hypervisor, the piece of software that controls all the virtual computers within a machine, an attacker can "escape" from any single virtual computer hosted on the machine and quickly multiply his or her access to a company's data.

But the attack Rutkowska outlined goes even further: she described how an intruder could install what she calls a "blue pill," a second, malicious hypervisor that controls the original hypervisor and all of the virtual machines beneath it. Examining any PC or server hosted on the machine, it would appear that the machines were hosted normally by a hypervisor, but, she argues, it would be tough to detect another hidden hypervisor intercepting data or manipulating the virtualized computers.

"When you use virtualization to build malware, there are no hooks, nothing you can see within an operating system," she says.

So this reporter walked away from your presentation and basically represents -- like every other reporter I have seen -- that every virtualization platform is covered under your research and is susceptible to attack regardless of chipset, operating system or application, a fact that you already conceded during our short exchange as not being the case. Do you not see how this can be confusing?

In this scenario, I can personally attest that Fortune 100 companies deploying VMware ESX 3i are unable to determine whether they are at risk or not. You could certainly take the low-road and blame this on those interpreting your presentations this way, or perhaps recognize that this could be a direct result of your efforts.

Despite the fascinating research, I'm really disappointed in how you choose to continue to allow inaccurate representation of your research to continue unabated. Instead of inflammatory, sensational and inaccurate portrayals, you could instead be really helping to educate the world in a way not dependent on fear, uncertainty and doubt.

I look forward to your next presentation. I just hope it's more accurately tempered next time so as not to cause the figurative stampede from the theater when there's actually no fire.

/Hoff

Update: Joanna responded here. I retorted playing ping to her pong here. Enjoy.

Comments

An Open Letter to Joanna Rutkowska

Dear Joanna:

I attended your session at the RSA conference last week titled "Security Challenges in Virtualized Environments" and was compelled to write you given our debate which you ended somewhat abruptly at the conclusion of your presentation.

Before I start in on the meat of the topic, I'm going to do what you seem to continue NOT to do. Specifically, I am going to make clear certain disclosures and frame the context of this note in a way that I hope everyone can understand.

Sadly, there will not be an accompanying eight-slide melange of virtual machine state transitions, mention of TLB misses, GIF0 emulation or ASID conflicts...

Back to your presentation.

As the room filled to over capacity before your talk began, you were upset and couldn't seem to understand why the conference organizers would not let people spill over from seats and sit on the floor and in the aisles to hear you speak. The fact that fire and safety codes prohibit packing a room beyond capacity was something you attributed to people being "...crazy in America." Go figure.

So let me further raise your ire by introducing you to another crazy American rule of law that is somewhat related: we don't think it's a good idea to yell "fire!" in a crowded theater, either.

What does this have to do with your presentation? It's quite simple actually. I think that the way in which you are presenting your research is intentionally designed to be sensational first and concise and accurately portrayed a distant last.

During your presentation at RSA and throughout other presentations,
you have illustrated how your research featuring Blue Pill technology affects hardware-based type-2 hosted virtualized environments rather than type-1 bare-metal installs.

In many cases, given the depth and complexity of your presentations,
less experienced audience members and members of the press have
completely confused or overlooked this distinction and left your presentation thinking that your research and your testing applies directly and unequivocally to both environments, despite the fact that you continue to highlight Microsoft's Vista desktop operating system as your test case.

When I spoke to you at the end of your presentation and made sure
that I understood correctly that you were referring specifically to type-2 hosted virtualization on specific Intel and AMD chipsets, you conceded that this was the case.

When I attempted to suggest that while really interesting and intriguing, your presentation was not only confusing to many people but also excluded somewhere north of 80% of how most adopters have deployed virtualization (type-1 "bare-metal" vs. type-2 hosted) as well as excluding the market-leading virtualization platform, your response (and I quote directly) was:

"I don't give a shit, I'm a researcher."

So my problem with that answer is three-fold Joanna:

As a researcher who is also actively courting publicity for commercial gain and speaking at
conferences like RSA which are less technical and more "executive" in
nature, you have a responsibility to clarify and not obfuscate
(intentionally or otherwise) the facts surrounding your research.
Allowing the continued sensationalized coverage of your research
without clarification is not allowing concerned people to make clearly
informed decisions regarding risk.

No less than five times during your presentation, you highlighted marketing material in the form of graphics from Phoenix, positioned their upcoming products and announced/credited both Phoenix and AMD as funding your research.

Further, there have been announcements suggesting that Phoenix is looking to commercialize Blue Pill not as a rootkit but as an "ultra-thin" hypervisor. This makes it hard to decide where the breakpoint between your "research" versus their "commercial" begins.

Continuing to openly and negatively disparage those who seek to challenge your assertions is unprofessional. Certainly you can disagree with them, but regardless of their approach or attitude, the continued pejorative nature of your rebuttals is getting stale.

I think it's only fair to point out that given your performance, you're not only an "independent researcher" but more so an "independent contractor." Using the "I'm a researcher" excuse doesn't cut it.

I know it's subtle and lots of folks are funded by third parties, but they also do a much better job of drawing the line than you do.

Despite your position on the matter and unlike you, I do give a shit, Joanna. I care very much that your research as presented to the press and at conferences like RSA isn't only built to be understood by highly skilled technicians or researchers because the continued thrashing that they generate without recourse is doing more harm than good, quite frankly.

Now, I know you can't control the press or what they print, but you certainly don't seem to invest much in terms of ensuring accuracy or clarifying the corner cases you're talking about. Here's an example from a Forbes article based upon your RSA presentation:

At the security industry's big annual confab, the RSA Conference, going on this week in San Francisco, security researcher Joanna Rutkowska described a new type of virtualization-based malware that could be used to take control of a machine running virtualization software. Because virtualization allows companies to store many virtualized software "images" of computers on a single physical machine, an attack like the one Rutkowska envisions would allow a hacker not only to control a single machine but to siphon data from any virtual machine it contains.

Rutkowska, the founder of security research firm Invisible Things Lab, in Warsaw, Poland, isn't the first to target virtualization as a weak point in the emerging IT landscape. In the past few months, security researchers have revealed bugs in practically every piece of virtualization software, including products from virtualization heavyweights VMware (nyse: VMW - news - people ) and Microsoft (nasdaq: MSFT - news - people ).

Exploiting those bugs, attackers can use what researchers call "virtual machine escape," or "hyperjacking." By taking control of the hypervisor, the piece of software that controls all the virtual computers within a machine, an attacker can "escape" from any single virtual computer hosted on the machine and quickly multiply his or her access to a company's data.

But the attack Rutkowska outlined goes even further: she described how an intruder could install what she calls a "blue pill," a second, malicious hypervisor that controls the original hypervisor and all of the virtual machines beneath it. Examining any PC or server hosted on the machine, it would appear that the machines were hosted normally by a hypervisor, but, she argues, it would be tough to detect another hidden hypervisor intercepting data or manipulating the virtualized computers.

"When you use virtualization to build malware, there are no hooks, nothing you can see within an operating system," she says.

So this reporter walked away from your presentation and basically represents -- like every other reporter I have seen -- that every virtualization platform is covered under your research and is susceptible to attack regardless of chipset, operating system or application, a fact that you already conceded during our short exchange as not being the case. Do you not see how this can be confusing?

In this scenario, I can personally attest that Fortune 100 companies deploying VMware ESX 3i are unable to determine whether they are at risk or not. You could certainly take the low-road and blame this on those interpreting your presentations this way, or perhaps recognize that this could be a direct result of your efforts.

Despite the fascinating research, I'm really disappointed in how you choose to continue to allow inaccurate representation of your research to continue unabated. Instead of inflammatory, sensational and inaccurate portrayals, you could instead be really helping to educate the world in a way not dependent on fear, uncertainty and doubt.

I look forward to your next presentation. I just hope it's more accurately tempered next time so as not to cause the figurative stampede from the theater when there's actually no fire.

/Hoff

Update: Joanna responded here. I retorted playing ping to her pong here. Enjoy.