I’m a little concerned that Simon’s response to me was issued on what is listed as the "beta" version of Citrix’s official blog. Perhaps the virtualized version hasn’t made it out of QA yet? 😉

Simon’s response was extremely well crafted to avoid responding to most of my actual points, was contextually oblique at points, and was a fantastic marketing piece for Xen Citrix, but I wish he’d paid more attention to the actual points within my post.

Further his little quips/comments on his hyperlinks "Who is this guy, anyway? Think before you type dude, we’re not idiots," etc. didn’t go unnoticed – cute but juvenile)

I am, however, honored that Simon would accord me the high-status of being "…normally fairly clued-in:"

I
reckon that Hoff, who is normally fairly clued-in, has put the smoking
end of the cigar in his mouth before thinking through this argument.
He’s horribly confused, but as smug as always, so let me clarify what I
said, and what it means.

…but I can assure you that I’ve only ever done that with a cigar once,
and it was for a much better reason than blogging. If you must know,
it was Kentucky’s finest bourbon. That is all I’m going to say about
that.

I’m glad he’s "clarifying" what he said, since I will also. I seem to have that effect on people. Must be the accent thing…

The reason for my allergic reaction to Simon’s comments stem from my opinion that it is the responsibility of virtualization platform providers to ensure that their "[virtualized] data center operating system platforms of the future" don’t become the next generation of insecure infrastructure.

Simon sums up his opinion:

In summary an assertion that the virtualization platform vendor has
to fix the sad state of the OS/App world by making it secure is
demanding too much. It would mean that we have to be experts in every
piece of system software including all of the vulnerabilities of all
OSes and their apps. In my view the reason the state of security is
poor now is because of the monolithic approaches of traditional OS and
app vendors.

We will focus manically on our layer, make it
secure, tiny and bulletproof to attack in its own right. And we will
work closely with experts in security of OSes and Apps to give them an
opportunity to implement guest-level security outside the guest,
through privileged interfaces that themselves are secure.

After 15 years of dealing with this crap, I respectfully suggest that it is not too much to ask and it’s about time we stood up and did. First you criticize OS/App. vendors and blame them for the state of security because of their "monolithic approach" and then you go on to propose the exact same thing!

Focusing only on your little patch of grass is short-sighted and it won’t work. Just like it hasn’t worked in the past. It’s a disaster waiting to happen, and you’re enabling it.

I shudder at the potential tunnel vision of virtualization platform providers only focusing on the security of the hypervisor without taking the bigger picture into consideration and expect a piecemeal approach to securing the expanse of the virtualized environment to suffice.

It’s clear you’re making arguments about security from an engineering and code-base perspective that is simply disconnected from the realities of what it means to actually deploy these solutions.

Virtualization is more than just the hypervisor. You should know that by now, Simon. The company that acquired your company knows all about that. The hypervisor will shortly become a commodity, so in the long term the value brought to bear has to be more than just an ultra-thin layer of code:

…and furthermore, we’re going to deploy many of them:

I wish to make it clear that I hold all virtualization platform vendors to the same level of scrutiny and criticism, not just Citrix.

I happen to like Xen very much. I like VMware, also. I think the latter is more realistic and measured when it comes to addressing the need and approach in recognizing that as a major layer in the infrastructure, there’s more required than to just secure the hypervisor and leave the remaining mess to someone else to solve.

I think Simon’s blog title is apropos, but I think the misunderstanding is his.

It’s important to understand that I’m not suggesting that virtualization platform providers should secure the actual guest operating systems
but they should enable an easier and more effective way of doing so when virtualized.

I mean that the virtualization platform providers should ensure the security of the instantiation of those
guests as "hosted" by the virtualization platform. In some cases this means leveraging technology present in the virtualization platform to do things that non-virtualized instances cannot. That’s more than just securing the hypervisor.

Securing the hypervisor whilst closing your eyes to the likelihood
that the majority of attacks against it and other guests will come from "guests" within the same system is planting your head in the sand. That means that there will be a need to ensure that certain behaviors specific to the hosted guests are mitigated to ensure that bad things don’t happen — to the guest or the hypervisor.

Transferring the responsibility to secure the environment to third party security ISV’s in order
to secure the VM’s
and preventing them from compromising one another or the hypervisor is
difficult for me to comprehend, especially when they are playing catch up of what virtualization means within the context of security.

Fundamentally, attempting to mate static and topology-dependent policies to incredibly dynamic and transitive technology delivered by virtualization will simply fail. Third party security ISV’s will simply require a complete re-tool to even get close to delivering this and will need to provide intimate hooks to allow for this policy/guest affinity to occur in the first place.

I consider the virtualization infrastructure layer as that of an operating system and as such, I would expect that the underpinning mechanicals are as sound and secure as possible while also ensuring that anything running on top of it is as secure as possible, also.

Let’s take Microsoft (with or without Hyper-V) as an example:

Microsoft is fundamentally concerned now with making the OS as
resilient and secure as possible whilst preventing the applications and
interaction with elements riding on top of the OS from doing bad things
to the system as a whole; this isn’t just to protect the OS, but the
assets on it.

This is really what I’m getting at. Yes, Microsoft is an OS provider. Shortly, that OS provider will integrate virtualization directly into the operating system. That means more, not less, direct integration and security embedded as a function of the virtualization platform. Citrix, VMware, etc. are all just operating system vendors of a different shape and size.

It’s unclear to me, Simon, whether your arguments are meant to justify a business model, a lack of planning, a crafty plan to perpetuate the security hamster wheel of pain, or all of the above. It’s clear to me, however, that you’ve not felt the pain of actually having to use the products you suggest should be deployed in order to secure this mess.

I promised myself I wouldn’t turn this into one of those cut/paste blog pong entries, but the following really confused me:

But we are not in the business of specifically securing guests or their
applications, other than through offering a secure virtualization
platform. Even VMware with VMsafe simply exposes APIs to third party
security vendors, so that customers can choose their preferred security
partner to secure guests. I think that the VMware Determina
acquisition was very smart, and that hints to me that VMware sees
itself having a greater role in the security of guest OSes, since it
could choose to be in the vulnerability checking business without 3rd
party security vendors, but thus far they are working very openly with
the ecosystem.

So which is it? You’ve established that Citrix is not in the business of securing guests or applications (you must mean Xen specifically, because somebody at Citrix spent quite a bit of money on this stuff with their other acquisitions) and that you believe it to be a lousy idea, but you think that VMware’s approach through their Determina acquisition as well as the capabilities of VMsafe is "…very smart?"

Simon, you’re the CTO and I’m the security wonk. If we didn’t disagree, I’d be alarmed. However, I think you might want to rethink your approach to how you market the security of your platform.

I forget who said it, but an appropriate quote is "There is nothing more abhorrent than a vested interest masquerading as a moral principal." The one-upmanship is starting to look sad. Security IS everyone's issue, but it can't be the central focus.

@Ring0 To be honest, I can't tell who you're aiming at with that comment. Perhaps both of us?
I don't see this as one-upmanship on either front; I'm sure Simon believes wholly in what he's saying and I certainly am passionate about bringing attention to what I feel is a huge problem.
I don't think it's a "moral" principle or that anyone's approaching it as such. It's just an impassioned debate, is all.
Security *is* everyone's issue, that I certainly do agree with.
If I misinterpreted your comments, do let me know or at the very least please clarify.
Thanks,
/Hoff

Valid discussion and you argument holds merit Hoff. I am not sure who Ring0 was speaking to either but if it was to you, I fail to see the relevance of his comment to your post.
What I am fearful of overall is that the very valid company that Citrix acquired is likely to go the way of the DoDo bird if they do not grasp the role they could/should play going forward in this rapidly evolving space.
I am sure cheers went up from VMWare the day that acquisition was announced and yet I was hopeful. If his answer is any indication then that hope was possibly misplaced.
–D

I think there will be a close relationship between security and data center virtualization. The first vendor who truly gets this causal link (versus "damn the torpedos full speed ahead" will win. The last will lose.

@David:
I wrote a post about how I thought the acquisition of XenSource by Citrix was a great idea (it was also a really interesting block/tackle move that will ultimately help Microsoft and position against Cisco, also…another topic for another day…)
The post is here: http://rationalsecurity.typepad.com/blog/2007/08/…
I still maintain that XenSource to Citrix is still about application delivery.
I'm not sure Simon gets that yet…I'm also not sure who he thinks he works for. He keeps referring to Xen and XenSource almost to the exclusion of Citrix and the greater strategy for the company. It's really quite humorous.
Hey, but given Simon's embedded comments on his hyperlinks, he doesn't need to worry…I'm a nobody. He's right. In the grand scheme of things, I am. The problem is, of the thousands of people who read this blog aren't nobody's and many are the folks he hopes to sell to.
I don't think Citrix has to worry about the technology becoming a DoDo, but perhaps the folks that continue to talk about some of their technology in a vacuum representing them might.
/Hoff

Valid discussion and you argument holds merit Hoff. I am not sure who Ring0 was speaking to either but if it was to you, I fail to see the relevance of his comment to your post.
What I am fearful of overall is that the very valid company that Citrix acquired is likely to go the way of the DoDo bird if they do not grasp the role they could/should play going forward in this rapidly evolving space.
I am sure cheers went up from VMWare the day that acquisition was announced and yet I was hopeful. If his answer is any indication then that hope was possibly misplaced.
–D

Hoff:
Virtsec is really a "worlds in collision" phenom bringing at least two distinct IT groups together at a rapid pace (server ops and netsec). I think this collision will result in plenty of misunderstandings as each group grasps the implications of one action/viewpoint to another. It kind of reminds me of your comments about Oz and rogue virtualization projects.
I think Simon views the world from a server ops standpoint, which makes perfect sense given his background. It also makes sense that he would see security as another's problem.
That's why I think the first vendor who truly gets how strategic security is to the growth of virtualization in production environments will win. The worlds won't collide and create a messy scenario (leaving homes stranded along freeways- to refer again to your earlier blog) but rather bring about cohesion and synergy.
The vast virtsec vendor eco-system may help these worlds come together but I don't think that has happened yet. And clearly Simon is willing to wait for the market to "mature" and enable virtualization of production. Others may not want to wait.
Greg
Blue Lane

@Kevin – The answer is yes, and Simon alluded to providing that capability, but not when.
Further, when VMsafe was officially announced, Crosby's complaint against VMware was that they had developed a proprietary solution rather than an open federation across virtualization platform providers.
I wonder if they had announced "XenSecure" (or something like that) first, if his opinion would have been the same.
As I mentioned, the reason why VMware will continue to lead as the hypervisor continues to commoditize is because of how they have embraced solving the multidimensional problem of the next generation data center; it's about balancing how much you fork off to the ecosystem and how much you do yourself.
Choosing one over the other is a bad strategy.
/Hoff

I find it rather amusing that you get slammed by the industry people on one side and pilloried by the researchers on the other both calling you a heretic for nearly completely opposing reasons.
I think that is definitely one of the signs that you are providing value.
Keep it up.
–D
PS. As far as the nobody thing, it is my belief that we have been led around by our nose rings in this profession for far too long. I find it amusing that when practitioners/customers/consumers begin to step up and say "wait a sec"…the responses we get. The Ninja Monkeys get all antsy and stuff like "HOW DARE YOU!". Quite amusing.
–D

I find it rather amusing that you get slammed by the industry people on one side and pilloried by the researchers on the other both calling you a heretic for nearly completely opposing reasons.
I think that is definitely one of the signs that you are providing value.
Keep it up.
–D
PS. As far as the nobody thing, it is my belief that we have been led around by our nose rings in this profession for far too long. I find it amusing that when practitioners/customers/consumers begin to step up and say “wait a sec”…the responses we get. The Ninja Monkeys get all antsy and stuff like “HOW DARE YOU!”. Quite amusing.
–D

Depending on the situation and who you listen to, you can get different answers to this question. This is not the same debate that Simon Crosby (Citrix CTO) and Chris Hoff have been having. This question is aimed at the customer – who cares about virtu…