April 11, 2011

GigaSpaces OpenStack Explained

One of the major concerns of many IT organizations is cloud vendor lock-in. This concern was expressed recently in "Banks fear cloud vendor lock-in," from IT Wire:

The onset of cloud computing gives vendors the chance to lock customers in to their infrastructure, using proprietary protocols to ensure they’re on the monthly billing cycle as long as possible.

The OpenStack project emerged with a mission to address this concern by creating a community-led open source project enabling any organization to create and offer cloud computing services running on standard hardware.

GigaSpaces joined the OpenStack project with the mission to enable any organization to build its own Platform-as-a-Service (PaaS), with its own choice of language and best-of-breed middleware stack.

In this post, I’ll try to provide more insight into our current and future plans around OpenStack, and more specifically our joint collaboration with the Citrix OpenCloud initiative.

GigaSpaces OpenStack Explained

What does GigaSpaces' OpenStack support mean?

One of the goals for our second-generation PaaS/SaaS enablement platform was to enable smooth migration between different cloud providers. We were able to achieve this goal through the use of our own abstraction (the Scaling Handler) and through the integration with the JClouds project that provides common abstraction to most of the existing cloud providers. With that, we can ensure that any application can be moved from the likes of Amazon to OpenStack or to an organization's own private cloud with zero changes to the application code or configuration.

The only change involves is setting the user/key of the specific cloud.

By adding support for OpenStack, we now enable users to safely move to an OpenStack-based cloud when they're ready and with little effort, yet they gain all the benefits that comes with it in terms of cost, openness etc.

Can I use the OpenStack integration outside of GigaSpaces' platform?

Yes. The OpenStack integration was developed in close collaboration with GridDyanmics and Adrian Cole from JClouds. We intend to make the integration with OpenStack available to the entire Java community through the open source JClouds project. In this way, any Java application can easily run on OpenStack.

What specific application stack is currently supported?

Today, there are a wide variety of tools and frameworks available throughout the ecosystems in Java, .Net, Ruby, and more. We feel that limiting the platform to a specific stack that we control -- as in the case of Google App Engine -- is too restrictive. Instead, we wanted to create a foundation that enables users of the platform to easily host any service or framework they choose on the platform. We also wanted to do this in a way that provides consistent behavior in terms of deployment, monitoring, elasticity, and scaling. Therefore, we developed a Universal Service Manager and Service Orchestration framework. The framework makes it easier users to plug in their own choice of services that comprise the application stack (Tomcat, MySQL, NoSQL through Cassandra, Hadoop, Ruby...).

In addition, we will provide a set of built-in recipes. At first we will be targeting Big Data stacks, which include integration with NoSQL and Hadoop, and also reporting tools to make the deployment of Big Data applications significantly simpler.

The integration with OpenStack will make it possible to bring these benefits to any application running on OpenStack.

How is this related to the previously announced Open Elastic Platform with Citrix?

Openness at the IaaS layer -– The specific integration with NetScaler (load balancer) and XenServer would be done through more open interfaces provided through the Citrix/Openstack contribution, which means that OpenStack users can use those intefaces to plug in any hypervisor or load balancer.

Openness at the application layer –- The previous version of GigaSpaces XAP offered a limited set of application stack support features, mostly geared to services that we control and mostly in Java. With the new platform we offer significantly greater flexibility on various fronts:

Open to any application stack and language: As I mentioned earlier, users can now easily host any application or service and build their stack of choice, while at the same time managing and controling their application in a consistent way in terms of the deployment, monitoring, elasticity, etc.

Users can choose their container of choice: The new platform will include support for Tomcat in addition to our existing Jetty support.

Users can choose standard APIs to scale their data: In most cases users looking for data scalability have had to rewrite their applications. This is still the case with most of the NoSQL and in-memory data grid solutions. Through standard JPA support in our platform, we can finally reduce that lock-in barrier.

Better Application Monitoring, Specifically Geared for PaaS

One of the challenges with many of the existing monitoring systems is that they were not geared for PaaS-based deployment. In this new offering we provde PaaS-driven monitoring that is tightly integrated with the platform, and which can interact with the platform to deal with failure or scaling events without any human intervention. In addition, it provides a holistic view that includes the application and infrastructure, as can be seen in these screenshots:

Furthermore, the new monitoring was designed to integrate with the existing set of data-center monitoring tools.

Better Performance and Scalability

By leveraging the Citrix OpenCloud solution, GigaSpaces will provide tighter integration between the platform and the infrastructure layers to up-level services offered through OpenStack clouds. In this way, we leverage the years of experience of both platforms in delivering high performance and low latency to mission-critical applications and remove the hassle involved in tuning the entire stack. We believe that through this joint work we can bring a lot of that experience into the OpenStack community.

We can also now offer more fine-grained multi-tenancy support –- by combining process and VM-based multi-tenancy. This makes it possible to achieve significantly higher density and utilization of existing resources, and reduces the amount of resources -- and therefore the cost -- associated with serving a particular application load.

What does this means for Enterprises, ISV/SaaS, and Cloud/IaaS providers?

Any user of this new offering will benefit from the greater openess and reduced lock-in concern. They wouldn’t be relying on single vendor for their future. This also means that they will have better control over their own stack and cost margins, and will therefore gain the ability to offer more competitive prices with other equivalent cloud providers. More specifically

Enterprises can use this offering to build their own Enterprise PaaS layer that is specifically geared for big-data analytics, e-commerce, and financial applications.

SaaS ISVs can use this offering to SaaS-enable their application and provide the same solution off- and on-premises.

Infrastructure/Cloud providers can use this offering to offer an Amazon-like services stack, including RDS, SimpleDB, SQS, Cloudwatch, and Elastic Beanstalk as a pre-packaged solution.

Final Words

I’m very excited about all of these new developments. OpenStack fills in the missing piece in the cloud –- as its name suggests, it unlocks the cloud by providing a truly open cloud stack and provides an essential substrate that drives innovation and collaboration -- which couldn’t be done before. It's interesting to compare the level of effort that we had to invest in the past when we did the first integration with Citrix to the one with OpenStack. Openstack enables us to work completely in parallel and make progress without too much coordination and still achieve substantial progress. Moreover, every delivery of our work can be relevant to wider spectrum of users. It also enabled us to join forces with other members of the community, such as Adrian Cole from JClouds and GridDynamics, simply because we share a similar goal to support the OpenStack open cloud mission.

This is only the beginning. I hope that with this new development we can continue to work toward greater openess at the application platform layer together with Citrix and the rest of the OpenStack community.

I’m going to give a talk on on the subject (PaaS on Openstack) during the upcoming OpenStack Design Summit on April 26th, 2011, in Santa Clara, California, where I hope to share some of these thoughts and hopefully get more work on this happening within the community.

Availability

I will also be announcing the first GigaSpaces code contribution to OpenStack at the Design Summit. Note that the integration with OpenStack will be made available to the entire Java community through the open source JClouds project.

You can also get an early look at our upcoming 2nd-generation CEAP release by registering for the Early Access Program.

Comments

One of the major concerns of many IT organizations is cloud vendor lock-in. This concern was expressed recently in "Banks fear cloud vendor lock-in," from IT Wire:

The onset of cloud computing gives vendors the chance to lock customers in to their infrastructure, using proprietary protocols to ensure they’re on the monthly billing cycle as long as possible.

The OpenStack project emerged with a mission to address this concern by creating a community-led open source project enabling any organization to create and offer cloud computing services running on standard hardware.

GigaSpaces joined the OpenStack project with the mission to enable any organization to build its own Platform-as-a-Service (PaaS), with its own choice of language and best-of-breed middleware stack.

In this post, I’ll try to provide more insight into our current and future plans around OpenStack, and more specifically our joint collaboration with the Citrix OpenCloud initiative.

GigaSpaces OpenStack Explained

What does GigaSpaces' OpenStack support mean?

One of the goals for our second-generation PaaS/SaaS enablement platform was to enable smooth migration between different cloud providers. We were able to achieve this goal through the use of our own abstraction (the Scaling Handler) and through the integration with the JClouds project that provides common abstraction to most of the existing cloud providers. With that, we can ensure that any application can be moved from the likes of Amazon to OpenStack or to an organization's own private cloud with zero changes to the application code or configuration.

The only change involves is setting the user/key of the specific cloud.

By adding support for OpenStack, we now enable users to safely move to an OpenStack-based cloud when they're ready and with little effort, yet they gain all the benefits that comes with it in terms of cost, openness etc.

Can I use the OpenStack integration outside of GigaSpaces' platform?

Yes. The OpenStack integration was developed in close collaboration with GridDyanmics and Adrian Cole from JClouds. We intend to make the integration with OpenStack available to the entire Java community through the open source JClouds project. In this way, any Java application can easily run on OpenStack.

What specific application stack is currently supported?

Today, there are a wide variety of tools and frameworks available throughout the ecosystems in Java, .Net, Ruby, and more. We feel that limiting the platform to a specific stack that we control -- as in the case of Google App Engine -- is too restrictive. Instead, we wanted to create a foundation that enables users of the platform to easily host any service or framework they choose on the platform. We also wanted to do this in a way that provides consistent behavior in terms of deployment, monitoring, elasticity, and scaling. Therefore, we developed a Universal Service Manager and Service Orchestration framework. The framework makes it easier users to plug in their own choice of services that comprise the application stack (Tomcat, MySQL, NoSQL through Cassandra, Hadoop, Ruby...).

In addition, we will provide a set of built-in recipes. At first we will be targeting Big Data stacks, which include integration with NoSQL and Hadoop, and also reporting tools to make the deployment of Big Data applications significantly simpler.

The integration with OpenStack will make it possible to bring these benefits to any application running on OpenStack.

How is this related to the previously announced Open Elastic Platform with Citrix?

Openness at the IaaS layer -– The specific integration with NetScaler (load balancer) and XenServer would be done through more open interfaces provided through the Citrix/Openstack contribution, which means that OpenStack users can use those intefaces to plug in any hypervisor or load balancer.

Openness at the application layer –- The previous version of GigaSpaces XAP offered a limited set of application stack support features, mostly geared to services that we control and mostly in Java. With the new platform we offer significantly greater flexibility on various fronts:

Open to any application stack and language: As I mentioned earlier, users can now easily host any application or service and build their stack of choice, while at the same time managing and controling their application in a consistent way in terms of the deployment, monitoring, elasticity, etc.

Users can choose their container of choice: The new platform will include support for Tomcat in addition to our existing Jetty support.

Users can choose standard APIs to scale their data: In most cases users looking for data scalability have had to rewrite their applications. This is still the case with most of the NoSQL and in-memory data grid solutions. Through standard JPA support in our platform, we can finally reduce that lock-in barrier.

Better Application Monitoring, Specifically Geared for PaaS

One of the challenges with many of the existing monitoring systems is that they were not geared for PaaS-based deployment. In this new offering we provde PaaS-driven monitoring that is tightly integrated with the platform, and which can interact with the platform to deal with failure or scaling events without any human intervention. In addition, it provides a holistic view that includes the application and infrastructure, as can be seen in these screenshots:

Furthermore, the new monitoring was designed to integrate with the existing set of data-center monitoring tools.

Better Performance and Scalability

By leveraging the Citrix OpenCloud solution, GigaSpaces will provide tighter integration between the platform and the infrastructure layers to up-level services offered through OpenStack clouds. In this way, we leverage the years of experience of both platforms in delivering high performance and low latency to mission-critical applications and remove the hassle involved in tuning the entire stack. We believe that through this joint work we can bring a lot of that experience into the OpenStack community.

We can also now offer more fine-grained multi-tenancy support –- by combining process and VM-based multi-tenancy. This makes it possible to achieve significantly higher density and utilization of existing resources, and reduces the amount of resources -- and therefore the cost -- associated with serving a particular application load.

What does this means for Enterprises, ISV/SaaS, and Cloud/IaaS providers?

Any user of this new offering will benefit from the greater openess and reduced lock-in concern. They wouldn’t be relying on single vendor for their future. This also means that they will have better control over their own stack and cost margins, and will therefore gain the ability to offer more competitive prices with other equivalent cloud providers. More specifically

Enterprises can use this offering to build their own Enterprise PaaS layer that is specifically geared for big-data analytics, e-commerce, and financial applications.

SaaS ISVs can use this offering to SaaS-enable their application and provide the same solution off- and on-premises.

Infrastructure/Cloud providers can use this offering to offer an Amazon-like services stack, including RDS, SimpleDB, SQS, Cloudwatch, and Elastic Beanstalk as a pre-packaged solution.

Final Words

I’m very excited about all of these new developments. OpenStack fills in the missing piece in the cloud –- as its name suggests, it unlocks the cloud by providing a truly open cloud stack and provides an essential substrate that drives innovation and collaboration -- which couldn’t be done before. It's interesting to compare the level of effort that we had to invest in the past when we did the first integration with Citrix to the one with OpenStack. Openstack enables us to work completely in parallel and make progress without too much coordination and still achieve substantial progress. Moreover, every delivery of our work can be relevant to wider spectrum of users. It also enabled us to join forces with other members of the community, such as Adrian Cole from JClouds and GridDynamics, simply because we share a similar goal to support the OpenStack open cloud mission.

This is only the beginning. I hope that with this new development we can continue to work toward greater openess at the application platform layer together with Citrix and the rest of the OpenStack community.

I’m going to give a talk on on the subject (PaaS on Openstack) during the upcoming OpenStack Design Summit on April 26th, 2011, in Santa Clara, California, where I hope to share some of these thoughts and hopefully get more work on this happening within the community.

Availability

I will also be announcing the first GigaSpaces code contribution to OpenStack at the Design Summit. Note that the integration with OpenStack will be made available to the entire Java community through the open source JClouds project.

You can also get an early look at our upcoming 2nd-generation CEAP release by registering for the Early Access Program.