Ceph provides unified scale-out storage, using commodity x86 hardware that is self-healing and intelligently anticipates failures. It has become the de facto standard for software-defined storage. Because Ceph is open source, it has enabled many vendors the ability to provide Ceph based software-defined storage systems. Ceph is not just limited to companies like Red Hat, Suse, Mirantis, Ubuntu, etc. — SanDisk, Fujitsu, HP, Dell, Samsung and many more now offer integrated solutions, too. There are even large-scale community built environments — CERN comes to mind — that provide storage services for 10,000s of VMs.

Ceph is by no means limited to OpenStack, however this is where Ceph started gaining traction. Looking at latest OpenStack user survey, Ceph is the clear leader for OpenStack storage. Page 42 in OpenStack April 2016 User Survey reveals Ceph is 57 percent of OpenStack storage. The next is LVM (local storage) with 28 percent followed by NetApp with 9 percent. If we remove LVM, Ceph leads any other storage company by 48 percent, that is incredible. Why is that?

There are several reasons, but here are my top three:

Ceph is a scale-out unified storage platform. OpenStack needs two things from storage: ability to scale with OpenStack itself and do so regardless of block (Cinder), File (Manila) or Object (Swift). Traditional storage vendors need to provide two or three different storage systems to achieve this. They don’t scale the same and in most cases only scale-up in never-ending migration cycles. Their management capabilities are never truly integrated across broad spectrum of storage use-cases.

Ceph is cost-effective. Ceph leverages Linux as an operating system instead of something proprietary. You can choose not only whom you purchase Ceph from, but also where you get your hardware. It can be same vendor or different. You can purchase commodity hardware, or even buy integrated solution of Ceph + Hardware from single vendor. There are even hyper-converged options for Ceph that are emerging (running Ceph services on compute nodes).

Ceph is OpenSource project just like OpenStack. This allows for a much tighter integration and cross-project development. Proprietary vendors are always playing catch-up since they have secrets to protect and their influence is usually limited in Opensource communities, especially in OpenStack context.

Here is an architecture diagram that shows all the different OpenStack components that need storage. It shows how they integrate with Ceph and how Ceph provides a unified storage system that scales to fill all these use cases.

If you are interested in more topics relating to Ceph and OpenStack, I recommend this.

Ok enough talking about why Ceph and OpenStack are so great, let’s get our hands dirty and see how to hook it up!

If you don’t have a Ceph environment you can follow this article on how to set one up quickly.

Glance Integration

Glance is the image service within OpenStack. By default images are stored locally on controllers and then copied to compute hosts when requested. The compute hosts cache the images but they need to be copied again, every time an image is updated.

Ceph provides back end for Glance allowing images to be stored in Ceph, instead of locally on controller and compute nodes. This greatly reduces network traffic for pulling images and increases performance since Ceph can clone images instead of copying them. In addition, it makes migrating between OpenStack deployments or concepts like multi-site OpenStack much simpler.

Cinder Integration

Cinder is the block storage service in OpenStack. Cinder provides an abstraction around block storage and allows vendors to integrate by providing a driver. In Ceph, each storage pool can be mapped to a different Cinder back end. This allows for creating storage services such as gold, silver or bronze. You can decide for example that gold should be fast SSD disks that are replicated three times, while silver only should be replicated two times and bronze should use slower disks with erasure coding.

Integrating Ceph with Nova Compute

Nova is the compute service within OpenStack. Nova stores virtual disks images associated with running VMs by default, locally on Hypervisor under /var/lib/nova/instances. There are a few drawbacks to using local storage on compute nodes for virtual disk images:

Summary

In this article, we discussed how OpenStack and Ceph fit perfectly together. We discussed some of the use cases around Glance, Cinder and Nova. Finally we went through steps to integrate Ceph with those use cases. I hope you enjoyed the article and found the information useful. Please share your feedback.

I have followed what you did for nova integration with openstack neuton, but am facing issues with no suitable host errors if I used rbd_user = nova, while use cinder user its working fine but no disk under vms pool.

User Resources

OpenStack Superuser is a publication built to chronicle the work of superusers, and their many accomplishments personally, professionally, and organizationally. The emphasis is on a blend of original journalism and user-generated content, ranging from technical to business-level issues with feature stories, case studies, tips and videos for OpenStack cloud architects and administrators.