Rook is an orchestrator for storage services that run in a Kubernetes cluster. In the Rook v0.8 release, we are excited to say that the orchestration around Ceph has stabilized to the point to be declared Beta. If you haven’t yet started a Ceph cluster with Rook, now is the time to take it for a …Read more

Choosing the right number of PGs (“placement groups”) for your cluster is a bit of black art–and a usability nightmare. Getting a reasonable value can have a big impact on a cluster’s performance and reliability, both in a good way and a bad way. Unfortunately, over the past few years we’ve seen our share of …Read more

The Ceph manager service (ceph-mgr) was introduced in the Kraken release, and in Luminous it has been extended with a number of new python modules. One of these is a module exporting overall cluster status and performance to Zabbix. Enabling the dashboard module The Zabbix module is included in the ceph-mgr package, so if you’ve …Read more

RADOS is the reliable autonomic distributed object store that underpins Ceph, providing a reliable, highly available, and scalable storage service to other components. As with every Ceph release, Luminous includes a range of improvements to the RADOS core code (mostly in the OSD and monitor) that benefit all object, block, and file users. Parallel monitor …Read more

The Ceph file system uses a cluster of metadata servers to provide an authoritative cache for the CephFS metadata stored in RADOS. The most basic reason for this is to maintain a hot set of metadata in memory without talking to the metadata pool in RADOS. Another important reason is to allow clients to also …Read more

The Ceph file system (CephFS) allows for portions of the file system tree to be carved up into subtrees which can be managed authoritatively by multiple MDS ranks. This empowers the cluster to scale performance with the size and usage of the file system by simply adding more MDS servers into the cluster. Where possible, …Read more

The flexibility of the CRUSH map in controlling data placement in Ceph is one of the system’s great strengths. It is also one of the most painful and awkward parts of the cluster to manage. Previously, any non-trivial data placement policy required manual editing of the CRUSH map, either to adjust the hierarchy or to …Read more

CERN has been a long-time Ceph user and active community member, running one of the largest production OpenStack clouds backed by Ceph. They are also using Ceph for other storage use-cases, backing a range of high energy physics experiments. Overall scalability is understandably an area of interest. Big Bang I and II In 2015, CERN …Read more

The Ceph file system (CephFS) is the file storage solution for Ceph. Since the Jewel release it has been deemed stable in configurations using a single active metadata server (with one or more standbys for redundancy). Now in Luminous, multiple active metadata servers configurations are stable and ready for deployment! This allows the CephFS metadata …Read more

Congratulations on upgrading your cluster to the Luminous release of Ceph. You have diligently followed all the steps outlined in the upgrade notes to the point where you are ready to mark the upgrade complete by executing the following: $ ceph osd require-osd-release luminous Once your cluster has been flagged as fully upgraded to the …Read more

RGW metadata search is a new feature that was added in Ceph Luminous. It enables integration with Elasticsearch to provide a search API to query an object store based on object metadata. A new zone type A zone in the RGW multisite system is a set of radosgw daemons serving the same data, backed by …Read more

When upgrading any distributed system, it’s easy to miss a step and have an old daemon running long after you thought the previous version was history. A common mistake is installing new packages and forgetting to restart the processes using them. With distributed storage like Ceph, it’s also important to remember the client side: the VMs, containers, file …Read more