Managed caching layer architecture

Couchbase Server is built on a memory-first architecture and managing memory effectively is central to achieving high performance and scalability.

Caching layer

Each service in Couchbase Server tunes its caching based on its needs.

Data service uses a managed cache that is tuned to enable fast key based read and write operations with low latency under high concurrency.

Indexing service manages the cache to ensure index maintenance and fast scans for the most popular indexes in the system.

Query service manages memory to calculate query responses by processing streams effectively for the execution plans generated by the optimizer.

The Couchbase SDK never access the persistence layer directly, but communicate through the caching layer.
Couchbase Server moves the data to and from the disk internally as needed, thereby acting as both a read-through and a write-through cache.
This facilitates extremely high read-write rates and eliminates the need for an external caching tier.
Unlike many other database systems, Couchbase Server does not depend on external caching systems.
This simplifies development as developers do not have to deal with complex cache coherency issues or varying performance capabilities across technologies.

Couchbase Server automatically manages the caching layer and coordinates with persistent storage to ensure that enough cache space exists to maintain performance.
It automatically places items that come in to the caching layer into a disk queue so that it can write these items to disk.
If the server determines that a cached item is infrequently used, it removes the item from RAM to free up space for other incoming operations.
Similarly, when infrequently-used items are requested, the server retrieves the items from disk and stores them in the caching layer.
In order to provide the most frequently-used data while maintaining high performance, Couchbase Server manages a working set of your entire dataset.
The working set is the data most frequently accessed and is kept in memory for high performance.

By default, the Couchbase Server automatically keeps frequently used data in memory and less frequently used data on disk.
Couchbase Server moves data from the managed cache to disk asynchronously, in the background, to ensure there is enough memory that can be freed up for incoming operations.
The server constantly monitors the information accessed by clients and decides how to keep the active data within the caching layer.
Items may be ejected from memory when additional memory is needed to perform incoming operations.
These items have already been persisted to disk and require no additional I/O.
The managed cache ensures that reads and writes are handled at a very fast rate, while removing the typical load and performance spikes that would otherwise cause a traditional RDBMS to produce erratic performance.