Automatic Rebalancing

This page walks you through a simple demonstration of how CockroachDB automatically rebalances data as you scale. Starting with a 3-node local cluster, you'll lower the maximum size for a single range, the unit of data that is replicated in CockroachDB. You'll then download and run the block_writer example program, which continuously inserts data into your cluster, and watch the replica count quickly increase as ranges split. You'll then add 2 more nodes and watch how CockroachDB automatically rebalances replicas to efficiently use all available capacity.

Before You Begin

In this tutorial, you'll use an example Go program to quickly insert data into a CockroachDB cluster. To run the example program, you must have a Go environment with a 64-bit version of Go 1.7.1.

Step 4. Lower the max range size

In CockroachDB, you use replication zones to control the number and location of replicas. Initially, there is a single default replication zone for the entire cluster that is set to copy each range of data 3 times. This default replication factor is fine for this demo.

However, the default replication zone also defines the size at which a single range of data spits into two ranges. Since you want to create many ranges quickly and then see how CockroachDB automatically rebalances them, reduce the max range size from the default 67108864 bytes (64MB) to cause ranges to split more quickly:

Step 5. Download and run the block_writer program

CockroachDB provides a number of example programs in Go for simulating client workloads. The program you'll use for this demonstration is called block_writer. It will simulate multiple clients inserting data into the cluster.

Download and install the program:

copy

icon/buttons/copy

$ go get github.com/cockroachdb/examples-go/block_writer

Then run the program for 1 minute, long enough to generate plenty of ranges:

copy

icon/buttons/copy

$ block_writer -duration 1m

Once it's running, block_writer will output the number of rows written per second:

Step 8. Watch data rebalance across all 5 nodes

Back in the Admin UI, you'll now see 5 nodes listed. At first, the bytes and replica count will be lower for nodes 4 and 5. Very soon, however, you'll see those metrics even out across all nodes, indicating that data has been automatically rebalanced to utilize the additional capacity of the new nodes.

Step 9. Stop the cluster

Once you're done with your test cluster, stop each node by switching to its terminal and pressing CTRL-C.

Tip:

For the last node, the shutdown process will take longer (about a minute) and will eventually force kill the node. This is because, with only 1 node still online, a majority of replicas are no longer available (2 of 3), and so the cluster is not operational. To speed up the process, press CTRL-C a second time.

If you don't plan to restart the cluster, you may want to remove the nodes' data stores: