Running multiple nodes on a single host is useful for testing out CockroachDB, but it's not recommended for production deployments. To run a physically distributed cluster in production, see Manual Deployment, Cloud Deployment, or Orchestration.

The second command creates the Certificate Authority (CA) certificate and key: ca.crt and ca.key.

The third command creates the client certificate and key, in this case for the root user: client.root.crt and client.root.key. These files will be used to secure communication between the built-in SQL shell and the cluster (see step 4).

The fourth command creates the node certificate and key: node.crt and node.key. These files will be used to secure communication between nodes. Typically, you would generate these separately for each node since each node has unique addresses; in this case, however, since all nodes will be running locally, you need to generate only one node certificate and key.

This command starts a node in secure mode, accepting most cockroach start defaults.

The --certs-dir directory points to the directory holding certificates and keys.

Since this is a purely local cluster, --host=localhost tells the node to listens only on localhost, with default ports used for internal and client traffic (26257) and for HTTP requests from the Admin UI (8080).

The Admin UI defaults to listening on all interfaces. The --http-host flag is therefore used to restrict Admin UI access to the specified interface, in this case, localhost.

Node data is stored in the cockroach-data directory.

The standard output gives you helpful details such as the CockroachDB version, the URL for the admin UI, and the SQL URL for clients.

Tip:

By default, each node's cache is limited to 25% of available memory. This default is reasonable when running one node per host. When you run multiple nodes on a single host, however, this default may lead to out-of-memory errors, especially if you test in a serious way. To avoid such errors, you can limit each node's cache size by setting the --cache flag in the start command.

Step 3. Add nodes to the cluster

At this point, your cluster is live and operational. With just one node, you can already connect a SQL client and start building out your database. In real deployments, however, you'll always want 3 or more nodes to take advantage of CockroachDB's automatic replication, rebalancing, and fault tolerance capabilities. This step helps you simulate a real deployment locally.

The main difference in these commands is that you use the --join flag to connect the new nodes to the cluster, specifying the address and port of the first node, in this case localhost:26257. Since you're running all nodes on the same machine, you also set the --store, --port, and --http-port flags to locations and ports not used by other nodes, but in a real deployment, with each node on a different machine, the defaults would suffice.

Step 4. Test the cluster

Now that you've scaled to 3 nodes, you can use any node as a SQL gateway to the cluster. To demonstrate this, open a new terminal and connect the built-in SQL client to node 1:

Note:

The SQL client is built into the cockroach binary, so nothing extra is needed.

$ cockroach sql \
--certs-dir=certs
# Welcome to the cockroach SQL interface.# All statements must be terminated by a semicolon.# To exit: CTRL + D.

As you can see, node 1 and node 2 behaved identically as SQL gateways.

Exit the SQL shell on node 2:

>\q

Step 5. Monitor the cluster

To access the Admin UI for your cluster, point a browser to https://localhost:8080, or to the address in the admin field in the standard output of any node on startup.

Note that your browser will consider the CockroachDB-created certificate invalid; you’ll need to click through a warning message to get to the UI.

As mentioned earlier, CockroachDB automatically replicates your data behind-the-scenes. To verify that data written in the previous step was replicated successfully, scroll down to the Replicas per Node graph and hover over the line:

The replica count on each node is identical, indicating that all data in the cluster was replicated 3 times (the default).

Step 6. Stop the cluster

Once you're done with your test cluster, switch to the terminal running the first node and press CTRL-C to stop the node.

At this point, with 2 nodes still online, the cluster remains operational because a majority of replicas are available. To verify that the cluster has tolerated this "failure", connect the built-in SQL shell to nodes 2 or 3. You can do this in the same terminal or in a new terminal.

Now stop nodes 2 and 3 by switching to their terminals and pressing CTRL-C.

Tip:

For node 3, the shutdown process will take longer (about a minute) and will eventually force kill the node. This is because, with only 1 of 3 nodes left, a majority of replicas are not available, and so the cluster is no longer operational. To speed up the process, press CTRL-C a second time.

If you don't plan to restart the cluster, you may want to remove the nodes' data stores:

$ rm -rf cockroach-data node2 node3

Step 7. Restart the cluster

If you decide to use the cluster for further testing, you'll need to restart at least 2 of your 3 nodes from the directories containing the nodes' data stores.