How-to Guides¶
Scope
Task recipes for CockroachDB v26.x self-hosted clusters: install, start and secure a cluster, apply a license, plan the topology, configure multi-region, tune, back up and restore, stream changes, use vector search, upgrade, and troubleshoot. Commands were checked against the v26.2 docs. For defaults and version matrices, see Reference. For the internals behind the recipes, see Explanation.
License key required for multi-node clusters
Since v24.3 (2024-11-18), a multi-node cluster without a license key is throttled to 5 concurrent transactions 7 days after initialization. cockroach start-single-node and cockroach demo need no key. See Apply a License.
Install CockroachDB¶
# Linux x86-64 binary (replace the version with the latest patch from the Reference page)
curl https://binaries.cockroachdb.com/cockroach-v26.2.6.linux-amd64.tgz | tar -xz
sudo cp -i cockroach-v26.2.6.linux-amd64/cockroach /usr/local/bin/
cockroach version
# Docker: single-node dev instance (no license key needed)
docker run -d --name=crdb -p 26257:26257 -p 8080:8080 \
cockroachdb/cockroach:v26.2.6 start-single-node --insecure
# Throwaway in-memory cluster with sample data (MovR)
cockroach demo
Pick a Regular release for production
v26.2 is the current Regular release. v26.3 is an Innovation release with 6 months of support. See Reference: Release and Support Matrix.
Start a Local Cluster¶
A three-node local cluster in insecure mode, for learning only. The commands follow the official local cluster tutorial.
cockroach start --insecure --store=node1 --listen-addr=localhost:26257 --http-addr=localhost:8080 --join=localhost:26257,localhost:26258,localhost:26259 &
cockroach start --insecure --store=node2 --listen-addr=localhost:26258 --http-addr=localhost:8081 --join=localhost:26257,localhost:26258,localhost:26259 &
cockroach start --insecure --store=node3 --listen-addr=localhost:26259 --http-addr=localhost:8082 --join=localhost:26257,localhost:26258,localhost:26259 &
# One-time cluster initialization
cockroach init --insecure --host=localhost:26257
# SQL shell (PostgreSQL wire protocol)
cockroach sql --insecure --host=localhost:26257
# or: psql "postgresql://root@localhost:26257/defaultdb?sslmode=disable"
The DB Console is at http://localhost:8080. Stop nodes with cockroach node drain --insecure --host=localhost:26258 --shutdown or Ctrl+C.
Secure a Cluster¶
Create Certificates¶
mkdir certs my-safe-directory
cockroach cert create-ca --certs-dir=certs --ca-key=my-safe-directory/ca.key
# One node certificate per node, listing every hostname/IP clients use to reach it
cockroach cert create-node node1.example.com localhost 127.0.0.1 \
--certs-dir=certs --ca-key=my-safe-directory/ca.key
# Client certificates (root, and one per application user)
cockroach cert create-client root --certs-dir=certs --ca-key=my-safe-directory/ca.key
cockroach cert create-client app_user --certs-dir=certs --ca-key=my-safe-directory/ca.key
# Start nodes without --insecure
cockroach start --certs-dir=certs --store=/mnt/crdb1 \
--advertise-addr=node1.example.com --join=node1.example.com,node2.example.com,node3.example.com
cockroach init --certs-dir=certs --host=node1.example.com
cockroach sql --certs-dir=certs --user=app_user --host=node1.example.com:26257
Create Users and Roles¶
CREATE ROLE read_only;
GRANT CONNECT ON DATABASE appdb TO read_only;
GRANT SELECT ON ALL TABLES IN SCHEMA appdb.public TO read_only;
CREATE USER analyst WITH PASSWORD 'change-me'; -- stored as SCRAM-SHA-256
GRANT read_only TO analyst;
GRANT SELECT, INSERT ON TABLE orders TO app_writer;
REVOKE DELETE ON TABLE orders FROM app_writer;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO read_only;
Restrict Connections (Host-Based Authentication)¶
-- pg_hba.conf syntax. The first matching rule wins.
SET CLUSTER SETTING server.host_based_authentication.configuration = '
host all root 0.0.0.0/0 cert
host all all 10.0.0.0/8 cert-password
host all all 0.0.0.0/0 reject
';
-- Kerberos/GSSAPI for all non-root users (node needs KRB5_KTNAME pointing to a keytab)
SET CLUSTER SETTING server.host_based_authentication.configuration = 'host all all all gss include_realm=0';
Enable Encryption at Rest¶
# Generate a 256-bit store key (keep it outside the store directory)
cockroach gen encryption-key -s 256 /secure/keys/aes-256.key
# First start with encryption (old-key=plain means data was not encrypted before)
cockroach start --certs-dir=certs --store=/mnt/crdb1 \
--enterprise-encryption=path=/mnt/crdb1,key=/secure/keys/aes-256.key,old-key=plain \
--join=node1.example.com,node2.example.com,node3.example.com
# Rotate the store key later: restart with the new key and the previous key as old-key
# --enterprise-encryption=path=/mnt/crdb1,key=/secure/keys/aes-256-v2.key,old-key=/secure/keys/aes-256.key
Once set for a store, the --enterprise-encryption flag must be present on every restart (Encryption at rest).
Enable Audit Logging¶
-- Role-based: audit every statement by members of the 'admin' and 'ops' roles
SET CLUSTER SETTING sql.log.user_audit = '
admin ALL
ops ALL
';
-- Table-based: log every read and write on a sensitive table
ALTER TABLE customers EXPERIMENTAL_AUDIT SET READ WRITE;
-- Recent system events (DDL, grants, node changes)
SELECT timestamp, "eventType", "reportingID" FROM system.eventlog ORDER BY timestamp DESC LIMIT 50;
Apply a License¶
- In the CockroachDB Cloud Console, go to Organization > Enterprise Licenses > Create License. Choose Enterprise Trial (30 days), or toggle the Enterprise Free qualification (organizations under $10M in annual revenue).
- Apply the key as
root:
- Keep telemetry enabled on Free or Trial clusters, and alert on the
seconds_until_enterprise_license_expiryPrometheus metric. An expired Free license throttles the cluster after 30 days, and a Trial after 7 days.
Deployment Patterns¶
Cluster Topology¶
| Pattern | Min Nodes | Survival Goal | Use Case |
|---|---|---|---|
| Single-region, multi-zone | 3 (one per zone) | Zone failure | Standard HA |
| Multi-region, zone survival | 3 per region | Zone failure. Writes stay in-region | Low-latency regional apps |
| Multi-region, region survival | 3 regions x 3 nodes | Region failure | Global distribution, strict availability |
| Global tables | Any multi-region layout | Same as the database | Rarely written reference data read everywhere |
On Kubernetes, use the official Helm chart (helm repo add cockroachdb https://charts.cockroachdb.com/, then helm install crdb cockroachdb/cockroachdb) or the CockroachDB operator announced with v25.3. Match the image version to the operator's supported list. See Kubernetes.
Configure Multi-Region (Geo-Partitioning)¶
Node localities must be set at start: cockroach start --locality=region=us-east1,zone=us-east1-b ....
ALTER DATABASE mydb SET PRIMARY REGION "us-east1";
ALTER DATABASE mydb ADD REGION "europe-west1";
ALTER DATABASE mydb ADD REGION "asia-southeast1";
-- Survive a full region outage (5 voters per range, writes cross regions)
ALTER DATABASE mydb SURVIVE REGION FAILURE;
-- Home each row in its region (data residency, for example GDPR)
ALTER TABLE users SET LOCALITY REGIONAL BY ROW; -- adds hidden crdb_region column
-- Fast consistent reads everywhere, slower writes
ALTER TABLE currencies SET LOCALITY GLOBAL;
-- v26.1+: drop read replicas by making every replica a voter
-- (num_replicas = num_voters). Check the survival goal first.
Performance Tuning¶
| Knob | Where | Default | When to change |
|---|---|---|---|
range_max_bytes |
Zone config | 512 MiB | Lower (for example 128 MiB) only for very hot small tables. Prefer load-based splitting |
gc.ttlseconds |
Zone config | 14400 (4 h) | Raise if you need longer AS OF SYSTEM TIME or incremental-backup windows |
kv.snapshot_rebalance.max_rate |
Cluster setting | 32 MiB/s | Raise (for example 64 MiB/s) for faster rebalancing on fast disks and networks |
server.time_until_store_dead |
Cluster setting | 5m | Raise during planned maintenance to avoid needless re-replication |
sql.defaults.distsql |
Cluster setting | auto | Rarely. Per session: SET distsql = on |
-- Range size for one table
ALTER TABLE orders CONFIGURE ZONE USING range_max_bytes = 134217728;
-- Range distribution and leaseholders (crdb_internal.ranges no longer exposes table_name since v23.1)
SHOW RANGES FROM TABLE orders WITH DETAILS;
-- Hot statements
SELECT key AS fingerprint, count, service_lat_avg
FROM crdb_internal.node_statement_statistics ORDER BY count DESC LIMIT 10;
-- Opt a contended transaction into READ COMMITTED (GA since v24.1)
BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;
Use EXPLAIN ANALYZE to find full scans, and add covering indexes with STORING. Avoid sequential primary keys: use UUID DEFAULT gen_random_uuid() or hash-sharded indexes (USING HASH) to spread writes.
Backup & Recovery¶
-- Full cluster backup to cloud storage
BACKUP INTO 's3://bucket/backup?AUTH=implicit' AS OF SYSTEM TIME '-10s';
-- Incremental backup onto the latest full backup
BACKUP INTO LATEST IN 's3://bucket/backup?AUTH=implicit';
-- Scheduled backups: daily full, hourly incrementals
CREATE SCHEDULE daily_backup FOR BACKUP INTO 's3://bucket/backup?AUTH=implicit'
RECURRING '@hourly' FULL BACKUP '@daily' WITH SCHEDULE OPTIONS first_run = 'now';
-- Inspect and restore
SHOW BACKUPS IN 's3://bucket/backup?AUTH=implicit';
RESTORE FROM LATEST IN 's3://bucket/backup?AUTH=implicit'; -- full cluster
RESTORE DATABASE mydb FROM LATEST IN 's3://bucket/backup?AUTH=implicit'; -- one database
v26.2 previews two restore improvements: RESTORE ... WITH EXPERIMENTAL COPY (up to 4x faster) and restoring by backup ID from SHOW BACKUPS.
Stream Changes (CDC)¶
SET CLUSTER SETTING kv.rangefeed.enabled = true; -- required on self-hosted
CREATE CHANGEFEED FOR TABLE orders
INTO 'kafka://broker:9092'
WITH format = json, updated, resolved = '10s';
SHOW CHANGEFEED JOBS;
For Kafka sink design, see Kafka. v25.2 added a Debezium-style "enriched" envelope in preview.
Use Vector Search¶
CREATE TABLE docs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id INT NOT NULL,
body STRING,
embedding VECTOR(1536),
VECTOR INDEX (tenant_id, embedding vector_cosine_ops) -- prefix column + opclass
);
-- k-NN search inside one tenant
SELECT id, body FROM docs
WHERE tenant_id = 42
ORDER BY embedding <=> '[0.01, 0.02, ...]'
LIMIT 10;
-- Trade latency for recall
SET vector_search_beam_size = 64;
To add an index to an existing table, run SET sql_safe_updates = false; and then CREATE VECTOR INDEX ON docs (tenant_id, embedding vector_cosine_ops);. Writes to the table are blocked during the backfill.
Upgrade a Cluster¶
- Check that the target is allowed: Regular releases cannot be skipped. Innovation releases can (for example v25.4 -> v26.2 is valid, skipping v26.1).
- Optionally disable auto-finalization so you can roll back:
SET CLUSTER SETTING cluster.auto_upgrade.enabled = false;. - Roll nodes one at a time: drain (
cockroach node drain --certs-dir=certs --host=<node>), stop, replace the binary, restart, and wait for zero under-replicated ranges. - Finalize:
SET CLUSTER SETTING version = '26.2';(or re-enable auto-upgrade). Confirm withSHOW CLUSTER SETTING version;.
Until finalization completes, the cluster can be rolled back to the previous binary. Afterwards it cannot (Upgrade docs).
Common Issues¶
| Issue | Diagnosis | Fix |
|---|---|---|
| Range under-replicated | DB Console > Replication dashboard, SHOW RANGES ... WITH DETAILS |
Add nodes, free disk space, check zone constraints are satisfiable |
| High query latency | EXPLAIN ANALYZE, Insights page, statement statistics |
Add indexes, fix full scans, check leaseholder locality |
Transaction retry errors (40001) |
Contention in Insights, crdb_internal.transaction_contention_events |
Retry loop in the client, smaller transactions, SELECT ... FOR UPDATE, or READ COMMITTED |
| Clock skew / node crash with offset error | cockroach node status, logs |
Run chrony/NTP. Offsets must stay below --max-offset (500 ms default) |
| Cluster throttled to 5 transactions | License notices in SQL clients, DB Console banner | Apply or renew the license key, restore telemetry egress |
| Node decommission stuck | cockroach node status --decommission |
Re-issue cockroach node decommission, check that constraints leave somewhere to move replicas |
| Disk stall crash | storage.max_sync_duration exceeded (20 s) |
Investigate the storage volume. The crash is deliberate protection |
Commands & Recipes¶
Cluster Setup¶
# Single node (dev, no license key needed)
cockroach start-single-node --insecure --store=node1 --listen-addr=localhost:26257 --http-addr=localhost:8080
# Initialize a multi-node cluster once
cockroach init --insecure --host=localhost:26257
# SQL shell
cockroach sql --insecure --host=localhost:26257 -e "SHOW DATABASES;"
Geo-Partitioning¶
SHOW REGIONS FROM DATABASE mydb;
SHOW SURVIVAL GOAL FROM DATABASE mydb;
SELECT crdb_region, count(*) FROM users GROUP BY 1;
Operations¶
# Cluster status, including decommission progress
cockroach node status --certs-dir=certs --host=node1.example.com
cockroach node status --decommission --certs-dir=certs --host=node1.example.com
# Decommission a node safely (node ID from 'node status')
cockroach node decommission 4 --certs-dir=certs --host=node1.example.com
# Collect a debug bundle for support
cockroach debug zip ./debug.zip --certs-dir=certs --host=node1.example.com
# Backup and restore from the shell
cockroach sql --certs-dir=certs --host=node1.example.com -e "BACKUP DATABASE mydb INTO 's3://bucket/backup?AUTH=implicit';"
cockroach sql --certs-dir=certs --host=node1.example.com -e "RESTORE DATABASE mydb FROM LATEST IN 's3://bucket/backup?AUTH=implicit';"
# Load-test with the built-in workload generator
cockroach workload init tpcc --warehouses=10 'postgresql://root@localhost:26257?sslmode=disable'
cockroach workload run tpcc --warehouses=10 --duration=5m 'postgresql://root@localhost:26257?sslmode=disable'