Explanation¶
Related Notes
Overview¶
HashiCorp Vault is an identity-based secrets and encryption management system. It centralizes secret storage, rotates credentials, generates dynamic secrets on demand, encrypts application data, and keeps detailed audit logs. Vault exposes every secret type through one HTTP API and enforces access through path-based ACL policies attached to tokens.
This page explains how Vault works and why it is designed that way. Look-up tables (auth methods, secrets engines, seal types, token types, sizing) are in Reference. Tasks and commands are in How-to Guides.
Component Architecture¶
The diagram shows the main components inside one Vault server and how they connect to clients, storage, and the seal.
graph TD
subgraph Clients["Clients"]
CLI["vault CLI"]
HTTP["HTTP API clients / SDKs"]
AGENT["Vault Agent / Vault Proxy / VSO"]
end
subgraph Server["Vault server"]
LISTENER["TCP listener :8200 (TLS)"]
CORE["Vault core / router"]
TOKEN["Token store"]
POLICY["Policy store (ACL, Sentinel)"]
EXPIRY["Expiration manager (leases)"]
ROLLBACK["Rollback manager"]
AUDIT["Audit broker"]
AUTH["Auth methods (plugins)"]
SECRET["Secrets engines (plugins)"]
AUDIT_DEV["Audit devices (file, syslog, socket)"]
BARRIER["Encryption barrier (AES-256-GCM keyring)"]
end
subgraph Storage["Storage backend (untrusted)"]
RAFT["Integrated Storage (Raft + BoltDB)"]
CONSUL["Consul"]
end
subgraph Seal["Seal"]
SHAMIR["Shamir unseal key shares"]
KMS["Auto-unseal: AWS KMS / GCP CKMS / Azure KV / OCI / Transit / PKCS#11"]
end
CLI --> LISTENER
HTTP --> LISTENER
AGENT --> LISTENER
LISTENER --> CORE
CORE --> TOKEN
CORE --> POLICY
CORE --> EXPIRY
CORE --> ROLLBACK
CORE --> AUDIT
AUDIT --> AUDIT_DEV
CORE --> AUTH
CORE --> SECRET
TOKEN --> BARRIER
AUTH --> BARRIER
SECRET --> BARRIER
EXPIRY --> BARRIER
BARRIER --> RAFT
BARRIER --> CONSUL
SHAMIR -->|"reconstructs unseal key"| BARRIER
KMS -->|"decrypts root key"| BARRIER
Key components:
- Router and core: map a request path such as
secret/data/appto the mount that owns it. Every auth method and secrets engine is a plugin mounted at a path. - Token store: every authenticated request carries a token. Auth methods only exchange an external identity for a token.
- Policy store: holds ACL policies (and Sentinel EGP/RGP policies in Enterprise).
- Expiration manager: tracks leases and revokes them when their TTL ends.
- Rollback manager: periodically asks mounts to clean up partial failures.
- Audit broker: sends each request and response to all enabled audit devices before Vault answers.
- Barrier: encrypts everything written to storage.
Encryption Barrier¶
The barrier is Vault's cryptographic boundary. Every piece of data that passes between the Vault core and the storage backend is encrypted with AES-256-GCM. Vault treats the storage backend as untrusted: an attacker with full access to storage sees only ciphertext.
Key properties:
- All data written to storage is encrypted before it leaves the barrier.
- All data read from storage is decrypted after it enters the barrier.
- The encryption keys (the keyring) exist in plaintext only in Vault's memory while unsealed.
- GCM authenticates the ciphertext, so tampering in storage is detected on read.
Seal and Unseal¶
When a Vault server starts, it is sealed. It knows where its storage is but cannot decrypt it. It can only answer seal-status requests and accept unseal input.
Key Hierarchy¶
Vault uses three layers of keys (Seal concepts). The data is encrypted with keys in the keyring. The keyring is encrypted with the root key (called the "master key" in older docs). The root key is encrypted with the unseal key, or by the auto-unseal KMS/HSM. Shamir shares reconstruct the unseal key, never the root key directly.
graph BT
SHARES["Unseal key shares<br/>(Shamir, K of N)"]
KMS["Auto-unseal KMS / HSM key"]
UK["Unseal key"]
RK["Root key<br/>(formerly master key)"]
KR["Keyring<br/>(AES-256-GCM encryption keys)"]
DATA["Encrypted data in storage"]
SHARES -->|"K shares reconstruct"| UK
UK -->|"decrypts"| RK
KMS -->|"decrypts (auto-unseal)"| RK
RK -->|"decrypts"| KR
KR -->|"encrypts / decrypts"| DATA
style RK fill:#f96,stroke:#333,stroke-width:2px
style KR fill:#6cf,stroke:#333,stroke-width:2px
style DATA fill:#ccc,stroke:#333
The layering explains two operations. vault operator rekey changes the unseal key (or the shares) without re-encrypting data. vault operator rotate adds a new key to the keyring for new writes, again without touching the unseal key.
Initialization and Unseal Flow¶
The sequence shows initialization, then the two unseal paths.
sequenceDiagram
participant Op as Operator
participant Vault as Vault server
participant Storage as Storage backend
participant KMS as Auto-unseal KMS
Op->>Vault: vault operator init
Vault->>Vault: Generate keyring, root key, unseal key
Vault->>Storage: Write keyring encrypted by root key
alt Shamir seal
Vault->>Vault: Split unseal key into N shares (default 5, threshold 3)
Vault-->>Op: Unseal key shares + initial root token
else Auto-unseal
Vault->>KMS: Encrypt root key
Vault-->>Op: Recovery key shares + initial root token
end
Note over Vault: Every restart begins SEALED
alt Shamir seal
loop Until threshold reached
Op->>Vault: vault operator unseal (one share)
end
Vault->>Vault: Reconstruct unseal key, decrypt root key
else Auto-unseal
Vault->>KMS: Decrypt root key
KMS-->>Vault: Plaintext root key
end
Vault->>Storage: Read encrypted keyring
Vault->>Vault: Decrypt keyring, load mounts, policies, audit devices
Note over Vault: UNSEALED, serving requests
With the Shamir seal, each node in a cluster must be unsealed separately with the threshold of shares. That is why production clusters almost always use auto-unseal.
Shamir's Secret Sharing¶
The default vault operator init splits the unseal key into 5 shares with a threshold of 3. This tolerates the loss of 2 shares while requiring 3 people to act together. Each share should go to a different trusted person or location, and -pgp-keys can encrypt each share to its holder.
Shamir limitations
Shamir's Secret Sharing is not verifiable secret sharing, so a malicious holder can submit an invalid share and block an unseal. Reconstruction happens in the Vault server's memory, so a compromised server can capture the reconstructed key.
Auto-Unseal and Recovery Keys¶
Auto-unseal delegates protection of the root key to a trusted KMS or HSM. At startup, Vault asks the KMS to decrypt the root key. No human is involved, so nodes can restart and rejoin on their own.
With auto-unseal, vault operator init returns recovery keys instead of unseal keys. Recovery keys authorize sensitive operations such as generating a new root token or rekeying, but they cannot decrypt the root key. If the KMS key is lost or deleted, the data cannot be recovered. Treat the KMS key as the most important asset in the deployment.
The Transit seal uses the Transit engine of another Vault cluster. It creates a hierarchy where a small, tightly guarded "root" Vault unseals the application Vault clusters. Enterprise adds PKCS#11 HSM seals, seal wrapping, and Seal HA (several seals configured at once for provider redundancy). The seal types table is in Reference.
Storage and High Availability¶
Vault does not implement durable persistence itself in the classic design: it writes encrypted blobs through a pluggable storage interface. Since Vault 1.4 the recommended backend is Integrated Storage, which embeds Raft inside Vault. Before 1.4, Consul was the recommended backend.
Integrated Storage (Raft)¶
Integrated Storage removes the separate Consul cluster and its extra network hop. Each Vault node stores a full copy of the data.
- Consensus: HashiCorp Raft replicates a log of writes. An entry is committed once a quorum (
ceil((N+1)/2)) has it. - State machine: BoltDB (
bbolt) holds the applied data invault.db; Raft logs are inraft.dbby default. An experimentalraft-wallog store is available as an alternative toraft-boltdbfor logs. - Autopilot (enabled by default): new nodes join as non-voters, catch up to the Raft index, and are promoted to voters only after a stability threshold. Autopilot also removes dead servers when configured.
- Snapshots:
vault operator raft snapshot savecaptures a point-in-time copy; Enterprise can run automated snapshots to cloud storage.
| Raft role | Behavior in Vault |
|---|---|
| Leader | The active Vault node. It appends all writes to the Raft log and replicates them. |
| Follower | A standby node. It stores replicated data and votes in elections. |
| Candidate | A follower that requests votes after missing leader heartbeats. |
| Non-voter | Receives replication but does not count toward quorum (autopilot staging, Enterprise redundancy zones). |
A cluster of 3 voters tolerates 1 failure. A cluster of 5 voters tolerates 2. When the leader fails, the remaining voters elect a new one if they still have quorum, and the new leader becomes the active Vault node. A failed node rejoins as a follower and catches up from the log or a snapshot.
Active and Standby Nodes¶
Vault HA is active/standby, not active/active. One node is active. In Vault Community Edition, standby nodes do not serve client requests: they forward them to the active node (or redirect the client to api_addr). Load balancers usually health-check /v1/sys/health, which returns different HTTP codes for active, standby, and sealed nodes.
Vault Enterprise adds performance standby nodes. They serve read-only requests locally and forward writes to the active node. This is the main way to scale reads inside one cluster.
Memory Protection and mlock¶
Vault historically calls mlock() so the operating system never swaps its memory (keys, tokens, decrypted secrets) to disk. On Linux the process needs the CAP_IPC_LOCK capability, which the docs grant to a non-root binary with setcap cap_ipc_lock=+ep.
With Integrated Storage the trade-off changes. BoltDB memory-maps its files, and mlock pins those mapped files in RAM. Vault's whole dataset then sits in resident memory and can cause out-of-memory failures when the data grows beyond RAM. HashiCorp therefore strongly recommends disable_mlock = true with Integrated Storage, combined with disabled or encrypted swap (Server configuration). Since Vault 1.20, disable_mlock has no default when Raft is used: the server refuses to start unless the value is set explicitly.
Containers since Vault 2.0.2
Official Vault container images no longer carry the IPC_LOCK capability (removed in 2.0.2, and in matching 1.21/1.20/1.19 Enterprise patches) so they run under common container runtimes. Containerized Vault must set disable_mlock = true, and the host or node should run without swap.
Auth Methods and Tokens¶
Auth methods verify an external identity and turn it into a Vault token with policies. After login, Vault checks only the token. The full list is in Reference.
The sequence shows a Kubernetes pod logging in and reading a dynamic database credential.
sequenceDiagram
participant Pod as App pod
participant K8sAuth as auth/kubernetes
participant TR as Kubernetes TokenReview API
participant Core as Vault core
participant DB as database/ engine
participant PG as PostgreSQL
Pod->>K8sAuth: POST auth/kubernetes/login (role, SA JWT)
K8sAuth->>TR: Review ServiceAccount JWT
TR-->>K8sAuth: Authenticated, SA name and namespace
K8sAuth->>Core: Match role, attach token_policies
Core-->>Pod: Vault token (TTL + policies)
Pod->>Core: GET database/creds/readonly (X-Vault-Token)
Core->>Core: ACL check on database/creds/readonly
Core->>DB: Forward request
DB->>PG: CREATE ROLE with VALID UNTIL
PG-->>DB: OK
DB-->>Pod: username, password, lease_id, lease_duration 1h
Note over DB,PG: Expiration manager revokes the role when the lease ends
Token design choices:
- Service tokens (
hvs.) are persisted and renewable. They form a parent/child tree, so revoking a parent revokes its children and their leases. - Batch tokens (
hvb.) are encrypted blobs that are not written to storage. They are cheap at high volume but cannot be renewed or used to create child tokens. - Periodic tokens can be renewed indefinitely inside their period, which suits long-running services.
- Root tokens have every capability and no TTL. Vault's design expects them to exist only briefly: create one, bootstrap, revoke it.
- Response wrapping puts a response in a single-use cubbyhole token. The receiver unwraps it once; if unwrapping fails, someone intercepted it. This addresses the "secret zero" delivery problem.
Since Vault 2.0 (Enterprise), the OAuth resource server and agent registry let registered AI agents and other OAuth clients send an OAuth 2.0 JWT directly instead of a Vault token. This "Agentic IAM" capability became GA in 2.1.0.
Secrets Engines¶
Secrets engines are plugins that store, generate, or encrypt data. Each one is mounted at its own path and is isolated from the others. The engine table is in Reference.
The diagram shows common engines and what each produces.
graph LR
CORE["Vault core / router"] --> KV["KV v2<br/>secret/"]
CORE --> TRANSIT["Transit<br/>transit/"]
CORE --> DB["Database<br/>database/"]
CORE --> PKI["PKI<br/>pki/"]
CORE --> AWS["AWS<br/>aws/"]
CORE --> SSH["SSH<br/>ssh/"]
CORE --> ID["Identity<br/>identity/"]
KV -->|"static secrets"| APP1["Applications"]
TRANSIT -->|"ciphertext / signatures"| APP2["Applications encrypting data"]
DB -->|"CREATE ROLE per lease"| DB_INST["PostgreSQL / MySQL / MSSQL"]
PKI -->|"X.509 certificates"| PKI_CLIENTS["TLS servers and clients"]
AWS -->|"IAM users / STS"| AWS_ACCT["AWS account"]
SSH -->|"signed SSH certs"| HOSTS["SSH hosts"]
ID -->|"OIDC identity tokens"| RP["Relying parties"]
Dynamic vs static secrets
Static secrets (KV) are stored and returned as written. Dynamic secrets (Database, AWS, Azure, GCP, SSH OTP) are created on request with a lease. When the lease expires or is revoked, Vault deletes the credential in the target system. No long-lived shared credential exists, and every credential maps to one requester in the audit log.
Leases and Dynamic Secrets¶
Every dynamic secret and every service token has a lease: an ID, a TTL, and a renewable flag. The expiration manager keeps an index of leases in storage and revokes them on expiry.
The state diagram shows a lease's lifecycle.
stateDiagram-v2
[*] --> Active: Engine creates credential
Active --> Active: Renew (up to max_ttl)
Active --> Revoked: TTL expires
Active --> Revoked: vault lease revoke / parent token revoked
Revoked --> [*]: Credential deleted in target system
Revoked --> Irrevocable: Target system unreachable after retries
Irrevocable --> [*]: Manual cleanup or force revoke
Why leases matter operationally:
- Leases are stored data. Millions of long-TTL leases slow down unseal and leader election, because the expiration manager loads them. Enterprise applies a default lease count quota of 300,000 on new 1.16+ clusters.
- Static secrets are not rotated by Vault. For KV, rotation is an external process. Database static roles and the Enterprise rotation manager rotate existing accounts on a schedule instead.
- Revocation is best-effort. If the target system is down, leases become irrevocable and must be cleaned up by hand (Enterprise 1.20 can auto-remove them with
remove_irrevocable_lease_after).
ACL Policies¶
Vault uses path-based ACL policies written in HCL or JSON. A policy lists paths and capabilities: create, read, update, patch, delete, list, sudo, and deny.
# Read-only access to one KV v2 path
path "secret/data/myapp/*" {
capabilities = ["read"]
}
# Request dynamic database credentials
path "database/creds/myapp-role" {
capabilities = ["read"]
}
Evaluation rules:
- Default deny. Without a matching rule, the request fails.
- Union of policies. A token with several policies gets the union of their capabilities, except that an explicit
denyon a path always wins. - Most specific path wins when several rules in the policy set match the same path.
- Templated policies can insert identity values, for example
{{identity.entity.name}}, to write one policy for many users. Since 2.0.1, a rendered template that produces wildcards or globs is rejected.
Enterprise adds two further layers. Sentinel endpoint-governing and role-governing policies add conditional logic (for example, source CIDR, time of day, MFA presence, or request field validation). Control groups require one or more authorized approvers before a request completes. Multi-party approval is a control-group feature, not a Sentinel one.
Request Processing Flow¶
The sequence shows how a single authenticated request moves through Vault.
sequenceDiagram
participant Client
participant Listener as TCP listener
participant Core as Vault core
participant Tokens as Token store
participant Audit as Audit broker
participant Engine as Secrets engine
participant Barrier as Encryption barrier
participant Storage as Storage backend
Client->>Listener: HTTPS request with X-Vault-Token
Listener->>Core: Parsed request
Core->>Tokens: Look up token, policies, TTL
Tokens-->>Core: Token entry
Core->>Core: Evaluate ACL (and Sentinel in Enterprise)
Core->>Audit: Log request to every audit device
alt Allowed
Core->>Engine: Route by mount path
Engine->>Barrier: Read or write
Barrier->>Storage: Ciphertext only
Storage-->>Barrier: Ciphertext
Barrier-->>Engine: Plaintext
Engine-->>Core: Response (and lease)
Core->>Audit: Log response
Core-->>Client: 200 with data
else Denied
Core->>Audit: Log denied request
Core-->>Client: 403 permission denied
end
If no enabled audit device can record a request, Vault refuses the request. This is deliberate: Vault prefers unavailability over unaudited access.
Audit Devices¶
Audit devices (file, syslog, socket) record every request and response. Values that could be secret are HMAC-SHA256 hashed with a per-device salt by default (log_raw=false), so the log shows that a specific value was used without revealing it. The sys/audit-hash endpoint computes the same hash to search logs for a known value.
Several devices can be enabled at once, and the audit broker writes to all of them. Enable at least two so one failing device does not block Vault.
Replication (Enterprise)¶
Vault Enterprise has two replication modes. Both use a primary/secondary model with a write-ahead-log stream over mutually authenticated TLS on the cluster port.
| Mode | Purpose | Behavior |
|---|---|---|
| DR (disaster recovery) | Business continuity | Replicates all data, including tokens and leases. The secondary serves no client requests until it is promoted. |
| Performance | Latency and scale | Replicates configuration and secrets, but not tokens or leases. Secondaries serve reads and issue their own tokens and leases; writes to replicated data are forwarded to the primary. Paths can be filtered per secondary. |
Since Vault 2.0, DR secondaries accept root tokens from the primary, and event notifications can be subscribed to from secondaries.
Deployment Topology¶
A typical self-managed production deployment uses one Integrated Storage cluster across three availability zones, auto-unseal through a cloud KMS, and a load balancer that sends traffic to the active node.
graph TB
CLIENTS["Clients, Vault Agent, VSO"] --> LB["Load balancer<br/>health check /v1/sys/health"]
subgraph AZ1["Zone A"]
N1["vault-0 (active, Raft leader)"]
end
subgraph AZ2["Zone B"]
N2["vault-1 (standby, voter)"]
N4["vault-3 (standby, voter)"]
end
subgraph AZ3["Zone C"]
N3["vault-2 (standby, voter)"]
N5["vault-4 (standby, voter)"]
end
LB --> N1
LB -.->|"forwarded to active"| N2
N1 <-->|"Raft :8201"| N2
N1 <-->|"Raft :8201"| N3
N1 <-->|"Raft :8201"| N4
N1 <-->|"Raft :8201"| N5
N1 --> KMSK["Cloud KMS key (auto-unseal)"]
N1 --> SIEM["Audit devices to SIEM"]
Design choices behind this layout:
- 5 voters across 3 zones survive the loss of one zone. 3 voters are fine for smaller setups.
- Auto-unseal lets nodes restart unattended.
- Two audit devices avoid a single point of failure in logging.
- Dedicated hosts reduce the chance that another process can read Vault's memory.
Hardware guidance from HashiCorp's reference architecture is in Reference.
Performance and Scaling¶
Vault's throughput is bounded by different resources depending on the workload:
- Writes (KV writes, token creation, dynamic secrets, leases) go through Raft on the active node and are limited by disk latency and IOPS. This is why the reference architecture asks for 3000 to 10000+ IOPS on SSDs.
- Reads in Community Edition are all served by the active node. Enterprise performance standbys and performance replication scale reads horizontally.
- Crypto-heavy operations (PKI signing with RSA, Transit on large payloads) are CPU-bound.
- Leases and tokens add storage writes and expiration-manager work. Batch tokens and shorter TTLs cut this load.
An indicative (unsourced) throughput table is kept, with a warning, in Reference. Use vault-benchmark against your own hardware for real numbers.
Security Model¶
Threat Model¶
| Threat vector | Impact | Mitigation |
|---|---|---|
| Storage backend compromise | Attacker reads encrypted data only | Barrier encryption (AES-256-GCM); storage is untrusted |
| Memory dump or swap | Keys and secrets written to disk | mlock (non-Raft), or no swap / encrypted swap with disable_mlock = true; dedicated hosts |
| Unseal key exposure | Full decryption of storage | Shamir K-of-N with PGP-encrypted shares, or auto-unseal with a tightly scoped KMS key |
| Root token compromise | Unrestricted access | Revoke after bootstrap; regenerate only for break-glass |
| Token theft | Unauthorized secret access | Short TTLs, token_bound_cidrs, response wrapping, batch tokens |
| Policy misconfiguration | Over-broad access | Default deny, deny rules, policy review, Sentinel (Enterprise) |
| Audit log tampering | Loss of forensic trail | Multiple devices, append-only or remote storage, SIEM forwarding |
| Network interception | Credential theft | TLS on every listener; mutual TLS or cert auth where possible |
| Denial of service on unauthenticated endpoints | Blocked rekey or root generation | Vault 2.0 requires a token on sys/rekey, sys/generate-root, and DR operation-token endpoints |
Why Root Tokens Should Not Persist¶
The root token bypasses policy entirely. Vault's intended model is that it exists only during bootstrap (enabling auth methods, writing policies, enabling audit devices) and is then revoked. vault operator generate-root can create a new one later with a quorum of unseal or recovery key holders. Since Vault 2.0 that endpoint also requires a Vault token unless enable_unauthenticated_access includes generate-root. Any use of a root token in audit logs should raise an alert.
Namespaces (Enterprise)¶
Namespaces give each tenant its own auth methods, secrets engines, policies, identities, and tokens, with delegated administration. They are hierarchical: a policy in a parent namespace can grant access to paths in child namespaces, but a child cannot reach its parent. Tokens are created in a namespace and are valid in that namespace and its children only. HCP Vault Dedicated reserves the root namespace for the platform and gives customers the admin namespace.
Transit Encryption¶
Transit is encryption as a service. Applications send plaintext and receive ciphertext such as vault:v1:.... Vault stores no application data.
- Key separation: each named key is independent.
- Key versioning:
rotateadds a version. New encryptions use the latest version, old ciphertext still decrypts, andrewrapupgrades ciphertext without exposing plaintext.min_decryption_versionretires old versions. - Export control: keys are not exportable unless created with
exportable=true, and that choice cannot be reversed. - Convergent encryption (with derived keys) makes the same plaintext produce the same ciphertext for a given context. This allows lookups on encrypted fields but leaks equality, so use it only where that is acceptable.
- Envelope encryption (2.0): Vault protects data encryption keys while the application encrypts large data locally.
TLS¶
Vault assumes TLS on every listener. Mutual TLS on the listener (tls_require_and_verify_client_cert) proves the client host, while the cert auth method maps client certificates to Vault identities. Vault's own PKI engine can issue the listener certificate, but the CA that signs Vault's certificate should not depend on the same Vault being up.
Licensing, Ownership, and the OpenBao Fork¶
- August 2023: HashiCorp moved its products, including Vault, from MPL 2.0 to the Business Source License 1.1. Vault 1.15.0 and later are BSL. Each release converts to MPL 2.0 four years after publication. Production use is allowed unless you offer Vault to third parties as a competing hosted or embedded paid product.
- December 2023: the Linux Foundation announced OpenBao, a community fork of the last MPL-licensed Vault code, under LF Edge. OpenBao 2.0.0 shipped on 2024-07-16. In May 2025 OpenBao moved to the OpenSSF as a sandbox project.
- February 27, 2025: IBM completed its acquisition of HashiCorp. The Vault
LICENSEfile now names IBM as licensor. - April 14, 2026: Vault 2.0.0 became GA. The jump from 1.21 to 2.0 marks the move to IBM versioning and the IBM Support Cycle-2 lifecycle, not an API break. Vault 2.0 still has breaking changes to review (see How-to Guides).
OpenBao has since diverged. It re-added namespaces (2.3.1, API-compatible with Vault Enterprise), added horizontal read scaling, declarative self-initialization, KMS plugins for auto-unseal, and in 2.7.0 (2026-09-23) external keys for PKI and Transit, ML-DSA post-quantum signatures, a PebbleDB backend, and control groups. It does not implement Vault Enterprise replication or Sentinel. Migration from Vault is simplest from pre-1.15 versions; later Vault storage formats and Enterprise-only features may not carry over. See Reference for a side-by-side table.